建筑配件架构和批量插入
我开发了汽车服务的自动化应用程序。我还启动了附件模块,但我无法想象应该如何构建数据模型架构。
我在文本文件中逐行获取了附件的数据(不是 cvs 或 ext..,因此,我按子字符串分割主题)。每个月,工厂都会将数据文件发送给服务人员。它包括价格、名称、代码等。价格每月更新。我认为bulkinsert(我确实这么做了)是将数据传输到SQL 的一个不错的选择,但这并不是我的问题的解决方案。我不想要重复的数据只是为了获得新的价格。我想只将价格插入到另一个表中,并在 Accessories - AccesoriesPrices 之间建立关系,但有时,可以将一些新配件添加到列表中,因此我必须检查 Accessories 表的每一行。而且,另一方面,我必须保留配件的数量、发票等。
顺便说一句,他们每个月发送 70,000 行。那么,有人可以帮助我吗? :)
谢谢。
I developed an automation application of a car service. I started accessories module yet but i cant imagine how should I build the datamodel schema.
I've got data of accessories in a text file, line by line (not a cvs or ext.., Because of that, i split theme by substring). Every month, the factory send the data file to the service. It includes the prices, the names, the codes and etc. Every month the prices are updated. I thought the bulkinsert (and i did) was a good choice to take the data to SQL, but it's not a solution to my problem. I dont want duplicate data just for having the new prices. I thought to insert only the prices to another table and build a relation between the Accessories - AccesoriesPrices but sometimes, some new accessories can be added to the list, so i have to check every line of Accessories table. And, the other side, i have to keep the quantity of the accessories, the invoices, etc.
By the way, they send 70,000 lines every month. So, anyone can help me? :)
Thanks.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
70,000 行并不是一个大文件。您必须自己解析该文件,并根据其中包含的数据发出普通的
insert
和update
语句。无需对这种大小的数据使用批量操作。处理此类问题的最常见方法是编写一个接受所有参数的简单 SQL 语句,然后执行如下操作:(
或者,您可以尝试重写此语句以使用
merge
T-SQL 语句)其中...
<存在condition>
表示您需要检查的内容以查看该项目是否已存在
是列的一组Column = value
语句您想要更新的
是要为新项插入数据的列集
是值集 的列列表对应于之前 然后循环文件中的每一行,将数据解析为参数值,然后使用这些参数运行上述 SQL 语句。
70,000 lines is not a large file. You'll have to parse this file yourself and issue ordinary
insert
andupdate
statements based upon the data contained therein. There's no need for using bulk operations for data of this size.The most common approach to something like this would be to write a simple SQL statement that accepts all of the parameters, then does something like this:
(Alternatively, you could try rewriting this statement to use the
merge
T-SQL statement)Where...
<exists condition>
represents whatever you would need to check to see if the item already exists<new values>
is the set ofColumn = value
statements for the columns you want to update<columns>
is the set of columns to insert data into for new items<values>
is the set of values that corresponds to the previous list of columnsYou would then loop over each line in your file, parsing the data into parameter values, then running the above SQL statement using those parameters.