多线程处理列表数据
我有超过 300,000 项的清单。 我当前正在对列表执行的操作是验证地址并将原始地址和更正后的地址写入指定的文件。 我想做的是将列表均匀地分配给给定数量的线程,并同时对它们进行处理。 谁能帮我举一个例子来说明我如何去做这样的事情?
谢谢
I have a list of 300,000 + items.
What i am currently doing with the list is validating the Address and writing both the original address and corrected address to a specified file.
What i'd like to do is split the list evenly among a given number of threads and do processes on them concurrently.
Can anyone help me with an example on how i can go about doing something like this?
Thanks
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
如果您使用 2.0 并且该列表仅以只读方式使用(在此处理发生时不会更改),那么您可以简单地划分索引。例如 ...
If you're working in 2.0 and the list is only being used in a read only fashion (not being changed while this processing is occuring) then you can simply divide up the indexes. For instance ...
从概念上讲,它相当简单,给出几个假设:
加工。
基本上,您的算法是这样的:
一个。处理项目
b.锁定对输出文件的访问
c.写出原始地址和更正后的地址
d.解锁对输出文件的访问
Conceptually, it's fairly simple, given a couple of assumptions:
processing.
Basically, your algorithm is this:
a. Process an item
b. Lock access to the output file
c. Write the original and corrected address
d. Unlock access to the output file