多线程处理列表数据

发布于 2024-08-10 08:34:12 字数 134 浏览 1 评论 0原文

我有超过 300,000 项的清单。 我当前正在对列表执行的操作是验证地址并将原始地址和更正后的地址写入指定的文件。 我想做的是将列表均匀地分配给给定数量的线程,并同时对它们进行处理。 谁能帮我举一个例子来说明我如何去做这样的事情?

谢谢

I have a list of 300,000 + items.
What i am currently doing with the list is validating the Address and writing both the original address and corrected address to a specified file.
What i'd like to do is split the list evenly among a given number of threads and do processes on them concurrently.
Can anyone help me with an example on how i can go about doing something like this?

Thanks

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

傻比既视感 2024-08-17 08:34:12

如果您使用 2.0 并且该列表仅以只读方式使用(在此处理发生时不会更改),那么您可以简单地划分索引。例如 ...

public void Process(List<Item> list, int threadCount) {
  int perThread = list.Count < threadCount ? list.Count : list.Count / threadCount;
  int index = 0;
  while ( index < list.Count ) {
     int start = index;
     int count = Math.Min(perThread,list.Count-start);
     WaitCallBack del = delegate(object state) { ProcessCore(list, start, count); };
     ThreadPool.QueueUserWorkItem(del);
     index += count;
  }
}

private void ProcessCore(List<Item> list, int startIndex, int count) {
  // Do work here
}

If you're working in 2.0 and the list is only being used in a read only fashion (not being changed while this processing is occuring) then you can simply divide up the indexes. For instance ...

public void Process(List<Item> list, int threadCount) {
  int perThread = list.Count < threadCount ? list.Count : list.Count / threadCount;
  int index = 0;
  while ( index < list.Count ) {
     int start = index;
     int count = Math.Min(perThread,list.Count-start);
     WaitCallBack del = delegate(object state) { ProcessCore(list, start, count); };
     ThreadPool.QueueUserWorkItem(del);
     index += count;
  }
}

private void ProcessCore(List<Item> list, int startIndex, int count) {
  // Do work here
}
你怎么这么可爱啊 2024-08-17 08:34:12

从概念上讲,它相当简单,给出几个假设:

  • 您不会在
    加工。
  • 您提前知道线程数。

基本上,您的算法是这样的:

  1. 根据线程数将列表拆分为大部分偶数部分。
  2. 为每个线程提供其部分的开始和结束索引以及输出文件。
  3. 在每个线程中:
    一个。处理项目
    b.锁定对输出文件的访问
    c.写出原始地址和更正后的地址
    d.解锁对输出文件的访问

Conceptually, it's fairly simple, given a couple of assumptions:

  • You're not changing the list during
    processing.
  • You know the number of threads ahead of time.

Basically, your algorithm is this:

  1. Split the list into mostly-even sections based on the number of threads.
  2. Give each thread the start and end index of its section, and the output file.
  3. In each thread:
    a. Process an item
    b. Lock access to the output file
    c. Write the original and corrected address
    d. Unlock access to the output file
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文