C#：将作业分派给多处理器计算机上的工作进程
我有一个需要在 ca 上运行的通用检查。 1000 个对象。检查大约需要 3 秒。我们有一台带有 4 个处理器的服务器（并且我们的网络中还有其他多处理器服务器），因此我们想创建一个 exe/dll 来进行检查并将结果返回给“master”。
有谁知道这方面的框架，或者如何在 C# 中实现它？
具体来说：
- 在主进程和工作进程之间传输数据的最佳方式是什么？
- master 如何确保任何时候始终有 4 个进程正在运行，并且一旦工作进程完成就启动一个新的进程。
- 如何注册工作人员已完成并将其结果附加到列表中？
希望它足够清楚，但很乐意澄清。
A.
一些说明
* 在进程的调用和返回之外实际上没有进程间通信eg
ResultObject = WorkerProcess(HeresYourDataSonDoSomethingWithIt);* Initially one machine is a must, but thinking further down the line we will probably have some cases where we have 6000 objects to check and we would like to farm it out to multiple servers, so we would like to make the right design choice from the beginning, or at least not develop a solution for one server that would have to be completely rewritten for multiple. Thanks!
I have a generic check that needs to be run on ca. 1000 objects. The check takes about 3 seconds. We have a server with 4 processors (and we also have other multi-processor servers in our network) so we would like to create an exe / dll to do the checking and return the results to the "master".
Does anyone know of a framework for this, or how would one go about it in C#?
Specifically:
- What's the best way to transfer data between the master and the worker process?
- How would the master ensure that always 4 processes are running at any one time and as soon as a worker process is finished start a new one.
- How to register that the worker is finished and append it's results to a list?
Hope it's clear enough but happy to clarify.
A.
Some clarifications
* There is actually no interprocess communication outside the call and return of the process e.g.
ResultObject = WorkerProcess(HeresYourDataSonDoSomethingWithIt);* Initially one machine is a must, but thinking further down the line we will probably have some cases where we have 6000 objects to check and we would like to farm it out to multiple servers, so we would like to make the right design choice from the beginning, or at least not develop a solution for one server that would have to be completely rewritten for multiple.
Thanks!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
在 C# 进程之间传输数据的最佳方法是使用 .NET 远程处理。由于机器位于同一网络上，您可以使用二进制序列化和 IPC 通道，这应该非常快。
服务器上不需要 4 个进程，只需要多个线程。如果使用 .NET 4.0，请查看并行扩展如果您不熟悉多线程及其随之而来的陷阱，这可以极大地简化您的代码。
另外，这可能超出了您的需求，但您可以查看 DryadLINQ< /a> 来自微软研究实验室。
The best way to transfer data between processes in C# is using .NET remoting. As the machines are on the same network you can use binary serialisation and IPC channels which should be very fast.
You don't need 4 processes on the server machine, just multiple threads. If using .NET 4.0 check out the parallel extensions and that can greatly simplify your code, if you are not familiar with multi-threading and the pitfalls that come with it.
Also, this is probably overkill for your needs but you could look at DryadLINQ from the Microsoft Research Labs.
如果您必须使用单独的进程，您可能需要考虑定义所有任务并将它们放入队列中。 MSMQ 和 SQL Server 是两个选项。
然后让每个进程从队列中拉取一个任务，直到队列为空。
使用四个处理器，您可以简单地启动四个轮询进程，并将每个进程与特定的 CPU 关联。
If you must use separate processes you might want to consider defining all the tasks and put them in a queue. MSMQ and SQL Server are two options.
Then let each process pull a task from the queue until the queue is empty.
With four processors, you could simply spin up four polling processes and give each affinity to a particular CPU.
看一下内置的
ThreadPool。您可以为线程池排队“工作项”。 .NET 将管理线程，根据需要启动新线程，并确保“空闲”线程得到新的工作。此外，工作应该通过线程池在 CPU 之间平均分配。Take a look at the builtin
ThreadPool. You can queue a "work item" for the thread pool. .NET will manage the threads, start new threads as required and make sure the "idle" threads get new work. Also, work should be equally distributed among the CPUs by the thread pool.您在这里更关心的应该是可靠性。如果您使用工作进程而不是线程，特别是当这些进程在其他计算机上运行时，失败的几率会显着增加。这种故障可能非常难以处理，由于未处理的异常而导致进程死亡或有人被电源线绊倒，因此很难从进程中恢复。
避免致力于使代码显着复杂化但可能永远不会被实际使用的架构。当您使用线程时，这将容易容易得多。从长远来看，这仍然可以扩展，多核 CPU 是未来。
否则，WCF 是正确的技术。
Your bigger concern here should be reliability. If you use worker processes instead of threads, especially when those processes run on other machines, the odds for failure significantly increase. That kind of failure can be very difficult to deal with, it is hard to recover from a process dying on an unhandled exception or somebody tripping over a power cord.
Avoid committing to an architecture that significantly complicates the code but might never actually be used. This will be much easier to do when you use threads. That can still scale in the long run, many-core CPUs are the future.
WCF is otherwise the correct technology to use.