如果使用共享内存，进程相对于线程还有优势吗？
我编写了一个 Linux 应用程序，其中主“消费者”进程分叉出一堆“读取器”进程（~16），这些进程从磁盘读取数据并将其传递给“消费者”进行显示。数据通过在分叉之前使用套接字对创建的套接字传递。
我最初使用此进程边界编写它有 3 个原因：
消费者进程具有实时约束，因此我想避免消费者中的任何内存分配。读者可以自由地按照自己的意愿分配内存，甚至可以用另一种语言编写（例如，通过垃圾收集），并且这不会中断具有 FIFO 优先级的消费者。此外，读取器进程中的磁盘访问或其他 IO 不会中断消费者。我认为使用线程我无法获得这样的保证。
使用进程会阻止我这个程序员做一些愚蠢的事情，比如使用全局变量和破坏其他进程的内存。
我认为分叉一批工作人员将是利用多 CPU 架构的最佳方式，而且我认为使用进程而不是线程通常会更安全。
并非所有读取器都始终处于活动状态，但是，那些处于活动状态的读取器会不断发送大量数据。最近我在想，要通过避免与写入和读取套接字相关的内存复制来优化这一点，最好将数据直接读入共享内存缓冲区（shm_open/mmap）。然后，只有该共享内存的索引才会通过套接字传递，消费者将直接从中读取数据，然后再将其标记为可用。
无论如何，进程相对于线程的最大好处之一是避免破坏另一个线程的内存空间。您认为切换到共享内存会破坏我在此架构中拥有的任何优势吗？在这种情况下使用进程是否仍然有任何优势，或者我应该将我的应用程序切换为使用线程？
I have written a Linux application in which the main 'consumer' process forks off a bunch of 'reader' processes (~16) which read data from the disk and pass it to the 'consumer' for display. The data is passed over a socket which was created before the fork using socketpair.
I originally wrote it with this process boundary for 3 reasons:
The consumer process has real-time constraints, so I wanted to avoid any memory allocations in the consumer. The readers are free to allocate memory as they wish, or even be written in another language (e.g. with garbage collection), and this doesn't interrupt the consumer, which has FIFO priority. Also, disk access or other IO in the reader process won't interrupt the consumer. I figured that with threads I couldn't get such guarantees.
Using processes will stop me, the programmer, from doing stupid things like using global variables and clobbering other processes' memory.
I figured forking off a bunch of workers would be the best way to utilize multiple CPU architectures, and I figured using processes instead of threads would generally be safer.
Not all readers are always active, however, those that are active are constantly sending large amounts of data. Lately I was thinking that to optimize this by avoiding memory copies associated with writing and reading the socket, it would be nice to just read the data directly into a shared memory buffer (shm_open/mmap). Then only an index into this shared memory would be passed over the socket, and the consumer would read directly from it before marking it as available again.
Anyways, one of the biggest benefits of processes over threads is to avoid clobbering another thread's memory space. Do you think that switching to shared memory would destroy any advantages I have in this architecture? Is there still any advantage to using processes in this context, or should I just switch my application to using threads?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
您认为线程无法满足实时约束的假设是错误的。只要消费者线程本身不使用 malloc（这当然可能导致锁争用），读取器线程中的 IO 或内存分配就无法阻止消费者线程。如果你不确定。
至于使用进程而不是线程的其他原因（安全性、用不同语言编写读者的可能性等），这些都是完全合法的。只要您的消费者进程将共享内存缓冲区视为潜在不安全的外部数据，我认为您不会因为从管道切换到共享内存而失去任何显着的安全性。
Your assumption that you cannot meet your realtime constraints with threads is mistaken. IO or memory allocation in the reader threads cannot stall the consumer thread as long as the consumer thread is not using
mallocitself (which could of course lead to lock contention). I would recommend reading what POSIX has to say on the matter if you're unsure.As for the other reasons to use processes instead of threads (safety, possibility of writing the readers in a different language, etc.), these are perfectly legitimate. As long as your consumer process treats the shared memory buffer as potentially-unsafe external data, I don't think you lose any significant amount of safety by switching from pipes to shared memory.
是的，正是因为你所说的原因。最好保护每个进程的内存，并且只共享真正需要共享的内容。因此每个消费者都可以分配和使用其资源，而无需担心锁定。
至于任务之间的索引通信，应该注意的是，您可以使用共享内存中的一个区域，并使用互斥体进行访问，因为它可能比套接字通信轻。仅当存在争用时，文件描述符通信（套接字、管道、文件等）始终涉及内核、具有互斥锁或信号量的共享内存。
在多处理器环境中使用共享内存进行编程时需要注意的一点是避免对变量的错误依赖。当两个不相关的对象共享同一缓存行时，就会发生这种情况。当一个对象被修改时，它也会“弄脏”另一个对象，这意味着如果其他处理器访问另一个对象，它将触发 CPU 之间的缓存同步。这可能会导致缩放效果不佳。通过将对象与缓存行大小（通常为 64 字节，但不同架构可能有所不同）对齐，可以轻松避免这种情况。
Yes, exactly for the reason you told. It's better to have each processes memory protected and only share what is really necessary to share. So each consumer can allocate and use its resources without bothering with the locking.
As for your index communication between your task, it should be noted that you could then use an area in your shared memory for that and using mutex for the accesses, as it is likely less heavy than the socket communication. File descriptor communication (sockets, pipes, files etc) always involves the kernel, shared memory with mutex locks or semaphores only when there is contention.
One point to be aware of when programming with shared memory in a multiprocessor environment, is to avoid false dependencies on variables. This happens when two unrelated objects share the same cache line. When one is modified it "dirties" also the other, which means that if other processor access the other object it will trigger a cache synchronisation between the CPUs. This can lead to bad scaling. By aligning the objects to the cache line size (64 byte usually but can differ from architecture to architecture) one can easily avoid that.
根据我的经验，我遇到的用线程代替进程的主要原因是效率。
如果您的进程使用大量可以在多线程中共享的代码或非共享内存，那么您可以在高线程 CPU（例如每个 CPU 具有 64 个或更多线程的 SUN Sparc CPU）上获得大量性能。在这种情况下，CPU 缓存（尤其是代码缓存）在多线程进程中会更加高效（Sparc 上的缓存很小）。
如果您发现您的软件在具有更多 CPU 线程的新硬件上运行时运行速度并不快，那么您应该考虑多线程。否则，你为避免这种情况而提出的论点对我来说似乎很好。
我还没有在英特尔处理器上遇到这个问题，但将来当他们为每个 CPU 添加更多内核时，它可能会发生。
The main reason I met in my experience to replace processes by threads was efficiency.
If your processes are using a lot of code or unshared memory that could be shared in multithreading, then you could win a lot of performance on highly threaded CPUs like SUN Sparc CPUs having 64 or more threads per CPU. In this case, the CPU cache, especially for the code, will be much more efficient with multithreaded process (cache is small on Sparc).
If you see that your software is not running faster when running on new hardware with more CPU threads, then you should consider multi-threading. Otherwise, your arguments to avoid it seem good to me.
I did not meet this issue on Intel processors yet, but it could happen in the future when they add more cores per CPU.