进程间同步消息队列
我正在尝试实现一个具有生产者和 N (N >= 1) 个工作人员的程序。他们使用消息队列进行通信。 这个想法是生产者向队列发送“任务”。工作人员执行 msgrcv() 调用来获取任务并执行一些代码。工作人员完成任务后，会将计算结果发送到队列。生产者将收到此消息并保存结果。
我正在使用 POSIX 消息队列，生产者和工作人员同时工作。
该程序背后的问题是存在一种损害通信的场景。 每条消息的大小约为 5000 字节。在 UNIX 系统中，最大队列大小约为 16000 字节，情况就是如此。
场景是：队列中有3个任务（5000*3 = 15000字节）。一些工作人员从队列中获取一条消息（现在队列有 10000 字节）。工作线程开始执行任务，并且由于工作线程必须在每个任务中处理的字节量，生产者向队列发送另一条消息（队列现在已满）。 现在，任务完成后，工作人员尝试将结果发送到队列并被阻塞（队列已满）。生产者尝试向队列发送另一个任务，但也被阻塞。
如果我只用一名工作人员运行这个程序，这种情况发生的可能性相当大。
有人有办法避免这种情况吗？
I'm trying to implement a program that has a producer and N (N >= 1) workers. They communicate using a message queue.
The idea is that the producer sends to the queue "tasks". The workers do a msgrcv() call to get a task and execute some code. After the worker accomplish the task it sends to the queue the result of the computation. The producer will get this message and save the results.
I'm using POSIX message queues and the producer and the workers work concurrently.
The problem behind this program is that exists a scenario that compromises the communication.
Each message has a size of ~5000 bytes. The maximum queue size is ~16000 bytes in UNIX Systems, which is the case.
The scenario is: There are 3 tasks in the queue (5000*3 = 15000 bytes). Some worker get one message from the queue (now the queue has 10000 bytes). The worker starts executing the task and, due to the amount of bytes that a worker has to process in each task, the producer sends to the queue another message (the queue is now full).
Now after the task is complete the worker tries to send the result to the queue and becomes blocked (the queue is full). The producer tries to send another task to the queue and becomes blocked too.
If I run this program with only one worker this scenario have a considerable probability to occur.
Does anyone have an idea to avoid this situation?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
如果您无法更改队列大小、要使用的队列数量或使用不同的排队 API，那么排队较少的数据又如何呢？
您可以将实际数据放置在共享内存对象或临时文件中。然后，您不必将数据放入消息中，而是将文件名或共享内存对象名称以及可能的偏移量放入消息中。生产者进程可以在收到结果后对其进行清理。
它不一定是共享内存或临时文件，但其想法是将数据放在消息中以外的其他位置，并在消息中包含其他进程访问它所需的任何信息。
If you can't change the queue size, number of queues to use, or use a different queuing API, then what about queuing less data?
You could place the actual data in shared memory objects or temporary files. Then instead of putting the data in the message, you'd put a file name or shared memory object name and possibly an offset in the message instead. The producer process can then clean it up after receiving the results.
It doesn't necessarily have to be shared memory or temporary files, but the idea is to put the data somewhere other than in the message, and include in the message whatever information is needed for the other process to access it.
我要么为客户端使用第二个消息队列 ->服务器响应或限制（#sent - #received）为安全数字。
I'd either use a second message queue for the client -> server responses or limit (#sent - #received) to be a safe number.