如果两个 python 脚本想要写入同一个文件中会发生什么？
我有一个管道，它在某些时候将工作分成多个子流程，这些子流程并行地执行相同的操作。因此它们的输出应该进入同一个文件。
说所有这些进程都应该写入同一个文件是否风险太大？或者如果 python 发现该资源被占用，它会尝试重试吗？
I have a pipeline which at some point splits work into various sub-processes that do the same thing in parallel. Thus their output should go into the same file.
Is it too risky to say all of those processes should write into the same file? Or does python try and retry if it sees that this resource is occupied?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（5）
这是系统相关的。在 Windows 中，资源被锁定并且您会收到异常。在 Linux 中，您可以使用两个进程写入文件（写入的数据可能会混合）。
理想情况下，在这种情况下，您应该使用信号量来同步对共享资源的访问。
如果使用信号量对于您的需求来说太重，那么唯一的选择就是写入单独的文件...
编辑：正如稍后的帖子，资源管理器是处理并发编写器的另一种选择
This is system dependent. In Windows, the resource is locked and you get an exception. In Linux you can write the file with two processes (written data could be mixed)
Ideally in such cases you should use semaphores to synchronize access to shared resources.
If using semaphores is too heavy for your needs, then the only alternative is to write in separate files...
Edit: As pointed out by eye in a later post, a resource manager is another alternative to handle concurrent writers
一般来说，这不是一个好主意，需要非常小心才能正确。由于写入必须被序列化，因此它也可能对可伸缩性产生不利影响。
我建议写入单独的文件并合并（或者只是将它们保留为单独的文件）。
In general, this is not a good idea and will take a lot of care to get right. Since the writes will have to be serialized, it might also adversely affect scalability.
I'd recommend writing to separate files and merging (or just leaving them as separate files).
更好的解决方案是实现资源管理器（编写器）以避免两次打开同一文件。该管理器可以使用线程同步机制（
threading.Lock）来避免某些平台上的同时访问。A better solution is to implement a resource manager (writer) to avoid opening the same file twice. This manager could use threading synchronization mechanisms (
threading.Lock) to avoid simultaneous access on some platforms.让所有不同的进程将其输出写入队列，并让一个进程读取该队列并写入文件怎么样？
How about having all of the different processes write their output into a queue, and have a single process that reads that queue, and writes to the file?
使用 multiprocessing.Lock() 而不是 threading.Lock()。请注意！可能会降低并发处理能力，因为一个进程只是等待锁被释放
Use multiprocessing.Lock() instead of threading.Lock(). Just a word of caution! might slow down your concurrent processing ability because one process just waits for the lock to be released