java中同步多个线程写入同一个文件
当然，有一种明显的使用“同步”的方式。 但我正在创建一个设计用于在多个内核上运行的系统 并在同一毫秒内多次写入该文件。 所以我相信使用同步会严重损害性能。 我正在考虑使用java的Pipe类（但不确定它是否有帮助） 或者让每个线程写入不同的文件，并有一个额外的线程收集 这些文字，创造了最终的结果。 我应该提一下，写作的顺序并不重要，它是有时间戳的 无论如何，在纳米时间里。 这两个更好的主意是什么？还有其他建议吗？ 谢谢。
Of course there is the obvious way of using "synchronized".
But I'm creating a system designed for running on several cores
and writing to that file various times at the same milisecond.
So I believe that using synchronize will hurt performance badly.
I was thinking of using the Pipe class of java (but not sure if it will help)
or having each thread write to a different file and an additional thread collecting
those writings, creating the final result.
I should mention that the order of the writings isn't important and it is timestamped
in nanotime anyway.
What is the better idea of those two? have any other suggestions?
thanks.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
使用某种同步（例如单个互斥体）很容易实现。
如果我有足够的 RAM，我会为每个日志生产者线程创建某种队列，并创建一个日志消费者线程以循环方式从所有队列中读取数据（或类似的方式） ）。
Using some sort of synchronization (eg. single mutexes) is quite easy to implement.
If I had enough RAM I would create a queue of some sort for each log-producer thread, and a log-consumer thread to read from all the queues in a round-robin fashion (or something like that).
不是对您问题的直接回答，但 logback 项目内置了同步设施，用于写入来自不同线程的同一个文件，因此如果它适合您的需要，您可以尝试使用它，或者至少看一下它的源代码。由于它是为了速度而构建的，我很确定它的实现不会被视为理所当然。
Not a direct answer to your question, but the logback project has synchronization facilities built into it for writing to the same file from different threads, so you might try to use it if it suits your needs, or at least take a look at it's source code. Since it's built for speed, I'm pretty sure the implementation isn't to be taken for granted.
您的担忧是正确的，您将无法让所有线程写入同一个文件而不出现性能问题。
当我遇到这个问题时（编写自己的日志记录，早在 Log4j 之前），我在内存中创建了两个固定大小的缓冲区，并将所有生产者线程写入一个缓冲区，而专用的消费者线程从另一个缓冲区读取并写入文件。这样，编写器线程必须仅在获取和增加缓冲区索引以及交换缓冲区时进行同步，并且仅在当前缓冲区已满时才阻塞。它占用大量内存，但速度很快。
对于其他想法，您可以查看 Log4j 和 Logback 等记录器如何工作，他们必须解决这个问题。
You are right to be concerned, you are not going to be able to have all the threads write to the same file without a performance problem.
When I had this problem (writing my own logging, way back before Log4j) I created two fixed-size buffers in memory and had all the producer threads write to one buffer while a dedicated consumer thread read from the other buffer and wrote to a file. That way the writer threads had to synchronize only on getting and incrementing the index to the buffer and when the buffers were being swapped, and it only blocked when the current buffer was full. It was memory-intensive but fast.
For other ideas you could check out how loggers like Log4j and Logback work, they would have had to solve this problem.
尝试使用JMS。在不同计算机上运行的所有进程可能会发送 JMS 消息，而不是写入文件。仅创建一个队列接收器来接收消息并将其写入文件。 Log4J 已经具有此功能：请参阅 JMSAppender。
Try to use JMS. All your processes running on different machines may send JMS message instead of writing to file. Create only one queue receiver that receives messages and writes them to file. Log4J already has this functionality: see JMSAppender.