大规模分布式系统中的日志文件
我在网格和 HPC 领域做了很多工作，对于分布在数百（或在某些情况下数千）服务器上的系统，我们面临的最大挑战之一是分析日志文件。
目前，日志文件本地写入每个刀片上的磁盘，但我们也可以考虑使用 UDP Appender 等发布日志信息并集中收集。
鉴于目标是能够尽可能实时地发现问题，我们应该做什么？
I do a lot of work in the grid and HPC space and one of the biggest challenges we have with a system distributed across hundreds (or in some case thousands) of servers is analysing the log files.
Currently log files are written locally to the disk on each blade but we could also consider publishing logging information using for example a UDP Appender and collect it centally.
Given that the objective is to be able to identify problems in as close to real time as possible, what should we do?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
你在使用阿帕奇吗？ 如果是这样，您可以看看 mod_log_spread 虽然您的基础设施可能太大了使其可维护。 另一种选择是查看“广播”或“多播”日志消息，并让专用的日志服务器订阅这些源并整理它们
Are you using Apache? If so you could have a look at mod_log_spread Though you may have too big an infrastructure to make it maintainable. The other option is to look at "broadcasting" or "multicasting" your log messages and having dedicated logging servers subscribing to those feeds and collating them
首先，使用 NTP 同步系统中的所有时钟。
其次，如果您在单个位置收集日志（例如您提到的 UDP 附加程序），请确保日志有足够的信息来提供实际帮助。 我至少会包括生成日志的服务器、发生的时间和消息。 如果存在任何类型的事务 ID 或作业 ID 类型概念，也请包括在内。
既然您提到了 UDP Appender，我猜测您正在使用 log4j （或其兄弟姐妹之一）。 Log4j 有一个 MDC 类，允许通过处理线程传递额外信息。 它可以帮助收集一些额外的信息并将其传递。
First, synchronize all clocks in the system using NTP.
Second, if you are collecting the logs in a single location (like the UDP appender you mention) make sure the logs have enough information to actually help. I would include at least the server that generated the log, the time it happened, and the message. If there is any sort of transaction id, or job id type concept, include that also.
Since you mentioned a UDP Appender I am guessing you are using log4j (or one of it's siblings). Log4j has an MDC class that allows extra information to be passed along through a processing thread. it can help collect some of the extra information and pass it along.