We have successfully used ZeroMQ for the logs of a distributed application scenario like the yours. It's very reliable and incredibly fast. We moved to ZeroMQ after a not-so-successful implementation with Spread. In our setup a single ZeroMQ server is able to handle more than 70 different logs from a medium to high busy distributed application. It receives data from LAN and via the Internet.
There're several alternatives recently. Notably, Scribe is not maintained any more. Facebook developed its successor called Caligraphus, and it's not open-sourced. Here's a list of alternatives.
syslog: installed at all Linux distributions
Fluentd: C+Ruby-based lightweight logger, which handles logs as JSON stream
Flume: developed at Cloudera, written in Java and works well with Hadoop ecosystems
Apache Kafka: developed at LinkedIn, pull-based architecture
Scribe: open-sourced by Facebook, but not maintained anymore
The other examples might be great, but I've had good luck with Syslog-NG. It is extremely flexible and configurable; although it's pretty easy to pick it up and do something useful with quickly.
Syslog is good if you intend to focus only on infrastructure logs (e.g. on system level). I heard that KIWI Syslog Server is a good one, though didn't try it myself. On the other hand, if you want to log an application related stuff, a syslog is perhaps not the best option for this. In case you use apache logging services (log4j, log4xxx and the rest), then logFaces would be a good solution as it's built particularly for aggregating multiple applications in one place. Works with both TCP or UDP connections and has decent log viewer and database integration.
Reviewed all the alternatives recommended in this thread. Looked for something python powered. Googled more and found sentry https://getsentry.com/welcome/ Open source, well-documented. Should be reliable for commercial as there is a business based on top this.
发布评论
评论（6）
我们已成功使用 ZeroMQ 来记录像您这样的分布式应用程序场景的日志。它非常可靠并且速度快得令人难以置信。在使用 Spread 实施不太成功之后，我们转向了 ZeroMQ。在我们的设置中，单个 ZeroMQ 服务器能够处理从中等到高度繁忙的分布式应用程序中的 70 多种不同日志。它从 LAN 和互联网接收数据。
如果您需要详细的队列服务器比较，请查看 Second Life wiki 中的此页面。
希望有帮助！
We have successfully used ZeroMQ for the logs of a distributed application scenario like the yours. It's very reliable and incredibly fast. We moved to ZeroMQ after a not-so-successful implementation with Spread. In our setup a single ZeroMQ server is able to handle more than 70 different logs from a medium to high busy distributed application. It receives data from LAN and via the Internet.
If you need a detailed queue server comparison, look at this page from the Second Life wiki.
Hope it helps!
最近有几个替代方案。值得注意的是，Scribe 不再被维护。 Facebook 开发了其继任者 Caligraphus，但它不是开源的。这是替代方案列表。
免责声明：我是 Fluentd 项目的提交者。
There're several alternatives recently. Notably, Scribe is not maintained any more. Facebook developed its successor called Caligraphus, and it's not open-sourced. Here's a list of alternatives.
Disclaimer: I'm a committer of Fluentd project.
其他示例可能很棒，但我在 Syslog-NG 方面运气不错。它非常灵活且可配置；尽管拿起它并快速做一些有用的事情非常容易。
The other examples might be great, but I've had good luck with Syslog-NG. It is extremely flexible and configurable; although it's pretty easy to pick it up and do something useful with quickly.
如果您打算只关注基础设施日志（例如系统级别），Syslog 是很好的选择。我听说 KIWI Syslog Server 是一个不错的服务器，但我自己没有尝试过。另一方面，如果您想记录与应用程序相关的内容，系统日志可能不是最佳选择。如果您使用 apache 日志记录服务（log4j、log4xxx 等），那么 logFaces 会是一个不错的选择解决方案，因为它是专门为将多个应用程序聚合到一个地方而构建的。适用于 TCP 或 UDP 连接，并具有良好的日志查看器和数据库集成。
披露：我是该产品的作者。
Syslog is good if you intend to focus only on infrastructure logs (e.g. on system level). I heard that KIWI Syslog Server is a good one, though didn't try it myself. On the other hand, if you want to log an application related stuff, a syslog is perhaps not the best option for this. In case you use apache logging services (log4j, log4xxx and the rest), then logFaces would be a good solution as it's built particularly for aggregating multiple applications in one place. Works with both TCP or UDP connections and has decent log viewer and database integration.
Disclosure: I am the author of this product.
您也可以考虑使用 SNMP 警报。
You might also consider using SNMP alerts.
查看了该线程中推荐的所有替代方案。寻找Python驱动的东西。谷歌搜索更多，发现sentry https://getsentry.com/welcome/开源，文档齐全。对于商业来说应该是可靠的，因为有一个基于此的业务。
Reviewed all the alternatives recommended in this thread. Looked for something python powered. Googled more and found sentry https://getsentry.com/welcome/ Open source, well-documented. Should be reliable for commercial as there is a business based on top this.