使用 log4net 进行高速日志记录

发布于 2024-08-08 21:25:31 字数 329 浏览 5 评论 0原文

我对 log4net 中的高速日志记录感兴趣(每秒大约 10K 消息)。 为此,我考虑实现以下模块:

  1. 基于协议缓冲区的布局(IRawlayout) - 实现卓越的序列化性能
  2. 共享内存追加器和插件 - 减少日志记录应用程序和日志记录服务器之间的 IPC。

这是整合这些技术的方式吗?

我还考虑过在 log4net 中使用 ETW 但它们似乎太不同了优雅地融合在一起。

I am interested in very high speed logging within log4net(around 10K messages per second).
to that end i thought of implementing the following modules:

  1. protocol buffers based Layout (IRawlayout) - for superior serialization performance
  2. shared memory appender and Plugin - to reduce IPC between the logging application and the logging server.

is this the way to integrate those technologies?

i also considered using ETW within log4net but they seem to be too different to be elegantly intergrated.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

知足的幸福 2024-08-15 21:25:31

前段时间我调查了 log4net 性能并创建了一个 博客文章

您可以找到几个 log4net 异步转发附加程序:

  1. 异步 log4net 解决方案 1:BlockingCollection
  2. 异步log4net解决方案2:热插拔技术
  3. 异步log4net解决方案3:.NET 端口LMAX Disruptor

使用 RollingFileAppender 的异步 log4net 解决方案的性能结果

在此处输入图像描述
输入图片此处描述
输入图片这里的描述

以及我关于log4net性能的一些注释:

  • 基于BlockingCollection的AsyncForwardingAppenderBlocking解决方案性能非常糟糕,特别是对于几个日志记录生产者而言。
  • 基于热交换技术的AsyncForwardingAppenderHotSwap解决方案表现最佳!然而,它可能需要额外的内存,甚至在日志事件产生得非常快时导致 OutOfMemoryException!
  • 基于 LMAX Disruptor .NET 端口的 AsyncForwardingAppenderDisruptor 解决方案表现良好,但即使使用 NullAppender,它仍然与 log4j2 相距甚远。调查disruptor-net 中可能存在的性能问题或其在日志记录情况下的错误使用是一个很好的方法。

Some time ago I made investigation of log4net performance and created a blog post

You can find there several log4net async forwarding appenders:

  1. Asynchronous log4net solution 1: BlockingCollection
  2. Asynchronous log4net solution 2: Hot swap technique
  3. Asynchronous log4net solution 3: .NET port of LMAX Disruptor

Performance results of asynchronous log4net solutions with RollingFileAppender:

enter image description here
enter image description here
enter image description here

And my some notes about log4net performance:

  • AsyncForwardingAppenderBlocking solution based on BlockingCollection performs very bad, especially for several logging producers.
  • AsyncForwardingAppenderHotSwap solution based on Hot swap technique performs the best! However it might requires additional memory and even causes OutOfMemoryException when logging events are produced very fast!
  • AsyncForwardingAppenderDisruptor solution based on .NET port of LMAX Disruptor performs good, but it is still far from log4j2 even with NullAppender. It is a good point to investigate possible performance issues in disruptor-net or its wrong usage for the logging case.
旧时浪漫 2024-08-15 21:25:31

我曾经查看过 google protobuffer 并得出结论,它对日志记录的帮助并不像乍看起来那么大。日志记录涉及大量文本,无论如何,这些文本到处都是相同的文本。因此,protobuffers 的可移植性并不是一个优势。至于速度我也不确定,你仍然必须通过线路将相同的文本传输到服务器,要么打包到 protobuffer 数据包中,要么用 xml 标记。当然,如果您要记录文本信息,这是相关的。对于二进制日志记录来说,这可能是一件很酷的事情。

I once looked at google protobuffer and came to a conclusion that it won't be as great help with logging as it seems at first. Logging involves lots of text, which is same text everywhere anyway. So, portability of protobuffers is not an advantage. As to the speed I'm also not sure, you still have to transmit the same text over the wire to the server either packed into a protobuffer packet or tagged by xml. This, of course, is relevant if you are logging textual information. In case of binary logging it would probably be a cool thing to do though.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文