通过 WCF 进行日志记录而不减慢速度
我们的应用程序中有一个每月运行一次的大型流程。此过程通常运行约 30 分钟,并生成约 342000 个日志事件。最近,我们使用 WCF 将日志记录更新为集中式模型,但现在遇到了性能问题。以前的解决方案大约需要 30 分钟即可完成,但采用新的日志记录后,现在需要 3 或 4 小时。问题似乎是因为应用程序实际上正在等待 WCF 请求完成才能继续执行。 WCF 方法已配置为 IsOneWay,我将客户端对该 WCF 方法的调用包装在不同的线程中,以尝试防止此类问题,但它似乎不起作用。我曾考虑过使用异步 WCF 调用,但在尝试其他方法之前我想我会在这里询问是否有更好的方法来处理这个问题。
We have a large process in our application that runs once a month. This process typically runs in about 30 minutes and generates 342000 or so log events. Recently we updated our logging to a centralized model using WCF and are now having difficulty with performance. Whereas the previous solution would complete in about 30 minutes, with the new logging, it now takes 3 or 4 hours. The problem it seems is because the application is actually waiting for the WCF request to complete before execution continues. The WCF method is already configured as IsOneWay and I wrapped the call on the client side to that WCF method in a different thread to try to prevent this type of problem but it doesn't seem to have worked. I have thought about using the async WCF calls but thought before I tried something else I would ask here to see if there is a better way to handle this.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(4)
30 分钟内记录了 342000 个日志事件,如果我计算正确的话,每秒会记录 190 个日志事件。我认为您的问题可能与 WCF 中的默认限制设置有关。即使您的方法设置为单向,根据您是否为每个记录的事件创建新代理,在创建代理、打开通道以及如果您使用的是基于 HTTP 的绑定,它将阻塞,直到服务接收到消息(基于 HTTP 的绑定在接收到消息时为单向方法调用发回 null 响应)。默认的 WCF 限制将服务端的并发实例限制为 10 个,这意味着一次只会处理 10 个请求,任何进一步的请求都将排队,因此将其与 HTTP 绑定配对,前 10 个请求之后的任何请求都会被处理。将阻塞客户端,直到它成为 10 个请求之一得到处理。在不知道您的服务如何配置(实例模式等)的情况下,很难说更多,但如果您使用每次调用实例,我建议设置
MaxConcurrentCalls
和将
设置为更高的值(默认值分别为 16 和 10)。ServiceBehavior
上的 MaxConcurrentInstances此外,为了构建其他人提到的聚合多个事件并一次提交所有事件的内容,我发现设置静态 Logger.LogEvent(eventData) 方法很有帮助。这样,在整个代码中使用起来就很简单,并且您可以在
LogEvent
方法中控制您希望日志记录在整个应用程序中的行为方式,例如配置一次应提交多少个事件。342000 log events in 30 minutes, if I did my math correctly, comes out to 190 log events per second. I think your problem may have to do with the default throttling settings in WCF. Even if your method is set to one-way, depending on if you're creating a new proxy for each logged event, calling the method will still block while the proxy is created, the channel is opened, and if you're using an HTTP-based binding, it will block until the message has been received by the service (an HTTP-based binding sends back a null response for a 1-way method call when the message is received). The default WCF throttling limits concurrent instances to 10 on the service side, which means only 10 requests will be handled at a time, and any further requests will get queued, so pair that with an HTTP binding, and anything after the first 10 requests are going to block at the client until it's one of the 10 requests getting handled. Without knowing how your services are configured (instance mode, etc.) it's hard to say more than that, but if you're using per-call instancing, I'd recommend setting
MaxConcurrentCalls
andMaxConcurrentInstances
on yourServiceBehavior
to something much higher (the defaults are 16 and 10, respectively).Also, to build on what others have mentioned about aggregating multiple events and submitting them all at once, I've found it helpful to setup a static
Logger.LogEvent(eventData)
method. That way it's simple to use throughout your code, and you can control in yourLogEvent
method how you want logging to behave throughout your application, such as configuring how many events should get submitted at a time.调用另一个进程或远程服务(即调用 WCF 服务)可能是应用程序中最昂贵的事情。重复 342,000 次简直就是疯狂!
如果您必须登录到集中式服务,则需要累积一批日志条目,然后,仅当内存中有 1000 条左右时,才将它们一次全部发送到该服务。这将为您带来合理的性能提升。
Making a call to another process or remote service (i.e. calling a WCF service) is about the most expensive thing you can do in an application. Doing it 342,000 times is just sheer insanity!
If you must log to a centralized service, you need to accumulate batches of log entries and then, only when you have say 1000 or so in memory, send them all to the service in one hit. This will give you a reasonable performance improvement.
log4net 有一个存在于调用线程上下文之外的缓冲系统,因此它不会'记录通话时请勿挂断电话。从许多 appender 配置示例 中应该可以清楚地看到它的用法 - 搜索术语
bufferSize
。它用在许多较慢的附加程序(例如远程处理、电子邮件)上,以保持源线程移动,而无需等待较慢的日志记录介质,并且还有一个通用的缓冲元附加程序,可以在任何其他“前面”使用附加器。我们在类似体积的系统中将其与 AdoNetAppender 一起使用,效果非常好。
log4net has a buffering system that exists outside the context of the calling thread, so it won't hold up your call while it logs. Its usage should be clear from the many appender config examples - search for the term
bufferSize
. It's used on many of the slower appenders (eg. remoting, email) to keep the source thread moving without waiting on the slower logging medium, and there is also a generic buffering meta-appender that may be used "in front of" any other appender.We use it with an AdoNetAppender in a system of similar volume and it works wonderfully.
总有传统的 syslog,有很多在 Windows 上运行的 syslog 守护进程。它被设计为比 WCF 更有效的集中日志记录方式,WCF 是为不太密集的操作而设计的,特别是在您不使用 tcpip WCF 配置的情况下。
换句话说,尝试一下这个——适合这项工作的正确工具。
There's always the traditional syslog there are plenty of syslog daemons that run on Windows. Its designed to be a more efficient way of centralised logging than WCF, which is designed for less intensive opertions, especially if you're not using the tcpip WCF configuration.
In other words, have a go with this - the correct tool for the job.