通过 MSMQ 到 WCF Web 服务的日志缓冲

发布于 2024-11-07 14:47:36 字数 2870 浏览 0 评论 0原文

我手头上有一个有趣的情况。几年来,我们已经拥有一个 WCF 服务,该服务在我们网络上的 IIS 机器上运行,用于记录日志。应用程序使用 basicHttpBinding 发送日志消息并将其记录到数据库中。来自所有其他应用程序的大多数日志记录仅限于每个操作的几十个日志。最近,我们移植了另一个应用程序来使用此日志记录服务。该应用程序的日志记录比我们的日志记录服务的任何其他客户端都更加积极。它有一个操作,在其过程中可以记录超过 100,000 条消息(这是通过 CSV 文件导入 100,000 条记录)。

在试图找到一种方法来使其更有效地运行时,建议我尝试 MSMQ 将日志请求排队并将它们转发到服务,而不是让应用程序直接与日志记录服务通信(请求是< /em> 出于性能目的在单独的线程中发送)。这样,在写入日志时应用程序就不会被阻止。我已经在本地计算机上实现了这个 MSMQ 解决方案的概念验证,但由于我以前从未使用过 MSMQ,而且我现在才了解它在技术上的地位,所以我发现自己有很多问题和很少的答案。我希望有人可以向我指出一些有关 MSMQ 如何工作的简明信息,以便我可以更好地调试我编写的这个概念证明。

实施 MSMQ 后我看到的是:

  • 我的应用程序将日志发送到 MSMQ 几乎立即返回 (而不是等待日志 致力于)这就是我想要的。
  • 即使我发送了 1000 条日志到 MSMQ应用少于30个 秒,需要几分钟 所有 1000 个日志才能到达 数据库。看起来好像我的 队列有某种限制 启用它阻碍的地方 记录请求,但我不知道如何/在哪里检查。
  • 如果我在从 MSMQ 服务接收日志的服务器上使用 net.tcp,我会收到大量服务超时错误,并且只有 90%(左右)的日志会发送到数据库,但如果我使用 basicHttpBinding,它会可靠地工作。

这是我的 MSMQ 服务中的代码:

public void QueueLog(Log log)
{
    try
    {
        ServiceClient writeLog = getLoggingService();

        string exceptionText = log.Exception == null ? string.Empty : log.Exception.ToString();
        writeLog.WriteCompatibleLog(log.LoggerName, (LoggingService.Logging.LogType)log.LogType, exceptionText, log.Message, log.ApplicationName, Utility.GetAssemblyVersion(Assembly.GetCallingAssembly()),
            Utility.GetFirstIPAddress(), Utility.GetHostName(), log.Category);
    }
    catch (Exception ex)
    {
        Debug.WriteLine("LoggingServiceWrapper.DotNet Error\n\n" + ex.GetBaseException().Message + "\n\n" + ex.GetBaseException().StackTrace);
    }
}

private ServiceClient getLoggingService()
{
    return new ServiceClient(new BasicHttpBinding(), new EndpointAddress("http://MyServer/Service.svc"));

}

这是我创建 MSMQ 服务的代码

if (!MessageQueue.Exists(Properties.Settings.Default.QueueName))
{
    MessageQueue.Create(Properties.Settings.Default.QueueName, true);
}

ServiceHost serviceHost = new ServiceHost(typeof(LoggingServiceQueue.LoggingServiceQueue), new Uri(Properties.Settings.Default.Address));
{
    serviceHost.Open();

    Console.ReadLine();

    serviceHost.Close();
}

这是我用来调用 MSMQ 服务的应用程序中的代码:

LoggingServiceQueueClient client = new LoggingServiceQueueClient(new NetMsmqBinding(NetMsmqSecurityMode.None), new EndpointAddress("net.msmq://localhost/private/Logging"));

client.QueueLog(new LoggingServiceQueue.Log
{
    LoggerName = loggerName,
    LogType = (LoggingServiceQueue.LogType)logType,
    ApplicationName = applicationName,
    Category = category,
    Exception = exception,
    Message = message,
    SourceHostName = Utility.GetHostName(),
    SourceIp = Utility.GetFirstIPAddress(),
    Version = Utility.GetAssemblyVersion(callingAssembly)
});

我感谢有人可以提供的任何帮助。我不太清楚使用 MSMQ 可以使用哪些功能。我通过谷歌做了大约 6 个小时的研究。我发现的大多数文档都很旧(2007 年)并且读起来很困难。也许他们希望我对这个主题有一些基础知识。

I have an interesting situation on my hands. For a few years now we've had a WCF service that runs on an IIS box on our network that we use for logging. Applications use basicHttpBinding to send log messages and it logs them to a database. Most of the logging from all other applications is limited to a few dozen logs per operation. Recently we've ported another application to use this logging service. This application logs quite a bit more aggressively than any of the other clients for our logging service. It has an operation that during its course could log over 100,000 messages (this is for importing 100,000 records via CSV file).

In trying to find a way to make this run more efficiently it was suggested that I try MSMQ to queue the log requests and forward them on to the service instead of having the application communicate directly with the logging service (the request was being sent in a separate thread for performance purposes). In this way the application isn't blocked while the log is being written. I have implemented a proof of concept for this MSMQ solution on my local machine but as I have never used MSMQ before, and as I am only now learning what its place is technologically, I'm finding myself with a lot of questions and few answers. I'm hoping someone can point me to some concise information on how MSMQ works so that I can better debug this proof of concept I have written.

What I'm seeing after implementing MSMQ:

  • My application that sends the logs to
    MSMQ returns almost immediately
    (rather than waiting for the log to
    be committed) which is what I want.
  • Even though I sent 1000 logs to the
    MSMQ application in fewer than 30
    seconds, it takes several minutes for
    all 1000 logs to make it to the
    database. It appears as though my
    queue has some sort of throttling
    enabled where it is holding back the
    log requests but I don't know how / where to check that.
  • If I use net.tcp on the server receiving the logs from the MSMQ service I get a ton of service timeout errors and only 90% (or so) of the logs make it to the database but if I use basicHttpBinding it works reliably.

Here is the code in my MSMQ service:

public void QueueLog(Log log)
{
    try
    {
        ServiceClient writeLog = getLoggingService();

        string exceptionText = log.Exception == null ? string.Empty : log.Exception.ToString();
        writeLog.WriteCompatibleLog(log.LoggerName, (LoggingService.Logging.LogType)log.LogType, exceptionText, log.Message, log.ApplicationName, Utility.GetAssemblyVersion(Assembly.GetCallingAssembly()),
            Utility.GetFirstIPAddress(), Utility.GetHostName(), log.Category);
    }
    catch (Exception ex)
    {
        Debug.WriteLine("LoggingServiceWrapper.DotNet Error\n\n" + ex.GetBaseException().Message + "\n\n" + ex.GetBaseException().StackTrace);
    }
}

private ServiceClient getLoggingService()
{
    return new ServiceClient(new BasicHttpBinding(), new EndpointAddress("http://MyServer/Service.svc"));

}

Here is the code of me creating the MSMQ service

if (!MessageQueue.Exists(Properties.Settings.Default.QueueName))
{
    MessageQueue.Create(Properties.Settings.Default.QueueName, true);
}

ServiceHost serviceHost = new ServiceHost(typeof(LoggingServiceQueue.LoggingServiceQueue), new Uri(Properties.Settings.Default.Address));
{
    serviceHost.Open();

    Console.ReadLine();

    serviceHost.Close();
}

Here is the code in the application I use to call the MSMQ service:

LoggingServiceQueueClient client = new LoggingServiceQueueClient(new NetMsmqBinding(NetMsmqSecurityMode.None), new EndpointAddress("net.msmq://localhost/private/Logging"));

client.QueueLog(new LoggingServiceQueue.Log
{
    LoggerName = loggerName,
    LogType = (LoggingServiceQueue.LogType)logType,
    ApplicationName = applicationName,
    Category = category,
    Exception = exception,
    Message = message,
    SourceHostName = Utility.GetHostName(),
    SourceIp = Utility.GetFirstIPAddress(),
    Version = Utility.GetAssemblyVersion(callingAssembly)
});

I appreciate any help someone can provide. I'm a little lost on exactly what set of features are available to me using MSMQ. I have done 6 or so hours of research via google. Most docs I have found are pretty old (2007) and are a tough read. Perhaps they are expecting me to have some baser level of knowledge on the subject.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

一笑百媚生 2024-11-14 14:47:36

您所看到的(如果您在 WCF 上使用 net.msmq 绑定)就是设计的结果。

MSMQ 作为存储转发工具,会将消息存储在发送客户端和接收服务器的队列中,直到网络或处理带宽可用。这样做的作用是通过使进程异步来减轻客户端的负载(正如您所观察到的)。

您可以使用性能监视器查看队列的状态,出站队列(在客户端)和服务器队列中的消息数量会实时写入性能计数器,并且可以查看和/或记录。

如果性能不是最佳的,您可以更改 WCF 限制设置,以允许服务行为中有更多并发连接。

例如

<serviceThrottling
         maxConcurrentCalls="20"
         maxConcurrentSessions="20"
         maxConcurrentInstances="20"
       />

What you're seeing (if you're using net.msmq binding on your WCF) is what is designed to happen.

MSMQ, as a store-and-forward facility, will store messages in a queue at the sending client and the receiving server until either network or processing bandwidth is available. What this does is take load off the client (as you've observed) by making the process asynchronous.

You can view the statuses of the queues using Performance Monitor, the number of messages in the outbound queue (at the client) and the server queue are written to performance counters in realtime and can be viewed and/or logged.

If the performance is sub-optimal, you could look to change the WCF throttling settings to allow more concurrent connections in your service behaviour.

e.g.

<serviceThrottling
         maxConcurrentCalls="20"
         maxConcurrentSessions="20"
         maxConcurrentInstances="20"
       />

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文