WebClient.DownloadFile 在BackgroundWorker 性能不可接受

发布于 2024-11-10 16:00:44 字数 1118 浏览 4 评论 0原文

我有一个 WinForms 应用程序，每天需要下载许多（数万）相当大（多 MB）的文件。我使用以下代码编写了一个简单的测试：

using (var wc = new System.Net.WebClient())
{
    foreach (string url in UrlsToDownload())
    {
        string targetPath = SafeFilePathOf(url);
        wc.DownloadFile(url, targetPath);
    }
}

目标机器具有 1 Gb/s 连接，但测试显示持续下载约为 1MB/s。这低于我的预期，但源服务器的连接速度可能较慢。一整天的下载需要几个小时的连接时间，这是可以接受的。网络利用率相当稳定在 1% 左右：

However, I need to perform downloads in a background thread and support cancelling and download progress. The .Net System.ComponentModel.BackgroundWorker seems designed for this, so I put the exact same code in a BackgroundWorker instance and call RunWorkerAsync: DownloadFile in BackgroundWorker

下载性能骤降至约 0.05 MB/s。一天的工作大约需要一周的时间才能完成；这不会飞。

为什么 BackgroundWorker 性能如此糟糕？ CPU和网络都没有过载。应用程序没有被阻止，我只是将下载代码从 UI 线程移至 BackgroundWorker。调用backgroundWorker.Priority = ThreadPriority.AboveNormal 没有任何效果。

原文

I have a WinForms application that needs to download many (tens of thousands) fairly large (multi-MB) files per day. I wrote a simple test using the following code:

using (var wc = new System.Net.WebClient())
{
    foreach (string url in UrlsToDownload())
    {
        string targetPath = SafeFilePathOf(url);
        wc.DownloadFile(url, targetPath);
    }
}

The target machine has a 1 Gb/s connection, but testing shows a sustained download of about 1MB/s. This is less than what I was expecting, but the source servers may have slower connections. A full day's download will require several hours of connectivity, which is acceptable. Network utilization is fairly constant at about 1%:

DownloadFile in UI thread

DownloadFile in BackgroundWorker

Download performance plummets to about 0.05 MB/s. A day's work will require about a week to perform; this is not going to fly.

Why is BackgroundWorker performance so bad? Neither the CPU nor network is overloaded. The application is not blocked, I simply moved the download code from the UI thread to a BackgroundWorker. Calling backgroundWorker.Priority = ThreadPriority.AboveNormal has no effect.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

何处潇湘 2024-11-17 16:00:44

增加 WebClient 上的连接限制：

internal class CustomWebClient : WebClient
{
    public CustomWebClient()
    {
    }

    protected override WebRequest GetWebRequest(Uri uri)
    {
        HttpWebRequest w = (HttpWebRequest)base.GetWebRequest(uri);

        w.ServicePoint.ConnectionLimit = 100;
        return w;
    }
}

Increase the connection limit on the WebClient:

internal class CustomWebClient : WebClient
{
    public CustomWebClient()
    {
    }

    protected override WebRequest GetWebRequest(Uri uri)
    {
        HttpWebRequest w = (HttpWebRequest)base.GetWebRequest(uri);

        w.ServicePoint.ConnectionLimit = 100;
        return w;
    }
}

回复收藏 0 原文