HttpWebRequest 如何处理（过早）关闭底层 TCP 连接？

发布于 2024-08-04 00:45:18 字数 2889 浏览 3 评论 0原文

当使用 .NET 的 HttpWebRequest 类调用远程服务器（特别是 REST Web 服务）时，我很难弄清楚是否有办法处理潜在的连接问题。根据我的调查，WebClient 类的行为是相同的，这在某种程度上是预期的，因为它似乎只为 HttpWebRequest 提供更简单的接口。

出于模拟目的，我编写了一个非常简单的 HTTP 服务器，其行为不符合 HTTP 1.1 RFC。它的作用是接受客户端连接，然后发送适当的 HTTP 1.1 标头和“Hello World!”将有效负载返回给客户端并关闭套接字，在服务器端接受客户端连接的线程如下所示：

    private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
    private void Listen()
    {
        while (true)
        {
            using (TcpClient clientConnection = m_listener.AcceptTcpClient())
            {
                NetworkStream stream = clientConnection.GetStream();
                StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
                httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
                httpData.AppendFormat(m_defaultResponse);

                Thread.Sleep(3000); // Sleep to simulate latency

                stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);

                stream.Close();

                clientConnection.Close();
            }
        }
    }

由于 HTTP 1.1 RFC 规定 HTTP 1.1 默认情况下保持连接处于活动状态，并且服务器必须发送“连接：关闭”响应header 如果它想关闭连接，这对于客户端来说是意外的行为。客户端按以下方式使用HttpWebRequest：

    private static void SendRequest(object _state)
    {
        WebResponse resp = null;

        try
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
            request.Timeout = 50 * 1000;

            DateTime requestStart = DateTime.Now;
            resp = request.GetResponse();
            TimeSpan requestDuration = DateTime.Now - requestStart;

            Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
        }
        catch (WebException ex)
        {
            if (ex.Status == WebExceptionStatus.Timeout)
            {
                Console.WriteLine("Timeout occurred");
            }
            else
            {
                Console.WriteLine(ex);
            }
        }
        finally
        {
            if (resp != null)
            {
                resp.Close();
            }

            ((ManualResetEvent)_state).Set();
        }
    }

上面的方法通过ThreadPool.QueueUserWorkItem(waitCallback, stateObject)进行排队。 ManualResetEvent 用于控制排队行为，以便整个线程池不会被等待任务填满（因为 HttpWebRequest 隐式使用工作线程，因为它在内部异步运行以实现超时功能）。

所有这一切的问题是，一旦 HttpWebRequest 底层 ServicePoint 的所有连接都“用完”（即被远程服务器关闭），将不会再打开新的连接。 ServicePoint 的 ConnectionLeaseTimeout 设置为较低值（10 秒）也没关系。一旦系统进入这种状态，它将不再正常工作，因为它不会自动重新连接，并且所有后续的 HttpWebRequest 都会超时。现在的问题确实是，是否有一种方法可以通过在某些条件下破坏 ServicePoint 或关闭底层连接来解决这个问题（我对 ServicePoint.CloseConnectionGroup() 还没有任何运气，该方法在如何执行方面也没有任何记录）才能正确使用它）。

有人知道我该如何解决这个问题吗？

原文

I have a hard time figuring out if there is a way to handle potential connectivity problems when using .NET's HttpWebRequest class to call a remote server (specifically a REST web service). From my investigations the behaviour of the WebClient class is the same, which is somewhat expected since it appears to only offer a more simple interface to the HttpWebRequest.

For simulation purposes, I've written a very simple HTTP server that does not behave according to the HTTP 1.1 RFC. What it does is it accepts a client connection, then sends appropriate HTTP 1.1 headers and a "Hello World!" payload back to the client and closes the socket, the thread accepting client connections on the server side looks as follows:

    private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
    private void Listen()
    {
        while (true)
        {
            using (TcpClient clientConnection = m_listener.AcceptTcpClient())
            {
                NetworkStream stream = clientConnection.GetStream();
                StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
                httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
                httpData.AppendFormat(m_defaultResponse);

                Thread.Sleep(3000); // Sleep to simulate latency

                stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);

                stream.Close();

                clientConnection.Close();
            }
        }
    }

Since the HTTP 1.1 RFC states that HTTP 1.1 by default keeps connections alive and that a server must send a "Connection: Close" response header if it wants to close a connection this is unexpected behaviour for the client-side. The client uses HttpWebRequest in the following way:

    private static void SendRequest(object _state)
    {
        WebResponse resp = null;

        try
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
            request.Timeout = 50 * 1000;

            DateTime requestStart = DateTime.Now;
            resp = request.GetResponse();
            TimeSpan requestDuration = DateTime.Now - requestStart;

            Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
        }
        catch (WebException ex)
        {
            if (ex.Status == WebExceptionStatus.Timeout)
            {
                Console.WriteLine("Timeout occurred");
            }
            else
            {
                Console.WriteLine(ex);
            }
        }
        finally
        {
            if (resp != null)
            {
                resp.Close();
            }

            ((ManualResetEvent)_state).Set();
        }
    }

The above method is queued via ThreadPool.QueueUserWorkItem(waitCallback, stateObject). The ManualResetEvent is used to control queuing behavior so that not the entire thread pool gets filled up with waiting tasks (since the HttpWebRequest implicitly uses worker threads because it functions asynchronously internally to implement the timeout functionality).

The problem with all this is that once all connections of the HttpWebRequest's underlying ServicePoint are "used up" (i.e. closed by the remote server), there will be no new ones opened up. It also does not matter if the ConnectionLeaseTimeout of the ServicePoint is set to a low value (10 seconds). Once the system gets into this state, it will no longer function properly because it does not reconnect automatically and all subsequent HttpWebRequests will time out. Now the question really is if there is a way to solve this by somehow destroying a ServicePoint under certain conditions or closing underlying connections (I did not have any luck with ServicePoint.CloseConnectionGroup() yet, the method is also pretty undocumented in terms of how to properly use it).

Does anybody have any idea how I could approach this problem?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

醉生梦死 2024-08-11 00:45:18

我根据这里的一些想法提出的解决方案是自己管理连接。如果将唯一的ConnectionGroupName 分配给WebRequest（例如Guid.NewGuid().ToString()），则将在ServicePoint 中为该请求创建一个具有一个连接的新连接组。请注意，此时不再有连接限制，因为 .NET 限制每个连接组而不是每个 ServicePoint，因此您必须自己处理它。您需要重用连接组，以便重用现有的 KeepAlive 连接，但如果发生 WebException 异常，则应销毁请求的连接组，因为它可能已过时。像这样的东西（为每个主机名创建一个新实例）：

public class ConnectionManager {
    private const int _maxConnections = 4;

    private Semaphore _semaphore = new Semaphore(_maxConnections, _maxConnections);
    private Stack<string> _groupNames = new Stack<string>();

    public string ObtainConnectionGroupName() {
        _semaphore.WaitOne();
        return GetConnectionGroupName();
    }

    public void ReleaseConnectionGroupName(string name) {
        lock (_groupNames) {
            _groupNames.Push(name);
        }
        _semaphore.Release();
    }

    public string SwapForFreshConnection(string name, Uri uri) {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.CloseConnectionGroup(name);
        return GetConnectionGroupName();
    }

    private string GetConnectionGroupName() {
        lock (_groupNames) {
            return _groupNames.Count != 0 ? _groupNames.Pop() : Guid.NewGuid().ToString();
        }
    }
}

The solution I came up with based on some of the ideas here is to manage the connections myself. If a unique ConnectionGroupName is assigned to a WebRequest (e.g. Guid.NewGuid().ToString()), a new connection group with one connection will be created in the ServicePoint for the request. Note that there's no more connection limiting at this point, since .NET limits per connection group rather than per ServicePoint, so you'll have to handle it yourself. You'll want to reuse connection groups so that existing connections with KeepAlive are reused, but if a WebException exception occurs, the request's connection group should be destroyed since it might be stale. Something like this (create a new instance for each host name):

public class ConnectionManager {
    private const int _maxConnections = 4;

    private Semaphore _semaphore = new Semaphore(_maxConnections, _maxConnections);
    private Stack<string> _groupNames = new Stack<string>();

    public string ObtainConnectionGroupName() {
        _semaphore.WaitOne();
        return GetConnectionGroupName();
    }

    public void ReleaseConnectionGroupName(string name) {
        lock (_groupNames) {
            _groupNames.Push(name);
        }
        _semaphore.Release();
    }

    public string SwapForFreshConnection(string name, Uri uri) {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.CloseConnectionGroup(name);
        return GetConnectionGroupName();
    }

    private string GetConnectionGroupName() {
        lock (_groupNames) {
            return _groupNames.Count != 0 ? _groupNames.Pop() : Guid.NewGuid().ToString();
        }
    }
}

回复收藏 0 原文

过潦 2024-08-11 00:45:18

这是一个可怕的黑客行为，但它确实有效。如果您发现连接卡住，请定期调用它。

    static public void SetIdle(object request)
    {
        MethodInfo getConnectionGroupLine = request.GetType().GetMethod("GetConnectionGroupLine", BindingFlags.Instance | BindingFlags.NonPublic);
        string connectionName = (string)getConnectionGroupLine.Invoke(request, null);

        ServicePoint servicePoint = ((HttpWebRequest)request).ServicePoint;
        MethodInfo findConnectionGroup = servicePoint.GetType().GetMethod("FindConnectionGroup", BindingFlags.Instance | BindingFlags.NonPublic);
        object connectionGroup;
        lock (servicePoint)
        {
            connectionGroup = findConnectionGroup.Invoke(servicePoint, new object[] { connectionName, false });
        }

        PropertyInfo currentConnections = connectionGroup.GetType().GetProperty("CurrentConnections", BindingFlags.Instance | BindingFlags.NonPublic);
        PropertyInfo connectionLimit = connectionGroup.GetType().GetProperty("ConnectionLimit", BindingFlags.Instance | BindingFlags.NonPublic);

        MethodInfo disableKeepAliveOnConnections = connectionGroup.GetType().GetMethod("DisableKeepAliveOnConnections", BindingFlags.Instance | BindingFlags.NonPublic);

        if (((int)currentConnections.GetValue(connectionGroup, null)) ==
            ((int)connectionLimit.GetValue(connectionGroup, null)))
        {
            disableKeepAliveOnConnections.Invoke(connectionGroup, null);
        }

        MethodInfo connectionGoneIdle = connectionGroup.GetType().GetMethod("ConnectionGoneIdle", BindingFlags.Instance | BindingFlags.NonPublic);
        connectionGoneIdle.Invoke(connectionGroup, null);
    }

This is a horrible hack, but it works. Call it periodically if you notice your connections are getting stuck.

    static public void SetIdle(object request)
    {
        MethodInfo getConnectionGroupLine = request.GetType().GetMethod("GetConnectionGroupLine", BindingFlags.Instance | BindingFlags.NonPublic);
        string connectionName = (string)getConnectionGroupLine.Invoke(request, null);

        ServicePoint servicePoint = ((HttpWebRequest)request).ServicePoint;
        MethodInfo findConnectionGroup = servicePoint.GetType().GetMethod("FindConnectionGroup", BindingFlags.Instance | BindingFlags.NonPublic);
        object connectionGroup;
        lock (servicePoint)
        {
            connectionGroup = findConnectionGroup.Invoke(servicePoint, new object[] { connectionName, false });
        }

        PropertyInfo currentConnections = connectionGroup.GetType().GetProperty("CurrentConnections", BindingFlags.Instance | BindingFlags.NonPublic);
        PropertyInfo connectionLimit = connectionGroup.GetType().GetProperty("ConnectionLimit", BindingFlags.Instance | BindingFlags.NonPublic);

        MethodInfo disableKeepAliveOnConnections = connectionGroup.GetType().GetMethod("DisableKeepAliveOnConnections", BindingFlags.Instance | BindingFlags.NonPublic);

        if (((int)currentConnections.GetValue(connectionGroup, null)) ==
            ((int)connectionLimit.GetValue(connectionGroup, null)))
        {
            disableKeepAliveOnConnections.Invoke(connectionGroup, null);
        }

        MethodInfo connectionGoneIdle = connectionGroup.GetType().GetMethod("ConnectionGoneIdle", BindingFlags.Instance | BindingFlags.NonPublic);
        connectionGoneIdle.Invoke(connectionGroup, null);
    }

回复收藏 0 原文

心碎的声音 2024-08-11 00:45:18

这是我的建议。我还没有测试过。
更改引用.cs

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        try
        {
            return base.GetWebResponse(request);
        }
        catch (WebException)
        {
            HttpWebRequest httpWebRequest = request as HttpWebRequest;
            if (httpWebRequest != null && httpWebRequest.ServicePoint != null)
                httpWebRequest.ServicePoint.CloseConnectionGroup(httpWebRequest.ConnectionGroupName);

            throw;
        }
    }

Here is my suggestion. I have not tested it.
Alter reference.cs

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        try
        {
            return base.GetWebResponse(request);
        }
        catch (WebException)
        {
            HttpWebRequest httpWebRequest = request as HttpWebRequest;
            if (httpWebRequest != null && httpWebRequest.ServicePoint != null)
                httpWebRequest.ServicePoint.CloseConnectionGroup(httpWebRequest.ConnectionGroupName);

            throw;
        }
    }

回复收藏 0 原文

~没有更多了~