.NET - 使用单个 DNS 查询从网站下载多个页面

发布于 2024-08-30 04:49:06 字数 568 浏览 3 评论 0原文

我正在使用 HttpRequest 从网站下载多个页面(循环)。简化它看起来像这样:

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(
            "http://sub.domain.com/something/" + someString
        );

HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

//do something

我实际上不太确定,但每个请求似乎都会再次解析该地址(我不知道如何测试我是否正确)。 我想稍微提升一下它并解析一次地址,然后将其重用于所有请求。不过,我无法弄清楚如何强制 HttpRequest 使用它。

我尝试使用 Dns.GetHostAddresses,将结果转换为字符串并将其作为地址传递给 HttpWebRequest.Create。不幸的是,服务器返回错误 404。我设法用谷歌搜索,这可能是因为http查询的“主机”标头与服务器期望的不匹配。

有没有简单的方法来解决这个问题?

I'm using HttpRequest to download several pages from a website (in a loop). Simplifying it looks like this:

HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(
            "http://sub.domain.com/something/" + someString
        );

HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

//do something

I'm not quite sure actually but every request seems to resolve the address again (I don't know how to test if I'm right). I would like to boost it a little and resolve the address once and then reuse it for all requests. I can't work out how to force HttpRequest into using it, though.

I have tried using Dns.GetHostAddresses, converting the result to a string and passing it as the address to HttpWebRequest.Create. Unfortunately, server returns error 404 then. I managed to google that's probably because the "Host" header of the http query doesn't match what the server expects.

Is there a simple way to solve this?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

四叶草在未来唯美盛开 2024-09-06 04:49:06

老实说,我怀疑 DNS 尚未被缓存,但有一种方法可以满足您的要求。

使用 IP 地址创建请求后,设置 Host 属性将其设置为 DNS 名称。这应该可以解决您的 404 问题。

设置 KeepAlive 属性设置为 true。这将使连接保持打开状态,并允许您发出多个请求,而不必每次都重新建立连接。

I doubt that the DNS isn't being cached already to be honest, but there is a way to do what you ask.

After creating the request with the IP address, set the Host property on it to the DNS name. This should solve your 404 problem.

Something that might help you speed up your multiple requests is to set the KeepAlive property to true. This will keep the connection open and allow you to make multiple requests without having to re-establish the connection each time.

太阳公公是暖光 2024-09-06 04:49:06

404 肯定是由于网站的“主机标头”造成的 - 一个 IP 地址上可以托管数千个网站,并且 Web 服务器使用该域来确定您想要哪个网站。

您的本地计算机应该缓存 DNS 查询的结果,因此即使每次访问域时都会生成请求,该请求在第一次之后甚至不会离开您的计算机,而只是使用本地缓存的查找结果。

The 404 is definately because of the site's "host header" - thousands of sites can be hosted at a single IP address, and the web server uses the domain to figure out which one you want.

Your local computer should be caching the results of the DNS query, so even though it will generate a request each time you access the domain, the request won't even leave your computer after the first time, just using the locally cached lookup results.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文