如何使用 java.net.URLConnection 来触发和处理 HTTP 请求
使用 java.net.URLConnection
在这里经常被问到,Oracle 教程< /a> 太简洁了。
该教程基本上只展示了如何触发 GET 请求并读取响应。它没有在任何地方解释如何使用它来执行 POST 请求、设置请求标头、读取响应标头、处理 cookie、提交 HTML 表单、上传文件等。
那么,我如何使用 java.net.URLConnection 来触发和处理“高级” “HTTP 请求?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(12)
首先预先声明:发布的代码片段都是基本示例。您需要处理简单的
IOException
和RuntimeException
,例如NullPointerException
、ArrayIndexOutOfBoundsException
并自行处理。如果您正在为 Android 而不是 Java 进行开发,另请注意,自引入 API 级别 28 以来,明文 HTTP 请求是 默认禁用。我们鼓励您使用
HttpsURLConnection
。确实有必要时,可以在应用程序清单中启用明文。Java 11
如果您已经使用 Java 11 或更高版本,那么很高兴知道
java.net 旁边有.URLConnection
另一个以不太详细的方式处理 HTTP 请求的 API:java.net.http.HttpClient
。准备
我们首先需要至少知道 URL 和字符集。这些参数是可选的,取决于功能要求。
查询参数必须采用
name=value
格式,并通过&
连接。通常,您还可以使用 URL 编码 使用指定的字符集URLEncoder#encode()
。String#format()
只是为了方便。当我需要字符串连接运算符+
两次以上时,我更喜欢它。触发 带有(可选)查询参数的 HTTP GET 请求
这是一项微不足道的任务。这是默认的请求方法。
任何查询字符串都应使用
?
连接到 URL。接受字符集
标头可能会提示服务器参数的编码方式。如果您不发送任何查询字符串,则可以保留Accept-Charset
标头。如果您不需要设置任何标头,那么您甚至可以使用URL#openStream()
快捷方法。无论哪种方式,如果另一方是
HttpServlet< /code>
,然后是
doGet()
方法将被调用,参数将由HttpServletRequest#getParameter()
。出于测试目的,您可以将响应正文打印到标准输出,如下所示下面:
使用查询参数触发 HTTP POST
请求
URLConnection#setDoOutput()
为true
隐式将请求方法设置为 POST。 Web 表单的标准 HTTP POST 类型为application/x-www-form-urlencoded
,其中查询字符串被写入请求正文。注意:每当您想以编程方式提交 HTML 表单时,请不要忘记获取任何
的
name=value
对> 元素添加到查询字符串中,当然还有您想要“按”的元素的
name=value
对“以编程方式(因为这通常在服务器端使用来区分是否按下了按钮,如果按下了,是哪个按钮)。您还可以转换获得的
URLConnection
到HttpURLConnection
并使用其HttpURLConnection#setRequestMethod()
代替。但是,如果您尝试使用连接进行输出,您仍然需要设置URLConnection#setDoOutput()
为true
。无论哪种方式,如果另一方是
HttpServlet< /code>
,然后是
doPost()
方法将被调用,参数将由HttpServletRequest#getParameter()
。实际触发 HTTP 请求
您可以使用
URLConnection#connect()
,但是当您想要获取有关 HTTP 响应的任何信息时,该请求将根据需要自动触发,例如使用URLConnection#getInputStream()
等等。上面的示例正是这样做的,因此connect()
调用实际上是多余的。超时
您可以使用
URLConnection#setConnectTimeout()
设置连接超时和URLConnection#setReadTimeout()
设置读取超时。默认值基本上是“无超时”。所以你想自己设置这些。例如:
但是,使用基于 Sun/Oracle 的 JRE 时,需要注意读取超时。它会在抛出超时异常之前默默地重试读取,很可能只是为了在缓存中准备好任何成功的读取。另请参阅 Android (Java) HttpURLConnection 在“读取”时静默重试' timeout 这对于 GET 来说还可以,但对于 POST 来说绝对是错误的。如果您使用的是基于 Sun/Oracle 的 JRE,则需要按如下方式关闭它:
如果您正在为 Android 编写代码,则上述内容将不起作用,您需要在 POST 上进行此解决方案:
这将仅轻微影响性能。如果不希望出现这种情况,请考虑切换到其他 HTTP 客户端,例如 OkHttp。
收集 HTTP 响应信息
您需要
HttpURLConnection
在这里。如有必要,请先施放。HTTP 响应标头 :
HTTP 响应编码:
当
Content -Type
包含一个charset
参数,那么响应正文可能是基于文本的,我们希望使用服务器端指定的字符编码来处理响应正文。维护会话
服务器端会话通常由 cookie 支持。某些 Web 表单要求您登录和/或由会话跟踪。您可以使用
CookieHandler
用于维护 cookie 的 API。您需要准备一个CookieManager
带有ACCEPT_ALL
在发送所有 HTTP 请求之前。请注意,众所周知,这并不总是在所有情况下都能正常工作。如果您失败了,那么最好是手动收集并设置 cookie 标头。您基本上需要从登录响应或第一个
GET
请求中获取所有Set-Cookie
标头,然后将其传递给后续请求。split(";", 2)[0]
用于删除与服务器端无关的 cookie 属性,例如expires
、pathcookie.substring(0, cookie.indexOf(';'))
而不是split()
。流模式
< code>HttpURLConnection 默认情况下会在实际发送之前缓冲整个请求正文,无论您是否使用
connection.setRequestProperty 自己设置固定内容长度(“内容长度”,内容长度);
。每当您同时发送大型 POST 请求(例如上传文件)时,这可能会导致OutOfMemoryException
。为了避免这种情况,您需要设置HttpURLConnection#setFixedLengthStreamingMode()
。但是,如果事先确实不知道内容长度,那么您可以通过设置
HttpURLConnection#setChunkedStreamingMode()
相应地。这将设置 HTTP传输编码
header 到chunked
这将强制请求正文以块的形式发送。下面的示例将以 1 KB 为单位发送正文。用户代理
请求可能会返回意外响应,而它与真正的网络浏览器配合良好。服务器端可能根据
User 阻止请求-Agent
请求标头。URLConnection
默认情况下将其设置为Java/1.6.0_19
,其中最后一部分显然是 JRE 版本。您可以按如下方式覆盖此设置:使用最近使用的浏览器中的用户代理字符串。
错误处理
如果 HTTP 响应代码为
4nn
(客户端错误)或5nn
(服务器错误),那么您可能需要读取HttpURLConnection#getErrorStream()< /code> 查看服务器是否发送了任何有用的错误信息。
如果 HTTP 响应代码为 -1,则连接和响应处理出现问题。旧版 JRE 中的 HttpURLConnection 实现在保持连接活动方面存在一些问题。您可能想通过将
http.keepAlive
系统属性设置为false
来关闭它。您可以在应用程序的开头以编程方式执行此操作,方法是:上传文件
您通常使用
multipart/form-data
混合 POST 内容(二进制和字符数据)的编码。 RFC2388 中更详细地描述了编码。如果另一方是
HttpServlet
,然后是doPost()
方法将被调用,并且这些部分将由HttpServletRequest#getPart()
(注,因此不是getParameter()
等等!)。另请参阅此答案了解示例。处理不受信任或配置错误的 HTTPS 站点
如果您正在为 Android 而不是 Java 进行开发,要小心:如果您没有正确的证书,下面的解决方法可能会节省您的时间开发期间部署。但您不应该将其用于生产。如今(2021 年 4 月),如果 Google 检测到不安全的主机名验证程序,将不允许您的应用在 Play 商店上分发,请参阅 https://support.google.com/faqs/answer/7188426.
有时您需要连接 HTTPS 网址,可能是因为您正在编写网络爬虫。在这种情况下,您可能会在某些未及时更新 SSL 证书的 HTTPS 站点上遇到
javax.net.ssl.SSLException: Not Trusted servercertificate
,或者java .security.cert.CertificateException:在某些配置错误的 HTTPS 站点上找不到与 [主机名] 匹配的主题备用 DNS 名称
或javax.net.ssl.SSLProtocolException:握手警报:unrecognized_name
。Web scraper 类中的以下一次性运行的
static
初始化程序应该使HttpsURLConnection
对这些 HTTPS 站点更加宽松,从而不再抛出这些异常。解析和提取 HTML
如果您想要的只是从 HTML 中解析和提取数据,那么最好使用 HTML 解析器,例如 Jsoup。
First a disclaimer beforehand: the posted code snippets are all basic examples. You'll need to handle trivial
IOException
s andRuntimeException
s likeNullPointerException
,ArrayIndexOutOfBoundsException
and consorts yourself.In case you're developing for Android instead of Java, note also that since introduction of API level 28, cleartext HTTP requests are disabled by default. You are encouraged to use
HttpsURLConnection
. When really necessary, cleartext can be enabled in the Application Manifest.Java 11
In case you're already on Java 11 or newer, then it's good to know that there's next to
java.net.URLConnection
another API to deal with HTTP requests in a less verbose manner:java.net.http.HttpClient
.Preparing
We first need to know at least the URL and the charset. The parameters are optional and depend on the functional requirements.
The query parameters must be in
name=value
format and be concatenated by&
. You would normally also URL-encode the query parameters with the specified charset usingURLEncoder#encode()
.The
String#format()
is just for convenience. I prefer it when I would need the String concatenation operator+
more than twice.Firing an HTTP GET request with (optionally) query parameters
It's a trivial task. It's the default request method.
Any query string should be concatenated to the URL using
?
. TheAccept-Charset
header may hint the server what encoding the parameters are in. If you don't send any query string, then you can leave theAccept-Charset
header away. If you don't need to set any headers, then you can even use theURL#openStream()
shortcut method.Either way, if the other side is an
HttpServlet
, then itsdoGet()
method will be called and the parameters will be available byHttpServletRequest#getParameter()
.For testing purposes, you can print the response body to standard output as below:
Firing an HTTP POST request with query parameters
Setting the
URLConnection#setDoOutput()
totrue
implicitly sets the request method to POST. The standard HTTP POST as web forms do is of typeapplication/x-www-form-urlencoded
wherein the query string is written to the request body.Note: whenever you'd like to submit a HTML form programmatically, don't forget to take the
name=value
pairs of any<input type="hidden">
elements into the query string and of course also thename=value
pair of the<input type="submit">
element which you'd like to "press" programmatically (because that's usually been used in the server side to distinguish if a button was pressed and if so, which one).You can also cast the obtained
URLConnection
toHttpURLConnection
and use itsHttpURLConnection#setRequestMethod()
instead. But if you're trying to use the connection for output you still need to setURLConnection#setDoOutput()
totrue
.Either way, if the other side is an
HttpServlet
, then itsdoPost()
method will be called and the parameters will be available byHttpServletRequest#getParameter()
.Actually firing the HTTP request
You can fire the HTTP request explicitly with
URLConnection#connect()
, but the request will automatically be fired on demand when you want to get any information about the HTTP response, such as the response body usingURLConnection#getInputStream()
and so on. The above examples does exactly that, so theconnect()
call is in fact superfluous.Timeouts
You can use
URLConnection#setConnectTimeout()
to set the connect timeout andURLConnection#setReadTimeout()
to set the read timeout.The default is basically "no timeout". So you'd like to set these yourself. For example:
There's however a caveat with the read timeout when using Sun/Oracle based JRE. It will silently retry the reading before throwing the timeout exception, most probably merely to have any successfull reading ready in the cache. See also Android (Java) HttpURLConnection silent retry on 'read' timeout This is okayish for GET, but absolutely wrong for POST. In case you're using a Sun/Oracle based JRE, you'll want to turn off that as follows:
In case you're writing for Android, above will not work, you'll need this work around on POST:
This will only slightly impact the performance. In case that's undesireable, then consider switching to a different HTTP client such as OkHttp.
Gathering HTTP response information
You need an
HttpURLConnection
here. Cast it first if necessary.HTTP response headers:
HTTP response encoding:
When the
Content-Type
contains acharset
parameter, then the response body is likely text based and we'd like to process the response body with the server-side specified character encoding then.Maintaining the session
The server side session is usually backed by a cookie. Some web forms require that you're logged in and/or are tracked by a session. You can use the
CookieHandler
API to maintain cookies. You need to prepare aCookieManager
with aCookiePolicy
ofACCEPT_ALL
before sending all HTTP requests.Note that this is known to not always work properly in all circumstances. If it fails for you, then best is to manually gather and set the cookie headers. You basically need to grab all
Set-Cookie
headers from the response of the login or the firstGET
request and then pass this through the subsequent requests.The
split(";", 2)[0]
is there to get rid of cookie attributes which are irrelevant for the server side likeexpires
,path
, etc. Alternatively, you could also usecookie.substring(0, cookie.indexOf(';'))
instead ofsplit()
.Streaming mode
The
HttpURLConnection
will by default buffer the entire request body before actually sending it, regardless of whether you've set a fixed content length yourself usingconnection.setRequestProperty("Content-Length", contentLength);
. This may causeOutOfMemoryException
s whenever you concurrently send large POST requests (e.g. uploading files). To avoid this, you would like to set theHttpURLConnection#setFixedLengthStreamingMode()
.But if the content length is really not known beforehand, then you can make use of chunked streaming mode by setting the
HttpURLConnection#setChunkedStreamingMode()
accordingly. This will set the HTTPTransfer-Encoding
header tochunked
which will force the request body being sent in chunks. The below example will send the body in chunks of 1 KB.User-Agent
It can happen that a request returns an unexpected response, while it works fine with a real web browser. The server side is probably blocking requests based on the
User-Agent
request header. TheURLConnection
will by default set it toJava/1.6.0_19
where the last part is obviously the JRE version. You can override this as follows:Use the User-Agent string from a recent browser.
Error handling
If the HTTP response code is
4nn
(Client Error) or5nn
(Server Error), then you may want to read theHttpURLConnection#getErrorStream()
to see if the server has sent any useful error information.If the HTTP response code is -1, then something went wrong with connection and response handling. The
HttpURLConnection
implementation is in older JREs somewhat buggy with keeping connections alive. You may want to turn it off by setting thehttp.keepAlive
system property tofalse
. You can do this programmatically in the beginning of your application by:Uploading files
You'd normally use
multipart/form-data
encoding for mixed POST content (binary and character data). The encoding is in more detail described in RFC2388.If the other side is an
HttpServlet
, then itsdoPost()
method will be called and the parts will be available byHttpServletRequest#getPart()
(note, thus notgetParameter()
and so on!). Also see this answer for examples.Dealing with untrusted or misconfigured HTTPS sites
In case you're developing for Android instead of Java, be careful: the workaround below may save your day if you don't have correct certificates deployed during development. But you should not use it for production. These days (April 2021) Google will not allow your app be distributed on Play Store if they detect insecure hostname verifier, see https://support.google.com/faqs/answer/7188426.
Sometimes you need to connect an HTTPS URL, perhaps because you're writing a web scraper. In that case, you may likely face a
javax.net.ssl.SSLException: Not trusted server certificate
on some HTTPS sites who doesn't keep their SSL certificates up to date, or ajava.security.cert.CertificateException: No subject alternative DNS name matching [hostname] found
orjavax.net.ssl.SSLProtocolException: handshake alert: unrecognized_name
on some misconfigured HTTPS sites.The following one-time-run
static
initializer in your web scraper class should makeHttpsURLConnection
more lenient as to those HTTPS sites and thus not throw those exceptions anymore.Parsing and extracting HTML
If all you want is parsing and extracting data from HTML, then better use a HTML parser like Jsoup.
使用 HTTP 时,引用
HttpURLConnection
几乎总是比引用基类URLConnection
更有用(因为当您询问时URLConnection
是一个抽象类)对于 HTTP URL 上的URLConnection.openConnection()
,无论如何您都会得到返回的内容)。然后,您可以不依赖
URLConnection#setDoOutput(true)
隐式将请求方法设置为 POST,而是执行httpURLConnection.setRequestMethod("POST")
code> 有些人可能会觉得更自然(并且还允许您指定其他请求方法,例如 PUT、DELETE ...)。它还提供有用的 HTTP 常量,因此您可以执行以下操作:
When working with HTTP it's almost always more useful to refer to
HttpURLConnection
rather than the base classURLConnection
(sinceURLConnection
is an abstract class when you ask forURLConnection.openConnection()
on a HTTP URL that's what you'll get back anyway).Then you can instead of relying on
URLConnection#setDoOutput(true)
to implicitly set the request method to POST instead dohttpURLConnection.setRequestMethod("POST")
which some might find more natural (and which also allows you to specify other request methods such as PUT, DELETE, ...).It also provides useful HTTP constants so you can do:
受到 StackOverflow 上的这个问题和其他问题的启发,我创建了一个最小的开源 basic-http-client 体现了此处找到的大部分技术。
google-http-java-client 也是一个很棒的开源软件资源。
Inspired by this and other questions on Stack Overflow, I've created a minimal open source basic-http-client that embodies most of the techniques found here.
google-http-java-client is also a great open source resource.
我建议你看一下 kevinsawicki/http-request 上的代码,它基本上是
之上的包装器HttpUrlConnection
它提供了一个更简单的 API,以防您现在只想发出请求,或者您可以查看源代码(不是太大)来了解如何处理连接。示例:使用内容类型
application/json
和一些查询参数发出GET
请求:I suggest you take a look at the code on kevinsawicki/http-request, its basically a wrapper on top of
HttpUrlConnection
it provides a much simpler API in case you just want to make the requests right now or you can take a look at the sources (it's not too big) to take a look at how connections are handled.Example: Make a
GET
request with content typeapplication/json
and some query parameters:更新
在 Java 9 中,您可以发送
GET
请求,如下所示:然后您可以检查返回的
HttpResponse
:由于这个新的 HTTP 客户端位于
java.httpclient 中
jdk.incubator.httpclient
模块,您应该在module-info.java
文件中声明此依赖项:Update
In Java 9, you can send a
GET
request like:Then you can examine the returned
HttpResponse
:Since this new HTTP Client is in
java.httpclient
jdk.incubator.httpclient
module, you should declare this dependency in yourmodule-info.java
file:HTTP URL Hits 有两个选项: GET / POST
GET 请求:
POST 请求:
There are two options you can go with HTTP URL Hits : GET / POST
GET Request:
POST request:
这个回应也让我深受启发。
我经常在需要执行一些 HTTP 的项目中,并且我可能不想引入很多第三方依赖项(这会引入其他依赖项等等)
我开始编写自己的实用程序基于一些对话(不是任何地方完成的):
然后只有一堆或静态方法。
然后发布...
好吧,你明白了...
以下是测试:
您可以在这里找到其余部分:
https://github.com/RichardHightower/boon
我的目标是以一种更简单的方式提供人们想要做的常见事情......
I was also very inspired by this response.
I am often on projects where I need to do some HTTP, and I may not want to bring in a lot of third-party dependencies (which bring in others and so on and so on, etc.)
I started to write my own utilities based on some of this conversation (not any where done):
Then there are just a bunch or static methods.
Then post...
Well, you get the idea....
Here are the tests:
You can find the rest here:
https://github.com/RichardHightower/boon
My goal is to provide the common things one would want to do in a bit more easier way then....
最初我被这篇文章误导了,它有利于
HttpClient
。后来我意识到
HttpURLConnection
将从 保留下来这篇文章。根据 Google 博客:
阅读这篇文章< /a> 和其他一些堆栈溢出问题,我确信
HttpURLConnection
将保留更长时间。一些支持
HttpURLConnections
的 SE 问题:在 Android 上,使用 URL 编码表单数据发出 POST 请求,而不使用 UrlEncodedFormEntity
HttpPost 适用于 Java 项目,但不适用于 Android
Initially I was misled by this article which favours
HttpClient
.Later I have realized that
HttpURLConnection
is going to stay from this article.As per the Google blog:
After reading this article and some other stack over flow questions, I am convinced that
HttpURLConnection
is going to stay for longer durations.Some of the SE questions favouring
HttpURLConnections
:On Android, make a POST request with URL Encoded Form data without using UrlEncodedFormEntity
HttpPost works in Java project, but not on Android
还有 OkHttp,它是一个默认情况下高效的 HTTP 客户端:
首先创建一个
OkHttpClient
实例:然后,准备您的
GET
请求:最后,使用
OkHttpClient
发送准备好的Request
:更多细节可以参考OkHttp的文档
There is also OkHttp, which is an HTTP client that’s efficient by default:
First create an instance of
OkHttpClient
:Then, prepare your
GET
request:finally, use
OkHttpClient
to send preparedRequest
:For more details, you can consult the OkHttp's documentation
如果您使用 HTTP GET,请删除此行:
If you are using HTTP GET, please remove this line:
您还可以使用
JdkRequest
来自 < a href="http://http.jcabi.com">jcabi-http(我是一名开发人员),它为您完成所有这些工作,例如装饰 HttpURLConnection、触发 HTTP 请求和解析响应:查看此博客文章了解更多信息:http://www.yegor256.com/2014 /04/11/jcabi-http-intro.html
You can also use
JdkRequest
from jcabi-http (I'm a developer), which does all this work for you, decorating HttpURLConnection, firing HTTP requests and parsing responses, for example:Check this blog post for more info: http://www.yegor256.com/2014/04/11/jcabi-http-intro.html
如果您使用的是 Java 11 或更高版本(Android 除外),而不是旧版 < code>HttpUrlConnection 类,您可以使用 Java 11 新的 HTTP 客户端 API 。
示例 GET 请求:
异步执行的相同请求:
示例 POST 请求:
用于以多部分形式发送表单数据 (
multipart/form-data
)或 url 编码 (application/x-www-form-urlencoded
) 格式,请参阅此解决方案。有关 HTTP 客户端 API 的示例和更多信息,请参阅本文。
旁注
要使用 Java 标准库创建简单的 HTTP 服务器,请参阅这篇文章。
If you are using Java 11 or newer (except on Android), instead of the legacy
HttpUrlConnection
class, you can use Java 11 new HTTP Client API.An example GET request:
The same request executed asynchronously:
An example POST request:
For sending form data as multipart (
multipart/form-data
) or url-encoded (application/x-www-form-urlencoded
) format, see this solution.See this article for examples and more information about HTTP Client API.
Sidenote
To create a simple HTTP server using Java standard library, see this post.