针对 Azure 服务管理 API 的 Python HTTPS 在 Windows 上失败
我最近扩展了 Windows Azure 存储 API (PyAzure) 的 Python API,以包含对服务管理 API 的支持。请参阅 https://github.com/bmb/pyazure。
我正在使用 HTTPSClientAuthHandler 就像 使用 pyOpenSSL 中建议的那样创建 urllib 自定义开启器。在 Linux 上,对于各种版本的 Python 2.6 和 2.7,这都可以很好地工作。然而,Windows 则是另一回事。针对 Azure 管理主机地址的所有请求都会失败,并显示:
[Errno 10054]现有连接被远程主机强制关闭
我认为,拖拽中的套接字 errno 10054“连接被对等重置”。
这在我的 API 代码中似乎不是问题(除非我使用的客户端证书身份验证方法在某种程度上是伪造的),而是较低级别的问题。我可以在没有 urllib2 或 httplib 的情况下重现该问题,只需设置 SSL 套接字并按照 urllib2 的方式通过管道发送相同的 HTTP 请求,例如列出有效的 Azure 数据中心位置:
>>> import socket, ssl, sys
>>> sys.version
'2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send("GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n")
202
>>> s.read()
Traceback (most recent call last):
c:\Users\blair\research\clouds\azure\pyazure\<ipython-input-63-3306c981d8a7>
in <module>()
----> 1 s.read()
C:\Python27\lib\ssl.pyc in read(self, len)
136
137 try:
--> 138 return self._sslobj.read(len)
139 except SSLError, x:
140 if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
error: [Errno 10054] An existing connection was forcibly closed by the remote host
将上面的 SUBSCRIPTION_ID 替换为您的 Azure 订阅 ID。调用 SSLSocket.read 后约 45 秒引发异常。该证书是一个格式正确的 PEM 文件,包括私钥和证书,它是使用以下命令从 pfx(在 Ubuntu 10.04 中)转换而来的:
openssl pkcs12 -in pfxfile -out pemfile -nodes
我认为这并不重要,但我也尝试过 unix2dos-ing PEM 文件,但无济于事。即使我不提供任何证书,我也会得到相同的行为,但在 Linux 上这样做会导致服务器出现正确的 API 错误:
'HTTP/1.1 403 禁止\r\n内容长度: 0\r\n服务器: Microsoft-HTTPAPI/2.0\r\n日期: 2011 年 12 月 1 日星期四 13:59:29 GMT\r\n连接: 关闭\ r\n\r\n'
这已由使用 Windows 7 的另一个人(与我相同)独立验证。这不是客户端防火墙问题 - 相同的代码可以在同一主机上运行的经过 NAT 处理的 Linux VM 中运行。
我很困惑。非常感谢这里的人们能够提供的任何帮助...
更新: 这似乎与 Python 中的底层 SSL 实现有关。 CPython 2.7.1 具有如上所示的错误行为,但我已经使用 ActiveState Python(2.7 和 2.6)进行了测试并取得了成功,例如:
>>> import sys, socket, ssl
>>> sys.version
'2.7.1 (r271:86832, Feb 7 2011, 11:30:38) [MSC v.1500 32 bit (Intel)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='\\\\VBOXSVR\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send('GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nUser-Agent: Python-urllib/2.6\r\n\r\n')
183
>>> s.read(4096)
'HTTP/1.1 200 OK\r\nContent-Length: 908\r\nContent-Type: application/xml; charset=utf-8\r\nServer: Microsoft-HTTPAPI/2.0\r\nx-ms-request-id: 08ca048cda6b445da6b3a8f3e4890197\r\nDate: Fri, 02 Dec 2011 03:02:14 GMT\r\n\r\n<Locations xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Location><Name>Anywhere US</Name><DisplayName>Anywhere US</DisplayName></Location><Location><Name>South Central US</Name><DisplayName>South Central US</DisplayName></Location><Location><Name>North Central US</Name><DisplayName>North Central US</DisplayName></Location><Location><Name>Anywhere Europe</Name><DisplayName>Anywhere Europe</DisplayName></Location><Location><Name>North Europe</Name><DisplayName>North Europe</DisplayName></Location><Location><Name>West Europe</Name><DisplayName>West Europe</DisplayName></Location><Location><Name>Anywhere Asia</Name><DisplayName>Anywhere Asia</DisplayName></Location><Location><Name>Southeast Asia</Name><DisplayName>Southeast Asia</DisplayName></Location><Location><Name>East Asia</Name><DisplayName>East Asia</DisplayName></Location></Locations>'
正如预期的那样,我的 API 也可以工作:
ActivePython 2.6.7.20 (ActiveState Software Inc.) based on
Python 2.6.7 (r267:88850, Jun 27 2011, 13:20:48) [MSC v.1500 64 bit (AMD64)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyazure import pyazure
>>> pa = pyazure.PyAzure(subscription_id=SUBSCRIPTION_ID, management_cert_path='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> list(pa.wasm.list_locations())
['Anywhere US', 'South Central US', 'North Central US', 'Anywhere Europe', 'North Europe', 'West Europe', 'Anywhere Asia', 'Southeast Asia', 'East Asia']
CPython2 中的 Lib\ssl.py 文件.7 和 ActivePython2.7 是相同的,所以我猜这一定是由于底层 C 库的一些差异,可能是 CPython 中的错误。有高手在吗?
I've recently extended a Python API for the Windows Azure storage APIs (PyAzure) to include support for the service management APIs. See https://github.com/bmb/pyazure.
I'm using a HTTPSClientAuthHandler like the one suggested in using pyOpenSSL to create urllib custom opener. On Linux, with various versions of Python 2.6 and 2.7 this works well. However, Windows is another story. All requests against the Azure management host address fail with:
[Errno 10054] An existing connection was forcibly closed by the remote host
Which I think, is the socket errno 10054 "Connection reset by peer", in drag.
This doesn't appear to be a problem in my API code (unless the client cert authentication method I'm using is bogus somehow), but something lower-level. I can reproduce the issue without urllib2 or httplib by simply setting up an SSL socket and sending the same HTTP request down the pipe as urllib2 would, e.g. to list the valid Azure data centre locations:
>>> import socket, ssl, sys
>>> sys.version
'2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send("GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nConnection: close\r\nUser-Agent: Python-urllib/2.6\r\n\r\n")
202
>>> s.read()
Traceback (most recent call last):
c:\Users\blair\research\clouds\azure\pyazure\<ipython-input-63-3306c981d8a7>
in <module>()
----> 1 s.read()
C:\Python27\lib\ssl.pyc in read(self, len)
136
137 try:
--> 138 return self._sslobj.read(len)
139 except SSLError, x:
140 if x.args[0] == SSL_ERROR_EOF and self.suppress_ragged_eofs:
error: [Errno 10054] An existing connection was forcibly closed by the remote host
Replace SUBSCRIPTION_ID above, with your Azure subscription ID. The exception is raised ~45s after calling SSLSocket.read. The cert is a properly formatted PEM file including both the private key and certificate, it was converted from the pfx (in Ubuntu 10.04) using:
openssl pkcs12 -in pfxfile -out pemfile -nodes
I don't think it matters here, but I also tried unix2dos-ing the PEM file, to no avail though. I get the same behaviour even when I don't provide any cert, but doing that on Linux results in a proper API error from the server:
'HTTP/1.1 403 Forbidden\r\nContent-Length: 0\r\nServer: Microsoft-HTTPAPI/2.0\r\nDate: Thu, 01 Dec 2011 13:59:29 GMT\r\nConnection: close\r\n\r\n'
This has been independently verified by another person using Windows 7 (same as me). It's not a client-side firewall issue - the same code works in a NAT-ed Linux VM running on the same host.
I'm stumped. Would really appreciate any help folks here might be able to provide...
Update:
This appears to be related to the underlying SSL implementation in Python. CPython 2.7.1 has the error behaviour as shown above, but I've since tested and had success with ActiveState Python (both 2.7 and 2.6), e.g.:
>>> import sys, socket, ssl
>>> sys.version
'2.7.1 (r271:86832, Feb 7 2011, 11:30:38) [MSC v.1500 32 bit (Intel)]'
>>> s = ssl.wrap_socket(socket.socket(), certfile='\\\\VBOXSVR\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> s.connect(('management.core.windows.net',443))
>>> s.send('GET /SUBSCRIPTION_ID/locations HTTP/1.1\r\nAccept-Encoding: identity\r\nX-Ms-Version: 2011-10-01\r\nHost: management.core.windows.net\r\nUser-Agent: Python-urllib/2.6\r\n\r\n')
183
>>> s.read(4096)
'HTTP/1.1 200 OK\r\nContent-Length: 908\r\nContent-Type: application/xml; charset=utf-8\r\nServer: Microsoft-HTTPAPI/2.0\r\nx-ms-request-id: 08ca048cda6b445da6b3a8f3e4890197\r\nDate: Fri, 02 Dec 2011 03:02:14 GMT\r\n\r\n<Locations xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Location><Name>Anywhere US</Name><DisplayName>Anywhere US</DisplayName></Location><Location><Name>South Central US</Name><DisplayName>South Central US</DisplayName></Location><Location><Name>North Central US</Name><DisplayName>North Central US</DisplayName></Location><Location><Name>Anywhere Europe</Name><DisplayName>Anywhere Europe</DisplayName></Location><Location><Name>North Europe</Name><DisplayName>North Europe</DisplayName></Location><Location><Name>West Europe</Name><DisplayName>West Europe</DisplayName></Location><Location><Name>Anywhere Asia</Name><DisplayName>Anywhere Asia</DisplayName></Location><Location><Name>Southeast Asia</Name><DisplayName>Southeast Asia</DisplayName></Location><Location><Name>East Asia</Name><DisplayName>East Asia</DisplayName></Location></Locations>'
And as expected my API works too:
ActivePython 2.6.7.20 (ActiveState Software Inc.) based on
Python 2.6.7 (r267:88850, Jun 27 2011, 13:20:48) [MSC v.1500 64 bit (AMD64)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyazure import pyazure
>>> pa = pyazure.PyAzure(subscription_id=SUBSCRIPTION_ID, management_cert_path='c:\\users\\blair\\research\\clouds\\azure\\BlairBethwaiteAzure1.pfx.pem')
>>> list(pa.wasm.list_locations())
['Anywhere US', 'South Central US', 'North Central US', 'Anywhere Europe', 'North Europe', 'West Europe', 'Anywhere Asia', 'Southeast Asia', 'East Asia']
The Lib\ssl.py files in CPython2.7 and ActivePython2.7 are identical, so I guess this must be due to some difference in the underlying C libs, perhaps a bug in CPython. Any gurus out there?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
我无法对此做出明确的解释,但经过一番尝试和错误后,我对问题所在...
简短的回答:这是 http://www.python.org/ Windows 捆绑包。请改用 ActiveState Python。
长答案:
Windows CPython 发行版可从 http://www.python.org/download/ 捆绑包获取版本的 OpenSSL (0.9.8l) 与 ActiveState Python 发行版相比,ActiveState Python 发行版基于 CPython,但(除其他外)提供对 3rd 方包含项的定期更新,例如 OpenSSL(当前为 0.9.8r)。
我下载了 OpenSSL 的 Windows 二进制文件并通过 openssl s_client 接口进行测试,例如:
当前版本按预期工作。不幸的是,似乎很难获得适用于 Windows 的旧 OpenSSL 二进制文件,考虑到它是一个安全库,也许并不奇怪...但无论如何,我在 Ubuntu 10.04 下从源代码构建了 0.9.8l,发现它在发送 HTTP 请求后挂起在管道中,可能是服务器由于某种原因默默地断开了连接:
在较新甚至稍旧的(例如,Ubuntu10.04 的 0.9.8e)OpenSSL 下,服务器响应请求预期的:
但是使用 OpenSSL 0.9.8l 我什么也没得到。
I haven't been able to pin down a definitive explanation for this, but after a bit of trial and error I'm confident of where the issue lays...
Short answer: it's the ssl implementation in the http://www.python.org/ Windows bundle. Use ActiveState Python instead.
Long Answer:
The Windows CPython distributions available from http://www.python.org/download/ bundle quite an old version of OpenSSL (0.9.8l), compared with the ActiveState Python distributions which are based on CPython but (amongst other things) provide regular updates to 3rd party inclusions such as OpenSSL (currently 0.9.8r).
I downloaded Windows binaries of OpenSSL and tested via the openssl s_client interface, e.g.:
The current version works, as expected. Unfortunately it seems difficult to get one's hands on old OpenSSL binaries for Windows, perhaps not surprising given it is a security library... But anyway, I built 0.9.8l from source under Ubuntu 10.04 and found that it hangs after sending a HTTP request down the pipe, presumably the server silently dropped the connection for some reason:
Under newer and even slightly older (e.g., Ubuntu10.04's 0.9.8e) OpenSSLs the server responds to the request with the expected:
But with OpenSSL 0.9.8l I get nothing.
在 Windows 7 上使用 IronPython 2.7.1,在 OS X 10.6.8 上使用 CPython 2.6.6,以下内容按预期工作:
[注意:我将 MYKEYFILENAME.pem 作为命令行参数传递。]
祝 Azure 黑客快乐!
The following works as expected using IronPython 2.7.1 on Windows 7 and CPython 2.6.6 on OS X 10.6.8:
[NOTE: I'm passing MYKEYFILENAME.pem as a command-line parameter.]
Happy Azure hacking!
我不是 Python 开发人员。但是在处理来自 iPhone 和 Windows Phone 的 Azure 服务时,我遇到了很多问题。请确保以下
I am not a Python developer .But I have faced so many issues when dealing with Azure services from iPhone and Windows Phone .Please ensure the following