BeautifulSoup 没有给我 Unicode
我正在使用 Beautiful soup 来抓取数据。 BS 文档指出 BS 应始终返回 Unicode，但我似乎无法获取 Unicode。这是一个代码片段
import urllib2
from libs.BeautifulSoup import BeautifulSoup
# Fetch and parse the data
url = 'http://wiki.gnhlug.org/twiki2/bin/view/Www/PastEvents2007?skin=print.pattern'
data = urllib2.urlopen(url).read()
print 'Encoding of fetched HTML : %s', type(data)
soup = BeautifulSoup(data)
print 'Encoding of souped up HTML : %s', soup.originalEncoding
table = soup.table
print type(table.renderContents())
从页面返回的原始数据是一个字符串。 BS 将原始编码显示为 ISO-8859-1。我认为 BS 会自动将所有内容转换为 Unicode，那么为什么当我这样做时：
table = soup.table
print type(table.renderContents())
..它给了我一个字符串对象而不是 Unicode？
我如何从 BS 获取 Unicode 对象？
我真的真的迷失了。有什么帮助吗？提前致谢。
I'm using Beautiful soup to scrape data. The BS documentation states that BS should always return Unicode but I can't seem to get Unicode. Here's a code snippet
import urllib2
from libs.BeautifulSoup import BeautifulSoup
# Fetch and parse the data
url = 'http://wiki.gnhlug.org/twiki2/bin/view/Www/PastEvents2007?skin=print.pattern'
data = urllib2.urlopen(url).read()
print 'Encoding of fetched HTML : %s', type(data)
soup = BeautifulSoup(data)
print 'Encoding of souped up HTML : %s', soup.originalEncoding
table = soup.table
print type(table.renderContents())
The original data returned from the page is a string. BS shows the original encoding as ISO-8859-1. I thought that BS automatically converted everything to Unicode so why is it that when I do this:
table = soup.table
print type(table.renderContents())
..it gives me a string object and not Unicode?
How can i get a Unicode objects from BS?
I'm really, really lost with this. Any help? Thanks in advance.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
originalEncoding正是源编码，因此 BS 在内部将所有内容存储为 unicode 的事实不会改变该值。当您遍历树时，所有文本节点都是 unicode，所有标签都是 unicode 等，除非您以其他方式转换它们（例如使用print、str、美化，或renderContents）。尝试执行以下操作：
不幸的是，到目前为止您所做的所有其他操作都发现 BS 中转换为字符串的方法非常少。
originalEncodingis exactly that - the source encoding, so the fact that BS is storing everything as unicode internally won't change that value. When you walk the tree, all text nodes are unicode, all tags are in unicode, etc., unless you otherwise convert them (say by usingprint,str,prettify, orrenderContents).Try doing something like:
Unfortunately everything else you've done up to this point has found the very few methods in BS that convert to strings.
您可能已经注意到，renderContent 返回（默认情况下）以 UTF-8 编码的字符串，但如果您确实想要表示整个文档的 Unicode 字符串，您也可以执行 unicode(soup) 或使用 unicode( soup.prettify()，“utf-8”）。
相关
As you may have noticed renderContent returns (by default) a string encoded in UTF-8, but if you really want a Unicode string representing the entire document you can also do unicode(soup) or decode the output of renderContents/prettify using unicode(soup.prettify(), "utf-8").
Related