在 lxml 中迭代时就地编辑树

发布于 2024-11-09 22:50:42 字数 1666 浏览 1 评论 0原文

我正在使用 lxml 来解析 html 并编辑它以生成一个新文档。本质上，我尝试像 javascript DOM 一样使用它 - 我知道这并不是真正的预期用途，但到目前为止它的大部分效果都很好。

目前，我使用 iterdescendants() 来获取可迭代的元素列表，然后依次处理每个元素。

但是，如果在迭代期间删除某个元素，则仍会考虑其子元素，因为删除不会影响迭代，正如您所期望的那样。为了获得我想要的结果，这个 hack 有效：

from lxml.html import fromstring, tostring
import urllib2
import re

html = '''
<html>
<head>
</head>

<body>
    <div>
        <p class="unwanted">This content should go</p>
        <p class="fine">This content should stay</p>
    </div>

    <div id = "second" class="unwanted">
        <p class = "alreadydead">This content should not be looked at</p>
        <p class = "alreadydead">Nor should this</>
        <div class="alreadydead">
            <p class="alreadydead">Still dead</p>
        </div>
    </div>

    <div>
        <p class="yeswanted">This content should also stay</p>
    </div>
</body>

for element in allElements:
   s = "%s%s" % (element.get('class', ''), element.get('id', ''))        
   if re.compile('unwanted').search(s):
       for i in range(len(element.findall('.//*'))):
           allElements.next()
       element.drop_tree()

print tostring(page.body)

输出：

<body>
    <div>

        <p class="yeswanted">This content should stay</p>
    </div>



    <div>
        <p class="yeswanted">This content should also stay</p>
    </div>
</body>

这感觉像是一个令人讨厌的黑客 - 有没有更明智的方法来使用库来实现这一点？

原文

I am using lxml to parse html and edit it to produce a new document. Essentially, I'm trying to use it somewhat like the javascript DOM - I know this is not really the intended use, but much of it works well so far.

Currently, I use iterdescendants() to get a iterable list of elements and then deal with each in turn.

However, if an element is dropped during the iteration, its children are still considered, since the dropping does not affect the iteration, as you would expect. In order to get the results I want, this hack works:

from lxml.html import fromstring, tostring
import urllib2
import re

html = '''
<html>
<head>
</head>

<body>
    <div>
        <p class="unwanted">This content should go</p>
        <p class="fine">This content should stay</p>
    </div>

    <div id = "second" class="unwanted">
        <p class = "alreadydead">This content should not be looked at</p>
        <p class = "alreadydead">Nor should this</>
        <div class="alreadydead">
            <p class="alreadydead">Still dead</p>
        </div>
    </div>

    <div>
        <p class="yeswanted">This content should also stay</p>
    </div>
</body>

for element in allElements:
   s = "%s%s" % (element.get('class', ''), element.get('id', ''))        
   if re.compile('unwanted').search(s):
       for i in range(len(element.findall('.//*'))):
           allElements.next()
       element.drop_tree()

print tostring(page.body)

This outputs:

<body>
    <div>

        <p class="yeswanted">This content should stay</p>
    </div>



    <div>
        <p class="yeswanted">This content should also stay</p>
    </div>
</body>

This feels like a nasty hack - is there a more sensible way to achieve this using the library?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

橙幽之幻 2024-11-16 22:50:42

为了简化事情，您可以使用 lxml 对 XPath 中正则表达式的支持找到并杀死不需要的节点，而无需迭代所有后代。

这会产生与您的脚本相同的结果：

EXSLT_NS = 'http://exslt.org/regular-expressions'
XPATH = r"//*[re:test(@class, '\bunwanted\b') or re:test(@id, '\bunwanted\b')]"

tree = lxml.html.fromstring(html)
for node in tree.xpath(XPATH, namespaces={'re': EXSLT_NS}):
    node.drop_tree()
print lxml.html.tostring(tree.body)

To simplify things you can use lxml's support for regular expressions within an XPath to find and kill the unwanted nodes without needing to iterate over all descendants.

This produces the same result as your script:

EXSLT_NS = 'http://exslt.org/regular-expressions'
XPATH = r"//*[re:test(@class, '\bunwanted\b') or re:test(@id, '\bunwanted\b')]"

tree = lxml.html.fromstring(html)
for node in tree.xpath(XPATH, namespaces={'re': EXSLT_NS}):
    node.drop_tree()
print lxml.html.tostring(tree.body)

回复收藏 0 原文

~没有更多了~