使用 nokogiri 提取 HTML 标签之间的文本

Question

我有这样的 HTML：

<h1> Header is here</h1>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>
  <h2> Next Header 2</h2>
     <p>not interested</p>
     <p>not interested</p>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>

我有一个基本的 Nokogiri CSS 节点搜索，返回

内容，但我找不到如何定位第 N 个关闭的 H2 和下一个打开的 H2 之间的所有文本的示例。我正在使用输出创建 CSV，因此我还想读取文件列表并将 URL 作为第一个结果。

Answer 1

require 'rubygems'
require 'nokogiri'

h = '<h1> Header is here</h1>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>
  <h2> Next Header 2</h2>
     <p>not interested</p>
     <p>not interested</p>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>
'

doc = Nokogiri::HTML(h)

# Specify the range between delimiter tags that you want to extract
# triple dot is used to exclude the end point
# 1...2 means 1 and not 2
EXTRACT_RANGES = [
  2...3,
  4...5
]

# Tags which count as delimiters, not to be extracted
DELIMITER_TAGS = [
  "h1",
  "h2"
]

extracted_text = []

i = 0
# Change /"html"/"body" to the correct path of the tag which contains this list
(doc/"html"/"body").children.each do |el|

  if (DELIMITER_TAGS.include? el.name)
    i += 1
  else
    extract = false
    EXTRACT_RANGES.each do |cur_range|
      if (cur_range.include? i)
        extract = true
        break
      end
    end

    if extract
      s = el.inner_text.strip
      unless s.empty?
        extracted_text << el.inner_text.strip
      end
    end
  end

end

# Print out extracted text (each element's inner text is separated by newlines)
puts extracted_text.join("\n")

Answer 2

有时您可以使用 NodeSet 的 &运算符获取节点之间的信息：

doc.xpath('//h2[1]/following-sibling::p') & doc.xpath('//h2[2]/preceding-sibling::p')

Answer 3

如果开始元素和停止元素具有相同的父元素，则这就像单个 XPath 一样简单。首先，为了清楚起见，我将用一个简化的文档来展示它，然后用您的示例文档来展示它：

XML = "<root>
  <a/><a1/><a2/>
  <b/><b1/><b2/>
  <c/><c1/><c2/>
</root>"

require 'nokogiri'
xml = Nokogiri::XML(XML)

# Find all elements between 'a' and 'c'
p xml.xpath('//*[preceding-sibling::a][following-sibling::c]').map(&:name)
#=> ["a1", "a2", "b", "b1", "b2"]

# Find all elements between 'a' and 'b'
p xml.xpath('//*[preceding-sibling::a][following-sibling::b]').map(&:name)
#=> ["a1", "a2"]

# Find all elements after 'c'
p xml.xpath('//*[preceding-sibling::c]').map(&:name)
#=> ["c1", "c2"]

现在，这是您的用例（按索引查找）：

HTML = "<h1> Header is here</h1>
  <h2>Header 2 is here</h2>
     <p>Extract me!</p>
     <p>Extract me too!</p>
  <h2> Next Header 2</h2>
     <p>not interested</p>
     <p>not interested</p>
  <h2>Header 2 is here</h2>
     <p>Extract me three!</p>
     <p>Extract me four!</p>"

require 'nokogiri'
html = Nokogiri::HTML(HTML)

# Find all elements between the first and second h2s
p html.xpath('//*[preceding-sibling::h2[1]][following-sibling::h2[2]]').map(&:content)
#=> ["Extract me!", "Extract me too!"]

# Find all elements between the third h2 and the end
p html.xpath('//*[preceding-sibling::h2[3]]').map(&:content)
#=> ["Extract me three!", "Extract me four!"]

Answer 4

这里不是 XPath 解决方案，而是一个简单（幼稚）的实现，它假设开始和停止元素共享相同的父元素，并允许独立指定开始和停止的 XPath：

HTML = "<h1>Header is here</h1>
  <h2>Header 2 is here</h2>
     <p>Extract me!</p>
     <p>Extract me too!</p>
  <h2> Next Header 2</h2>
     <p>not interested</p>
     <p>not interested</p>
  <h2>Header 2 is here</h2>
     <p>Extract me three!</p>
     <p>Extract me four!</p>"

require 'nokogiri'    
class Nokogiri::XML::Node
  # Naive implementation; assumes found elements will share the same parent
  def content_between( start_xpath, stop_xpath=nil )
    node = at_xpath(start_xpath).next_element
    stop = stop_xpath && at_xpath(stop_xpath)
    [].tap do |content|
      while node && node!=stop
        content << node
        node = node.next_element
      end
    end
  end
end

html = Nokogiri::HTML(HTML)
puts html.content_between('//h2[1]','//h2[2]').map(&:content)
#=> Extract me!
#=> Extract me too!
puts html.content_between('//h2[3]').map(&:content)
#=> Extract me three!
#=> Extract me four!

Answer 5

此代码可能对您有帮助，但它仍然需要有关标签位置的更多信息（如果需要提取的信息位于某些标签之间，那就更好了）

require 'rubygems'
require 'nokogiri'
require 'pp'

html = '<h1> Header is here</h1>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>
  <h2> Next Header 2</h2>
     <p>not interested</p>
     <p>not interested</p>
  <h2>Header 2 is here</h2>
     <p> Extract me!</p>
     <p> Extract me too!</p>
';

doc = Nokogiri::HTML(html);

doc.xpath("//p").each do |el|
  pp el
end