在Wikipedia锚文本中刮擦标题时，如何忽略Infobox？

Question

我试图在Wikipedia页面上刮擦前20个链接，但我想忽略右侧的Infobox。它具有“表”标签。这是我到目前为止所拥有的，任何帮助将不胜感激。

import requests

response = requests.get("https://en.wikipedia.org/wiki/Wales")
soup = BeautifulSoup(response.text, "html.parser")

all_links = {}
count = 0

IGNORE = ["Wikipedia:", "Category:", "Template:", "Template talk:", "User:",
               "User talk:", "Module:", "Help:", "File:", "Portal:", "#", " "]

content = soup.find('div', {'class':'mw-parser-output'})

for link in content.find_all("a"):
    if count <= 20:
        url = link.get("title", "")
        if not any(url.startswith(x) for x in IGNORE) and url != "":
            count = count + 1
            print(url)
    else:
        break

Answer 1

选择您的元素更具体。

soup.select('div.mw-parser-output a:not(.infobox  a)')

一种方法可能是使用CSS选择器和：not（）作为pseudo class：示例

import requests
from bs4 import BeautifulSoup
response = requests.get("https://en.wikipedia.org/wiki/Wales")
soup = BeautifulSoup(response.text, "html.parser")

all_links = {}
count = 0

IGNORE = ["Wikipedia:", "Category:", "Template:", "Template talk:", "User:",
               "User talk:", "Module:", "Help:", "File:", "Portal:", "#", " "]

for link in soup.select('div.mw-parser-output a:not(.infobox  a)'):
    if count <= 20:
        url = link.get("title", "")
        if not any(url.startswith(x) for x in IGNORE) and url != "":
            count = count + 1
            print(url)
    else:
        break

输出，

Wales (disambiguation)
Welsh language
About this sound
Cymru.ogg
Countries of the United Kingdom
United Kingdom
England
Wales–England border
Severn Estuary
Bristol Channel
Irish Sea
Snowdon
Temperateness
Maritime climate
Cardiff
Welsh people
Celtic Britons
Roman withdrawal from Britain
Celtic nations
Llywelyn ap Gruffudd
Edward I of England