网络刮擦标题，设法刮擦链接，但很难使用文件名

Question

如何为每个链接刮擦文件的所有名称？我设法刮擦链接，但似乎无法刮擦文件的名称。对于更多的上下文，我使用了Beautifulsoup并找到所有链接。

Answer 1

尝试：

import requests
from bs4 import BeautifulSoup

url = "https://m-selig.ae.illinois.edu/ads/coord_database.html"

soup = BeautifulSoup(requests.get(url).content, "html.parser")

for a in soup.select('a[href$=".dat"]'):
    link = a["href"]
    name = a.find_next_sibling(text=True).strip(" \\")
    print("{:<30} {}".format(link, name))

打印：

...
coord/wb13535sm.dat            Woody Blanchard WB135/35 R/C sailplane airfoil
coord/wb140.dat                Woody Blanchard WB140/35/FB R/C sailplane airfoil
coord/whitcomb.dat             NASA/Langley Whitcomb integral supercritical airfoil
coord/ys900.dat                YS900 hyrdofoil (Shen / Eppler)
coord/ys915.dat                YS915 hydrofoil (Shen / Eppler)
coord/ys930.dat                YS930 hydrofoil (Shen / Eppler)