美丽的汤问题 - 可以找到正确的问题

Question

我遇到美丽的汤遇到麻烦。我正在尝试刮擦皮划艇，但是当我打印find_all的长度时，它正在返回0。我也将硒与美丽的汤一起使用。

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
from bs4 import BeautifulSoup
from webdriver_manager.chrome import ChromeDriverManager


## Kayak URL
origin = "PIT"
destination = "ARN"
startdate = "2022-12-18"

url = "https://www.kayak.com/flights/" + origin + "-" + destination + "/" +\
      startdate + "?sort=price_a "

## Setting Up Webdriver
driver = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
driver.implicitly_wait(40)
driver.get(url)

soup = BeautifulSoup(driver.page_source, "lxml")
print(len(soup.find_all("span", attrs={'class': 'depart-time base-time'})))

deptimes = soup.find_all("span", attrs={'class': 'depart-time base-time'})
arrtimes = soup.find_all('span', attrs={'class': 'arrival-time base-time'})
meridies = soup.find_all('span', attrs={'class': 'time-meridiem meridiem'})

这就是我试图从皮划艇网站上拿走的东西。

<span class="depart-time base-time">12:45 </span>

Answer 1

使用剧作家 python，您可以做这样的事情，与硒相似。
您需要选择每个框，然后迭代每个元素提取所需的数据。
在这里您有一个示例：

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)

    ## Kayak URL
    origin = "PIT"
    destination = "ARN"
    startdate = "2022-12-18"
    url = "https://www.kayak.com/flights/" + origin + "-" + destination + "/" + \
          startdate + "?sort=price_a "
    page = browser.new_page()
    page.goto(url)
    
    #select all boxes of each flight
    allFlight = page.query_selector_all(".resultInner")
    #extract data from each flight
    for index, flight in enumerate(allFlight, start=1):
        print("Flight nº: "+str(index))
        print("From " + flight.query_selector(".depart-time").inner_text()
              + flight.query_selector(".depart-time+span").inner_text()
              + " To " + flight.query_selector(".arrival-time").inner_text()
              + flight.query_selector(".arrival-time+span").inner_text())
        print("Flight companies:  " + flight.query_selector(".section.times .bottom,.multi-airline-names").inner_text().strip())
   
    browser.close()

输出：

在这里您有 playwright 文档

我希望我能够为您提供帮助。