无法使用美丽的汤刮一些桌子

发布于 2025-02-12 10:44:20 字数 1250 浏览 1 评论 0原文

我正在使用BeautifulSoup来刮擦网站链接。总共有9个具有相同类名称的表，但我只能获得链接5表。我应该在代码中进行哪些更改，以便我可以从上面的链接中刮擦所有存在的表？

以下是我使用的代码：

def ScrapeSecScreen():
    options = webdriver.ChromeOptions()
    options.add_argument('--ignore-certificate-errors')
    options.add_argument('--incognito')
    options.add_argument('--headless')
    driver = webdriver.Chrome("C:/Users/pralo/Downloads/DE/chromedriver", chrome_options=options)
    driver.get('https://www.panamacompra.gob.pa/Inicio/v2/#!/vistaPreviaCP?NumLc=2022-0-30-0-08-CL-024792&esap=0&nnc=0&it=1')
    sleep(10)
    sourcecode = driver.execute_script("return document.getElementsByTagName('html')[0].innerHTML")
    # print(sourcecode)
    soup = BeautifulSoup(sourcecode,"html.parser")
    print(soup)
    l1 = []
    tablelist1=soup.findAll('table',{'class':'table table-condensed table-bordered last-line-table'})
    for tr in tablelist1:
        td = tr.find_all('tr')
        row = [tr.text for tr in td]
        l1.append(row)
    print(l1)
ScrapeSecScreen()

原文

I am using BeautifulSoup to scrape the data from the website link.
There are total 9 table with same class name but I am only able to get link 5 tables.
What changes should I make in the code so I can scrape all the tables present from the above link?

Below is the code I have used:

def ScrapeSecScreen():
    options = webdriver.ChromeOptions()
    options.add_argument('--ignore-certificate-errors')
    options.add_argument('--incognito')
    options.add_argument('--headless')
    driver = webdriver.Chrome("C:/Users/pralo/Downloads/DE/chromedriver", chrome_options=options)
    driver.get('https://www.panamacompra.gob.pa/Inicio/v2/#!/vistaPreviaCP?NumLc=2022-0-30-0-08-CL-024792&esap=0&nnc=0&it=1')
    sleep(10)
    sourcecode = driver.execute_script("return document.getElementsByTagName('html')[0].innerHTML")
    # print(sourcecode)
    soup = BeautifulSoup(sourcecode,"html.parser")
    print(soup)
    l1 = []
    tablelist1=soup.findAll('table',{'class':'table table-condensed table-bordered last-line-table'})
    for tr in tablelist1:
        td = tr.find_all('tr')
        row = [tr.text for tr in td]
        l1.append(row)
    print(l1)
ScrapeSecScreen()

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

古镇旧梦 2025-02-19 10:44:20

经检查后，事实证明，只有前五个表与您提到的类匹配。要获取所有表，您可以省略类参数或指定所有表共享的类。

检查此代码线：

soup.find_all('table', class_='table')

Upon inspection, it turns out that only first five tables match the classes you mentioned. To get all tables, you can omit the class argument or specify a class that all the tables share.

Check this line of code: