网络刮擦UL LI标签

Question

我正在尝试刮擦UL＆amp; Capterra产品页面的LI标签。我想在单独的变量中获取并存储的信息是“位于'country”，“ url地址”和产品功能的信息。

目前，我只知道如何为UL＆amp中的所有内容打印文本。李，不是具体的。

代码：

from selenium import webdriver
from selenium.webdriver.common.by import By
from webdriver_manager.firefox import GeckoDriverManager
import requests

driver = webdriver.Firefox()
driver.get("https://www.capterra.com/p/81310/AMCS/")

companyProfile = bs(driver.page_source, 'html.parser')

url = companyProfile.find("ul", class_="nb-type-md nb-list-undecorated undefined").text

features = companyProfile.find("div", class_="nb-col-count-1 sm:nb-col-count-2 md:nb-col-count-3 nb-col-gap-xl nb-my-0 nb-mx-auto").text 

print(url)
print(features)

driver.close()

输出：

AMCSLocated in United StatesFounded in 2004http://www.amcsgroup.com/
Billing & InvoicingBrokerage ManagementBuy / Sell TicketingContainer ManagementCustomer AccountsCustomer DatabaseDispatch ManagementElectronics RecyclingEquipment TrackingFingerprint ScanningID ScanningIntegrated CamerasInventory ManagementInventory TrackingLogistics Management

如何仅获得URL和国家 /地区，如何整齐地获得功能？

---

我能够通过：

url = driver.find_element(By. XPATH, "//*[starts-with(., 'http')]").text

location = driver.find_element(By. XPATH, "//*[starts-with(., 'Located in')]").text

仍在寻找功能的解决方案来获取URL和位置。

Answer 1

以下代码将与ul＆gt; li标签

soup = BeautifulSoup(driver.page_source, 'html.parser')

for li in soup.find_all('ul',class_="nb-type-md nb-list-undecorated undefined"):
    name = li.select_one('[class="nb-type-md nb-list-undecorated undefined"] li:nth-child(1) > span').get_text()
    location = li.select_one('[class="nb-type-md nb-list-undecorated undefined"] li:nth-child(2) > span').get_text()
    year = li.select_one('[class="nb-type-md nb-list-undecorated undefined"] li:nth-child(3) > span').get_text()
    link = li.select_one('[class="nb-type-md nb-list-undecorated undefined"] li:nth-child(4) > span').get_text()

    print(name)
    print(location)
    print(year)
    print(link)

输出：

AMCS
Located in United States 
Founded in 2004
http://www.amcsgroup.com/

更新：

li=[x.get_text() for x in soup.select('[class="nb-type-md nb-list-undecorated undefined"] li span')]
print(li) 

输出：

['AMCS', 'Located in United States', 'Founded in 2004', 'http://www.amcsgroup.com/']