如何刮擦跨度同级跨度的文本？

Question

您好，我正在尝试学习如何进行网络刮擦，因此我首先尝试网络刮擦我的学校菜单。

我遇到了一个问题，如果我无法将菜单项放在跨度类中，而是将单词在跨度类“显示”的同一行中获取。

这是我试图与

from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome(executable_path=chromedriver.exe')#changed this 
driver.get('https://housing.ucdavis.edu/dining/menus/dining-commons/tercero/')
results = []
content = driver.page_source
soups = BeautifulSoup(content, 'html.parser')
element=soups.findAll('span',class_ = 'collapsible-heading-status')
for span in element:
    print(span.text)

我试图将其纳入Span.span.text，但这不会返回我任何东西，所以有人可以给我一些指针，以指示如何在Coldapsible头上提取信息 -状态类。

Answer 1

美味的华夫饼 - 如前所述，他们已经消失了，但是要获得您的目标，方法是通过css selectors使用使用相邻的同胞组合组合：

for e in soup.select('.collapsible-heading-status + span'):
    print(e.text)

或使用。 find_next_sibling（）：

for e in soup.find_all('span',class_ = 'collapsible-heading-status'):
    print(e.find_next_sibling('span').text)

示例

以结构化的方式获取每个信息的

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("https://housing.ucdavis.edu/dining/menus/dining-commons/tercero/")

soup = BeautifulSoup(driver.page_source, 'html.parser')

data = []
    
for e in soup.select('.nutrition'):
    d = {
        'meal':e.find_previous('h4').text,
        'title':e.find_previous('h5').text,
        'name':e.find_previous('span').text,
        'description': e.p.text
        }
    d.update({n.text:n.find_next().text.strip(': ') for n in e.select('h6')})
    data.append(d)
data

：输出

[{'meal': 'Breakfast',
  'title': 'Fresh Inspirations',
  'name': 'Vanilla Chia Seed Pudding with Blueberrries',
  'description': 'Vanilla chia seed pudding with blueberries, shredded coconut, and toasted almonds',
  'Serving Size': '1 serving',
  'Calories': '392.93',
  'Fat (g)': '36.34',
  'Carbohydrates (g)': '17.91',
  'Protein (g)': '4.59',
  'Allergens': 'Tree Nuts/Coconut',
  'Ingredients': 'Coconut milk, chia seeds, beet sugar, imitation vanilla (water, vanillin, caramel color, propylene glycol, ethyl vanillin, potassium sorbate), blueberries, shredded sweetened coconut (desiccated coconut processed with sugar, water, propylene glycol, salt, sodium metabisulfite), blanched slivered almonds'},
 {'meal': 'Breakfast',
  'title': 'Fresh Inspirations',
  'name': 'Housemade Granola',
  'description': 'Crunchy and sweet granola made with mixed nuts and old fashioned rolled oats',
  'Serving Size': '1/2 cup',
  'Calories': '360.18',
  'Fat (g)': '17.33',
  'Carbohydrates (g)': '47.13',
  'Protein (g)': '8.03',
  'Allergens': 'Gluten/Wheat/Dairy/Peanuts/Tree Nuts',
  'Ingredients': 'Old fashioned rolled oats (per manufacturer, may contain wheat/gluten), sunflower seeds, seedless raisins, unsalted butter, pure clover honey, peanut-free mixed nuts (cashews, almonds, sunflower oil and/or cottonseed oil, pecans, hazelnuts, dried Brazil nuts, salt), light brown beet sugar, molasses'},...]