路透社的网络报废库存数据

Question

我是一个编程初学者，并试图从路透社提取关键的度量数据（例如beta）。但是，它总是像空白一样回来。

我的代码是这样的：

from bs4 import BeautifulSoup as bs
import requests
import re

url = 'https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume'
page = requests.get(url)
bs1 = bs(page.text, 'html.parser')

beta=bs1.find_all('th', class_ ='text__text__1FZLe text__dark-grey__3Ml43 text__regular__2N1Xr text__body__yKS5U body__base__22dCE body__body__VgU9Q',text=re.compile('Beta'))
print(beta)

我知道这是不正确的，但我无法弄清楚该怎么办。请帮忙。最终，我想从路透社提取股票的Beta信息。感谢您的帮助！！！

Answer 1

这是收集所需数据的一种方法：

from bs4 import BeautifulSoup as bs 
import requests
import re

url = 'https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume'
page = requests.get(url)
soup = bs(page.text, 'html.parser')

# Locate the Table you wish to scrape
table = soup.select_one('table.table__table__2px_A')

# Locate the Keys and Value for each of the rows
keys = [i.text for i in table.select('tr th') if i]
values = [i.text for i in table.select('tr td') if i]

# Convert the two lists into a dictionary for a neater output
data = dict(zip(keys,values))

这将返回：

{'% Change': '671.00',
 'Brent Crude Oil': '-1.40%Negative',
 'CBOT Soybeans': '1,626.00',
 'Copper': '111.91',
 'Future': '1,805.20',
 'Gold': '-0.57%Negative',
 'Last': '+0.35%Positive'}

Answer 2

您可以使用我以前的答案中的BS4刮擦网站（无需检查JavaScript/JSON），但您可以使用Seleniums功能。

from selenium import webdriver
from bs4 import BeautifulSoup as bs


# Initiate webdriver
driver = webdriver.Firefox()

# Fetch the web page
driver.get('https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume')

# Convert the driver page source to a soup object
soup = bs(driver.page_source, 'html.parser')

# Find the table you want to scrape
table = soup.find('table', attrs={'aria-label':'KeyMetrics'})

# Locate the Keys and Value for each of the rows
keys = [i.text for i in table.select('tbody tr th') if i]
values = [i.text for i in table.select('tbody tr td') if i]

# Convert the two lists into a dictionary for a neater output
data = dict(zip(keys,values))

driver.quit()
print(data)

这将返回：

{'Price Closing Or Last Bid': '699.20', 'Pricing Date': 'Jul 05', '52 Week High': '1,243.25', '52 Week High Date': 'Nov 04', '52 Week Low': '620.50', '52 Week Low Date': 'Jul 08', '10 Day Average Trading Volume': '31.36', '3 Month Average Trading Volume': '602.72', 'Market Capitalization': '724,644.30', 'Beta': '2.13', '1 Day Price Change': '2.55', '5 Day Price Return (Daily)': '-4.84', '13 Week Price Return (Daily)': '-35.93', '26 Week Price Return (Daily)': '-39.18', '52 Week Price Return (Daily)': '2.99', 'Month To Date Price Return (Daily)': '3.83', 'Year To Date Price Return (Daily)': '-33.84', 'Price Relative To S&P500 (4 Week)': '5.95', 'Price Relative To S&P500 (13 Week)': '-24.33', 'Price Relative To S&P500 (26 Week)': '-23.90', 'Price Relative To S&P500 (52 Week)': '16.99', 'Price Relative To S&P500 (YTD)': '-17.69'}