pd.read_html-找不到表错误-Aiohttp重构
好的,所以我在python中使用yahoo_fin,但是对于我的短期用例(redis/cron)的请求太慢了,我需要一个实施,可以向Yahoo Finance提出许多请求。
我已经将代码从请求转换为aiohttp/asyncio,尽管在等待,但我现在得到了错误在某些请求中找不到表 - 它有所不同吗?我不是正确地实施异步还是被迫使用硒? (现在fwiw更快〜10倍)
在
tables = pd.read_html(requests.get(site, headers=headers).text)
之后...(小摘录要说明)
async with aiohttp.ClientSession() as session:
async with session.get(site, headers=headers) as resp:
text = await resp.text()
tables = pd.read_html(text)
我拨打之后的代码
async def get_ranking(arr, expiration, optionType, symbol):
try:
print(symbol, flush=True)
price = await async_get_live_price(symbol)
chain = await async_get_options_chain(symbol, expiration)
async with ClientSession() as session:
await asyncio.gather(*[get_ranking(arr, expiration, optionType, symbol) for symbol in symbols])
Okay so I am using yahoo_fin in python however the requests were too slow for my short term use case (pre-redis/cron) and I need an implementation that can make many requests to yahoo finance.
I had converted the code from request to aiohttp/asyncio and despite being awaiting I now get the error No Tables Found on some of the requests -- it varies? Did I not implement the async correctly or am I forced to use selenium? (It's ~10x Faster Now fwiw)
BEFORE
tables = pd.read_html(requests.get(site, headers=headers).text)
AFTER
async with aiohttp.ClientSession() as session:
async with session.get(site, headers=headers) as resp:
text = await resp.text()
tables = pd.read_html(text)
I call the AFTER code like so... (small excerpts to illustrate)
async def get_ranking(arr, expiration, optionType, symbol):
try:
print(symbol, flush=True)
price = await async_get_live_price(symbol)
chain = await async_get_options_chain(symbol, expiration)
async with ClientSession() as session:
await asyncio.gather(*[get_ranking(arr, expiration, optionType, symbol) for symbol in symbols])
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论