pd.read_html-找不到表错误-Aiohttp重构

发布于 2025-01-21 04:18:07 字数 1128 浏览 0 评论 0原文

好的,所以我在python中使用yahoo_fin,但是对于我的短期用例(redis/cron)的请求太慢了,我需要一个实施,可以向Yahoo Finance提出许多请求。

我已经将代码从请求转换为aiohttp/asyncio,尽管在等待,但我现在得到了错误在某些请求中找不到表 - 它有所不同吗?我不是正确地实施异步还是被迫使用硒? (现在fwiw更快〜10倍)

 tables = pd.read_html(requests.get(site, headers=headers).text)

之后...(小摘录要说明)

    async with aiohttp.ClientSession() as session:
        async with session.get(site, headers=headers) as resp:
            text = await resp.text()
            tables = pd.read_html(text)

我拨打之后的代码

async def get_ranking(arr, expiration, optionType, symbol):
    try:
        print(symbol, flush=True)
        price = await async_get_live_price(symbol)
        chain = await async_get_options_chain(symbol, expiration)
async with ClientSession() as session:
        await asyncio.gather(*[get_ranking(arr, expiration, optionType, symbol) for symbol in symbols])

Okay so I am using yahoo_fin in python however the requests were too slow for my short term use case (pre-redis/cron) and I need an implementation that can make many requests to yahoo finance.

I had converted the code from request to aiohttp/asyncio and despite being awaiting I now get the error No Tables Found on some of the requests -- it varies? Did I not implement the async correctly or am I forced to use selenium? (It's ~10x Faster Now fwiw)

BEFORE

 tables = pd.read_html(requests.get(site, headers=headers).text)

AFTER

    async with aiohttp.ClientSession() as session:
        async with session.get(site, headers=headers) as resp:
            text = await resp.text()
            tables = pd.read_html(text)

I call the AFTER code like so... (small excerpts to illustrate)

async def get_ranking(arr, expiration, optionType, symbol):
    try:
        print(symbol, flush=True)
        price = await async_get_live_price(symbol)
        chain = await async_get_options_chain(symbol, expiration)
async with ClientSession() as session:
        await asyncio.gather(*[get_ranking(arr, expiration, optionType, symbol) for symbol in symbols])

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文