(已解决)爬取b站用户信息,爬到第二个用户时selenium报错NoSuchElementException是什么原因?

发布于 2022-09-12 00:48:34 字数 3769 浏览 27 评论 0

自己写的一个简单的爬取bilibili用户信息的小程序:

from selenium import webdriver

def main():
    chrome_driver = 'D:\webdrivers\chromedriver.exe'
    driver = webdriver.Chrome(executable_path=chrome_driver)
    uid = 1
    while uid <= 2:
        driver.get('https://space.bilibili.com/'+str(uid))
        user_name = driver.title[0:-35]
        print('UID:'+str(uid)+' Nickname:'+user_name)
        print('Fans:'+driver.find_element_by_id('n-fs').text)
        print('Likes:'+driver.find_element_by_id('n-bf').text)
        print('--------------------')
        uid = uid+1
    driver.quit()

if __name__ == "__main__":
    main()

但是只能输出uid为1的bishi的粉丝数和点赞数,但之后爬到uid为2的碧诗时,返回:

DevTools listening on ws://127.0.0.1:56145/devtools/browser/0afff0d4-6151-4963-93de-f317e3214dca
UID:1 Nickname:bishi
Fans:6.3万
Likes:2.1万
--------------------
UID:2 Nickname:碧诗
Traceback (most recent call last):
  File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 432, in main
    run()
  File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 316, in run_file
    runpy.run_path(target, run_name='__main__')
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 263, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "e:\CODE\Bili+\v1\main.py", line 19, in <module>
    main()
  File "e:\CODE\Bili+\v1\main.py", line 12, in main
    print('Fans:'+driver.find_element_by_id('n-fs').text)
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="n-fs"]"}
  (Session info: chrome=79.0.3945.130)

而如果对代码简单修改一下,先爬碧诗再爬bishi时,情况一样:

from selenium import webdriver

def main():
    chrome_driver = 'D:\webdrivers\chromedriver.exe'
    driver = webdriver.Chrome(executable_path=chrome_driver)
    uid = 2
    while uid >= 1:
        driver.get('https://space.bilibili.com/'+str(uid))
        user_name = driver.title[0:-35]
        print('UID:'+str(uid)+' Nickname:'+user_name)
        print('Fans:'+driver.find_element_by_id('n-fs').text)
        print('Likes:'+driver.find_element_by_id('n-bf').text)
        print('--------------------')
        uid = uid-1
    driver.quit()

if __name__ == "__main__":
    main()

爬完第二个用户的昵称后就会报错。


备注:问题已解决,应该是这个元素还没加载出来,用一个time.sleep()就好了。

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文