(已解决)爬取b站用户信息,爬到第二个用户时selenium报错NoSuchElementException是什么原因?
自己写的一个简单的爬取bilibili用户信息的小程序:
from selenium import webdriver
def main():
chrome_driver = 'D:\webdrivers\chromedriver.exe'
driver = webdriver.Chrome(executable_path=chrome_driver)
uid = 1
while uid <= 2:
driver.get('https://space.bilibili.com/'+str(uid))
user_name = driver.title[0:-35]
print('UID:'+str(uid)+' Nickname:'+user_name)
print('Fans:'+driver.find_element_by_id('n-fs').text)
print('Likes:'+driver.find_element_by_id('n-bf').text)
print('--------------------')
uid = uid+1
driver.quit()
if __name__ == "__main__":
main()
但是只能输出uid为1的bishi的粉丝数和点赞数,但之后爬到uid为2的碧诗时,返回:
DevTools listening on ws://127.0.0.1:56145/devtools/browser/0afff0d4-6151-4963-93de-f317e3214dca
UID:1 Nickname:bishi
Fans:6.3万
Likes:2.1万
--------------------
UID:2 Nickname:碧诗
Traceback (most recent call last):
File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\ptvsd_launcher.py", line 43, in <module>
main(ptvsdArgs)
File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 432, in main
run()
File "c:\Users\YSD\.vscode\extensions\ms-python.python-2020.1.58038\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 316, in run_file
runpy.run_path(target, run_name='__main__')
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 263, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "e:\CODE\Bili+\v1\main.py", line 19, in <module>
main()
File "e:\CODE\Bili+\v1\main.py", line 12, in main
print('Fans:'+driver.find_element_by_id('n-fs').text)
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 360, in find_element_by_id
return self.find_element(by=By.ID, value=id_)
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
return self.execute(Command.FIND_ELEMENT, {
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users\YSD\AppData\Local\Programs\Python\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="n-fs"]"}
(Session info: chrome=79.0.3945.130)
而如果对代码简单修改一下,先爬碧诗再爬bishi时,情况一样:
from selenium import webdriver
def main():
chrome_driver = 'D:\webdrivers\chromedriver.exe'
driver = webdriver.Chrome(executable_path=chrome_driver)
uid = 2
while uid >= 1:
driver.get('https://space.bilibili.com/'+str(uid))
user_name = driver.title[0:-35]
print('UID:'+str(uid)+' Nickname:'+user_name)
print('Fans:'+driver.find_element_by_id('n-fs').text)
print('Likes:'+driver.find_element_by_id('n-bf').text)
print('--------------------')
uid = uid-1
driver.quit()
if __name__ == "__main__":
main()
爬完第二个用户的昵称后就会报错。
备注:问题已解决,应该是这个元素还没加载出来,用一个time.sleep()就好了。
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论