用python requests包来get百度百科网页报错

Question

代码如下

import requests

headers = {
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36',
    'Connection': 'keep-alive',
    'Host': 'baike.baidu.com',
    'Accept-Encoding': 'gzip, deflate',
    'Accept-Language': 'zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3',
    'Upgrade-Insecure-Requests': 1
}
r = requests.get("https://baike.baidu.com/item/科比·布莱恩特/318773")

print(r.status_code)

运行报错：

 requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

浏览器访问就不会有重定向，难道是百度发现了这是爬虫行为？
求教大家如何解决？

Answer 1

你的代码这行没把headers使用上去

r = requests.get("https://baike.baidu.com/item/科比·布莱恩特/318773", headers=headers)

Answer 2

allow_redirect=false

Answer 3

需要加User-Agent信息 模拟浏览器访问

>>> import requests
>>> headers = {
...             'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)\
...                 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36'
...         }
>>> r = requests.get("https://baike.baidu.com/item/科比·布莱恩特/318773", headers=headers)
>>> r
<Response [200]>