python爬虫程序报错:ConnectionResetError

Question

程序如下，爬取过程中会报错，报错时间随机，有时候爬几百条就报错，有时候爬几千条报错：`
ConnectionResetError: [Errno 54] Connection reset by peer

    def get_page(url):
        headers = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36"
     }
        reqeust = urllib.request.Request(url=url, headers=headers)
     try:
            response = urllib.request.urlopen(reqeust)
            page = response.read().decode('utf-8')
            # print(html)
     except urllib.error.URLError as e:
            if hasattr(e, "code"):
                print(e.code)
            if hasattr(e, "reason"):
                print(e.reason)
     return page
     
     
     File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 33, in get_data
        page = get_page(url)
      File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 21, in get_page
        page = response.read().decode('utf-8')
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 471, in read
        s = self._safe_read(self.length)
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 612, in _safe_read
        data = self.fp.read(amt)
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto
        return self._sock.recv_into(b)

Answer 1

网络不稳定很正常的现象，你可以加个重试
for i in range(3):
response = urllib.request.urlopen(reqeust)