当网络重新连接时,Python urllib2恢复下载不起作用

发布于 2024-11-28 07:53:14 字数 2428 浏览 0 评论 0原文

我正在使用 urllib2 制作一个恢复下载器,大致基于 这个方法。我可以结束程序并重新启动它,它会从上次停止的地方开始下载,下载的文件大小与一次性下载的文件大小相同。

但是,我在禁用和重新启用网络时对其进行了测试,并且它无法正确下载。文件大小最终比文件应有的长度长,并且文件无法正常工作。有什么我错过的吗,或者这可能是 urllib2 的错误吗?

    import urllib2
    opener = urllib2.build_opener();

    self.count = 0 # Counts downloaded size.
    self.downloading = True
    while (not(self.success) and self.downloading):
        try:
            self.Err = ""
            self._netfile = self.opener.open(self.url)
            self.filesize = float(self._netfile.info()['Content-Length'])

            if (os.path.exists(self.localfile) and os.path.isfile(self.localfile)):
                self.count = os.path.getsize(self.localfile)
            print self.count,"of",self.filesize,"downloaded."
            if self.count >= self.filesize:
                #already downloaded
                self.downloading = False
                self.success = True
                self._netfile.close()
                return

            if (os.path.exists(self.localfile) and os.path.isfile(self.localfile)):
                #File already exists, start where it left off:
                #This seems to corrupt the file sometimes?
                self._netfile.close()
                req = urllib2.Request(self.url)
                print "file downloading at byte: ",self.count
                req.add_header("Range","bytes=%s-" % (self.count))
                self._netfile = self.opener.open(req)
            if (self.downloading): #Don't do it if cancelled, downloading=false.
                next = self._netfile.read(1024)
                self._outfile = open(self.localfile,"ab") #to append binary
                self._outfile.write(next)
                self.readsize = desc(self.filesize) # get size mb/kb
                self.count += 1024
                while (len(next)>0 and self.downloading):
                    next = self._netfile.read(1024)
                    self._outfile.write(next)
                    self.count += len(next)
                self.success = True
        except IOError, e:
            print e
            self.Err=("Download error, retrying in a few seconds: "+str(e))
            try:
                self._netfile.close()
            except Exception:
                pass
            time.sleep(8) #Then repeat

I'm using urllib2 to make a resuming downloader, roughly based on this method. I can end the program and re-start it, and it starts downloading where it left off, downloading the file that ends up the same size as if it were downloaded all at once.

However, I have tested it when disabling and reenabling network, and it doesn't download correctly. The file size ends up longer than the file should be, and the file doesn't work correctly. Is there something I missed, or could this be a urllib2 bug?

    import urllib2
    opener = urllib2.build_opener();

    self.count = 0 # Counts downloaded size.
    self.downloading = True
    while (not(self.success) and self.downloading):
        try:
            self.Err = ""
            self._netfile = self.opener.open(self.url)
            self.filesize = float(self._netfile.info()['Content-Length'])

            if (os.path.exists(self.localfile) and os.path.isfile(self.localfile)):
                self.count = os.path.getsize(self.localfile)
            print self.count,"of",self.filesize,"downloaded."
            if self.count >= self.filesize:
                #already downloaded
                self.downloading = False
                self.success = True
                self._netfile.close()
                return

            if (os.path.exists(self.localfile) and os.path.isfile(self.localfile)):
                #File already exists, start where it left off:
                #This seems to corrupt the file sometimes?
                self._netfile.close()
                req = urllib2.Request(self.url)
                print "file downloading at byte: ",self.count
                req.add_header("Range","bytes=%s-" % (self.count))
                self._netfile = self.opener.open(req)
            if (self.downloading): #Don't do it if cancelled, downloading=false.
                next = self._netfile.read(1024)
                self._outfile = open(self.localfile,"ab") #to append binary
                self._outfile.write(next)
                self.readsize = desc(self.filesize) # get size mb/kb
                self.count += 1024
                while (len(next)>0 and self.downloading):
                    next = self._netfile.read(1024)
                    self._outfile.write(next)
                    self.count += len(next)
                self.success = True
        except IOError, e:
            print e
            self.Err=("Download error, retrying in a few seconds: "+str(e))
            try:
                self._netfile.close()
            except Exception:
                pass
            time.sleep(8) #Then repeat

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

水水月牙 2024-12-05 07:53:15

我在 IOError 处理程序中添加了 self._outfile.close() 和 self._netfile.close() ,这似乎已经修复了它。我猜这个错误是由于再次打开追加而不关闭它而引起的。

I added self._outfile.close() with the self._netfile.close() in the IOError handler, that seems to have fixed it. I guess this error was caused by opening for appending again without closing it.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文