下载管理器:如何重新构建由多个连接获取的块
所以我正在开发自己的下载管理器用于教育目的。我有多个连接/线程下载文件,每个连接都适用于文件的特定范围。现在,在他们都获取了块之后,我不确切知道如何将这些块放在一起来重新制作原始文件。
我做了什么:
首先,我在“wb”模式下创建了一个临时文件,并允许每个连接/线程转储它们的块。但每次连接执行此操作时,它都会覆盖以前保存的块。我认为这是因为我使用了“wb”文件描述符。我将其更改为“ab”,但我无法再执行seek()操作
我正在寻找什么:
我需要一种优雅的方式来将此块重新打包到原始文件中。我想知道其他下载管理器是如何做到的。
提前致谢。
so i am developing my own download manager for educational purpose. I have multiple connections/threads downloading a file, each connection works on a particular range of the file. Now after they have all fetched their chunks, i dont exact know how to i bring this chunks together to re-make the original file.
What i did:
First, i created a temporary file in 'wb' mode, and allowed each connections/threads to dump their chunks. But everytime a connection does this, it overwrites previously saved chunks. I figured this was because i used the 'wb' file descriptor. I changed it to 'ab', but i can no longer perform seek() operations
What i am looking for:
I need an elegant way of re-packaging this chunk to the original file. I would like to know how other download managers do it.
Thank in advance.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
您需要将块写入不同的临时文件,然后按原始顺序将它们连接起来。如果为所有线程打开一个文件,则应按顺序访问该文件以保留正确的数据顺序,这会丢弃线程使用,因为线程应等待前一个文件。顺便说一句,您应该以
wb
模式打开文件。You need to write chunks it different temporary files and then join them in the original order. If you open one file for all the threads, you should make the access to it sequential to preserve to correct order of data, which discards thread usage since a thread should wait for the previous one. BTW, you should open files in
wb
mode.您做得很好:
seek()
和write()
。那应该有效!现在,如果您想要一个更清晰的结构,而不需要那么多线程在文件上到处移动,您可能需要考虑使用下载器线程和磁盘写入线程。最后一个可能会一直休眠,直到被其他一个唤醒,向磁盘写入一些 kb,然后返回休眠状态。
You were doing it just fine:
seek()
andwrite()
. That should work!Now, if you want a cleaner structure, without so many threads moving their hands all over a file, you might want to consider having downloader threads and a disk-writing thread. This last one may just sleep until woken by one of the others, write some kb to disk, and go back to sleep.