生产者/消费者 - I/O 磁盘
我的磁盘中有一个压缩文件，该文件按块分区。我从磁盘读取一个块，将其解压缩到内存并读取数据。
是否可以创建一个生产者/消费者，一个线程从磁盘恢复压缩块并将其放入队列中，另一个线程解压缩并读取数据？
性能会更好吗？
谢谢！
I have a compressed file in the disk, that a partitioned in blocks. I read a block from disk decompress it to memory and the read the data.
It is possible to create a producer/consumer, one thread that recovers compacted blocks from disk and put in a queue and another thread that decompress and read the data?
Will the performance be better?
Thanks!
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我怀疑解压缩数据的线程将花费大部分时间等待从磁盘读取压缩块的线程。
如果 CPU 密集型解压缩比 IO 密集型从磁盘读取块花费的时间更长，我会感到惊讶。
I suspect that the thread that decompresses the data would spend most of its time waiting for the thread that reads the compacted blocks from the disk.
I'd be surprised if the CPU-bound decompression took longer than the IO-bound reading the blocks from disk.
是的，可以这样设置。您是否会看到性能改进很大程度上取决于机器、您正在处理解压缩数据的确切性质等。如果这不是太麻烦，并且您的数据集很大，我建议您这样做测量看看是否更快。如果不出意外的话，这与利用某种映射缩减框架所需要做的工作类似。
Yes, it's possible to set it up that way. Whether you would see a performance improvement is wildly dependent on the machine, the exact nature of what you're doing with the decompressed data, etc. If it's not too much trouble, and your dataset is substantial, I'd suggest doing it and measuring to see if it's faster. If nothing else, it's similar to the work you'd need to do to leverage some sort of map-reduce framework.