popen 是否将整个输出加载到内存中或保存在 tmp 文件（在磁盘中）中？
我想用这段代码读取一个非常非常大的压缩文件（解压后为 119.2 GiB）的一部分。
FILE* trace_file;
char gunzip_command[1000];
sprintf(gunzip_command, "gunzip -c %s", argv[i]); // argv[i]: file path
trace_file = popen(gunzip_command, "r");
fread(¤t_cloudsuite_instr, instr_size, 1, trace_file)
c 中的 popen 是否将命令的整个输出加载到内存中？如果没有，popen 是否将命令的整个输出保存在 tmp 文件（在磁盘中）中？可以看到，解压的输出会太大。内存和磁盘都无法容纳它。 我只知道 popen 创建了一个管道。
$ xz -l 649.fotonik3d_s-1B.champsimtrace.xz
Strms Blocks Compressed Uncompressed Ratio Check Filename
1 1 24.1 MiB 119.2 GiB 0.000 CRC64 649.fotonik3d_s-1B.champsimtrace.xz
I want to read part of a very very large compressed file(119.2 GiB if decompressed) with this piece of code.
FILE* trace_file;
char gunzip_command[1000];
sprintf(gunzip_command, "gunzip -c %s", argv[i]); // argv[i]: file path
trace_file = popen(gunzip_command, "r");
fread(¤t_cloudsuite_instr, instr_size, 1, trace_file)
Does popen in c load whole output of the command into memory? If it does not, does popen save the the whole output of the command in a tmp file(in disk)? As you can see, the output of decompressing will be too large. Neither memory nor disk can hold it.
I only know that popen creates a pipe.
$ xz -l 649.fotonik3d_s-1B.champsimtrace.xz
Strms Blocks Compressed Uncompressed Ratio Check Filename
1 1 24.1 MiB 119.2 GiB 0.000 CRC64 649.fotonik3d_s-1B.champsimtrace.xz
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
据我所知，管道有两种实现方式：
在 MS-DOS 上，整个输出被写入磁盘；然后通过读取文件来完成读取。
可能仍然有（鲜为人知的）现代操作系统以这种方式工作。
然而，在大多数情况下，都会为管道保留一定量的内存。
xz命令可以写入数据，直到该内存量已满。如果内存已满，xz将停止，直到内存可用（因为调用popen()的程序读取数据）。如果调用
popen()的程序从管道读取数据，数据将从内存中删除，以便xz可以写入更多数据。当调用
popen()的程序关闭文件句柄时，写入管道不再有任何效果；错误消息报告给xz...There are two implementations of pipes that I know:
On MS-DOS, the whole output was written to disk; reading was then done by reading the file.
It might be that there are still (less-known) modern operating systems that work this way.
However, in most cases, a certain amount of memory is reserved for the pipe.
The
xzcommand can write data until that amount of memory is full. If the memory is full,xzis stopped until memory becomes available (because the program that calledpopen()reads data).If the program that called
popen()reads data from the pipe, data is removed from the memory soxzcan write more data.When the program that called
popen()closes the file handle, writing to the pipe has no more effect; an error message is reported toxz...