fprintf 与 std::ofstream 的性能非常令人惊讶(fprintf 非常慢)
我正在运行一些基准测试,以找到在 C++ 中将大型数组写入文件的最有效方法(在 ASCII 中超过 1Go)。
所以我将 std::ofstream 与 fprintf 进行了比较(请参阅下面我使用的开关),
case 0: {
std::ofstream out(title, std::ios::out | std::ios::trunc);
if (out) {
ok = true;
for (i=0; i<M; i++) {
for (j=0; j<N; j++) {
out<<A[i][j]<<" ";
}
out<<"\n";
}
out.close();
} else {
std::cout<<"Error with file : "<<title<<"\n";
}
break;
}
case 1: {
FILE *out = fopen(title.c_str(), "w");
if (out!=NULL) {
ok = true;
for (i=0; i<M; i++) {
for (j=0; j<N; j++) {
fprintf(out, "%d ", A[i][j]);
}
fprintf(out, "\n");
}
fclose(out);
} else {
std::cout<<"Error with file : "<<title<<"\n";
}
break;
}
而我的巨大问题是 fprintf 似乎比 std::ofstream 慢 12 倍以上。您知道我的代码中问题的根源是什么吗?或者也许 std::ofstream 与 fprintf 相比非常优化?
(还有另一个问题:您知道另一种更快的写入文件的方法吗)
非常感谢
(详细信息:我正在使用 g++ -Wall -O3 进行编译)
I was running some benchmarks to find the most efficient way to write a huge array to a file in C++ (more than 1Go in ASCII).
So I compared std::ofstream with fprintf (see the switch I used below)
case 0: {
std::ofstream out(title, std::ios::out | std::ios::trunc);
if (out) {
ok = true;
for (i=0; i<M; i++) {
for (j=0; j<N; j++) {
out<<A[i][j]<<" ";
}
out<<"\n";
}
out.close();
} else {
std::cout<<"Error with file : "<<title<<"\n";
}
break;
}
case 1: {
FILE *out = fopen(title.c_str(), "w");
if (out!=NULL) {
ok = true;
for (i=0; i<M; i++) {
for (j=0; j<N; j++) {
fprintf(out, "%d ", A[i][j]);
}
fprintf(out, "\n");
}
fclose(out);
} else {
std::cout<<"Error with file : "<<title<<"\n";
}
break;
}
And my huge problem is that fprintf seems to be more thant 12x slower compared to std::ofstream. Do you have an idea of what is the origin of the problem in my code ? Or maybe std::ofstream is very optimized compared to fprintf ?
(and an other question : do you know another faster way to write a file)
Thank you very much
(detail : I was compiling with g++ -Wall -O3)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(5)
fprintf("%d"
需要对格式字符串进行运行时解析,每个整数一次。ostream& 运算符<<(ostream&, int)
由编译器解析,每次编译一次。fprintf("%d"
requires runtime parsing of the format string, once per integer.ostream& operator<<(ostream&, int)
is resolved by the compiler, once per compilation.嗯,fprintf() 确实需要在运行时做更多的工作,因为它必须解析和处理格式字符串。然而,考虑到输出文件的大小,我预计这些差异影响不大,并且预计代码受 I/O 限制。
因此,我怀疑您的基准在某种程度上存在缺陷。
fsync()/sync()
会发生什么?Well,
fprintf()
does have to do a bit more work at runtime, since it has to parse and process the format string. However, given the size of your output file I would expect those differences to be of little consequence, and would expect the code to be I/O bound.I therefore suspect that your benchmark is flawed in some way.
fsync()/sync()
at the end?ofstream中有一个文件缓冲区,这可以减少访问磁盘的次数。另外,fprintf是一个可变参数的函数,它会调用一些va_#函数,但ofstream不会。我认为你可以使用fwrite()或putc()进行测试。
There is a file buffer in the ofstream, this may decrease the times accessing to the disk. in addition, fprintf is a function with variable parameters which will call some va_# functions, but ofstream won't.I think you can use fwrite() or putc() to have a test.
您是否在所显示的代码上游的某个位置设置了sync_with_stdio?
虽然您报告的内容与经验所见相反,但大多数人认为并相信您所看到的应该是常态。 iostream 是类型安全的,而 printf 系列函数是可变参数函数,必须从格式说明符推断 va_list 的类型。
Have you set sync_with_stdio somewhere upstream of the code you have shown?
While what you report is opposite that of what is empirically seen, most people think and believe what you see should be the norm. iostreams are type-safe, whereas the printf family of functions are variadic functions that have to infer the types of the va_list from the format specifier.
怎么样:
编译器无法解析它。
What about :
The compiler is not able to resolve it.