fprintf 与 std::ofstream 的性能非常令人惊讶(fprintf 非常慢)

发布于 2024-12-11 18:17:33 字数 1262 浏览 0 评论 0原文

我正在运行一些基准测试,以找到在 C++ 中将大型数组写入文件的最有效方法(在 ASCII 中超过 1Go)。

所以我将 std::ofstream 与 fprintf 进行了比较(请参阅下面我使用的开关),

    case 0: {
        std::ofstream out(title, std::ios::out | std::ios::trunc);
        if (out) {
            ok = true;
            for (i=0; i<M; i++) {
                for (j=0; j<N; j++) {
                    out<<A[i][j]<<" ";
                }
                out<<"\n";
            }
            out.close();
        } else {
            std::cout<<"Error with file : "<<title<<"\n";
        }
        break;
    }
    case 1: {
        FILE *out = fopen(title.c_str(), "w");
        if (out!=NULL) {
            ok = true;
            for (i=0; i<M; i++) {
                for (j=0; j<N; j++) {
                    fprintf(out, "%d ", A[i][j]);
                }
                fprintf(out, "\n");
            }
            fclose(out);
        } else {
            std::cout<<"Error with file : "<<title<<"\n";
        }
        break;
    }

而我的巨大问题是 fprintf 似乎比 std::ofstream 慢 12 倍以上。您知道我的代码中问题的根源是什么吗?或者也许 std::ofstream 与 fprintf 相比非常优化?

(还有另一个问题:您知道另一种更快的写入文件的方法吗)

非常感谢

(详细信息:我正在使用 g++ -Wall -O3 进行编译)

I was running some benchmarks to find the most efficient way to write a huge array to a file in C++ (more than 1Go in ASCII).

So I compared std::ofstream with fprintf (see the switch I used below)

    case 0: {
        std::ofstream out(title, std::ios::out | std::ios::trunc);
        if (out) {
            ok = true;
            for (i=0; i<M; i++) {
                for (j=0; j<N; j++) {
                    out<<A[i][j]<<" ";
                }
                out<<"\n";
            }
            out.close();
        } else {
            std::cout<<"Error with file : "<<title<<"\n";
        }
        break;
    }
    case 1: {
        FILE *out = fopen(title.c_str(), "w");
        if (out!=NULL) {
            ok = true;
            for (i=0; i<M; i++) {
                for (j=0; j<N; j++) {
                    fprintf(out, "%d ", A[i][j]);
                }
                fprintf(out, "\n");
            }
            fclose(out);
        } else {
            std::cout<<"Error with file : "<<title<<"\n";
        }
        break;
    }

And my huge problem is that fprintf seems to be more thant 12x slower compared to std::ofstream. Do you have an idea of what is the origin of the problem in my code ? Or maybe std::ofstream is very optimized compared to fprintf ?

(and an other question : do you know another faster way to write a file)

Thank you very much

(detail : I was compiling with g++ -Wall -O3)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

山田美奈子 2024-12-18 18:17:33

fprintf("%d" 需要对格式字符串进行运行时解析,每个整数一次。ostream& 运算符<<(ostream&, int) 由编译器解析,每次编译一次。

fprintf("%d" requires runtime parsing of the format string, once per integer. ostream& operator<<(ostream&, int) is resolved by the compiler, once per compilation.

暮光沉寂 2024-12-18 18:17:33

嗯,fprintf() 确实需要在运行时做更多的工作,因为它必须解析和处理格式字符串。然而,考虑到输出文件的大小,我预计这些差异影响不大,并且预计代码受 I/O 限制。

因此,我怀疑您的基准在某种程度上存在缺陷。

  1. 如果重复运行测试,您是否始终会得到 12 倍的差异?
  2. 如果颠倒运行测试的顺序,时间会发生什么变化?
  3. 如果最后调用 fsync()/sync() 会发生什么?

Well, fprintf() does have to do a bit more work at runtime, since it has to parse and process the format string. However, given the size of your output file I would expect those differences to be of little consequence, and would expect the code to be I/O bound.

I therefore suspect that your benchmark is flawed in some way.

  1. Do you consistently get a 12x difference if you run the tests repeatedly?
  2. What happens to the timings if you reverse the order in which you run the tests?
  3. What happens if you call fsync()/sync() at the end?
暖风昔人 2024-12-18 18:17:33

ofstream中有一个文件缓冲区,这可以减少访问磁盘的次数。另外,fprintf是一个可变参数的函数,它会调用一些va_#函数,但ofstream不会。我认为你可以使用fwrite()或putc()进行测试。

There is a file buffer in the ofstream, this may decrease the times accessing to the disk. in addition, fprintf is a function with variable parameters which will call some va_# functions, but ofstream won't.I think you can use fwrite() or putc() to have a test.

与风相奔跑 2024-12-18 18:17:33

您是否在所显示的代码上游的某个位置设置了sync_with_stdio?

虽然您报告的内容与经验所见相反,但大多数人认为并相信您所看到的应该是常态。 iostream 是类型安全的,而 printf 系列函数是可变参数函数,必须从格式说明符推断 va_list 的类型。

Have you set sync_with_stdio somewhere upstream of the code you have shown?

While what you report is opposite that of what is empirically seen, most people think and believe what you see should be the norm. iostreams are type-safe, whereas the printf family of functions are variadic functions that have to infer the types of the va_list from the format specifier.

吹泡泡o 2024-12-18 18:17:33

怎么样:

uint32_t foo;
// <some code setting foo>
std::cout<< std::setfill('0') << std::setw(8) << foo << std<<endl;

编译器无法解析它。

What about :

uint32_t foo;
// <some code setting foo>
std::cout<< std::setfill('0') << std::setw(8) << foo << std<<endl;

The compiler is not able to resolve it.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文