UNIX“time”命令对于基准测试来说足够准确吗?

发布于 2024-12-28 19:46:44 字数 1431 浏览 2 评论 0原文

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

深爱不及久伴 2025-01-04 19:46:44

time 为运行超过一秒的基准测试提供了足够好的时间,否则 exec() 进程所花费的时间与其运行时间相比可能会很大。

但是,在进行基准测试时,您应该注意上下文切换。也就是说,另一个进程可能正在使用 CPU,从而与您的基准竞争 CPU 并增加其运行时间。为了避免与其他进程争用,您应该运行如下所示的基准测试:

sudo chrt -f 99 /usr/bin/time --verbose <benchmark>

或者

sudo chrt -f 99 perf stat -ddd <benchmark>

sudo chrt -f 99 在 FIFO 实时类中以优先级 99 运行您的基准测试,这使您的进程成为最高优先级进程并避免上下文切换(您可以更改 /etc/security/limits.conf,这样就不需要特权进程来使用实时优先级)。

它还使 time 报告所有可用的统计信息,包括基准测试发生的上下文切换次数,通常应为 0,否则您可能需要重新运行基准测试。

perf stat -ddd/usr/bin/time 提供更多信息,并显示每周期指令、分支和缓存未命中等信息

。最好禁用 CPU 频率缩放和提升,以便 CPU 频率在基准测试期间保持恒定,以获得一致的结果。

time produces good enough times for benchmarks that run over one second otherwise the time it took exec()ing a process may be large compared to its run-time.

However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:

sudo chrt -f 99 /usr/bin/time --verbose <benchmark>

Or

sudo chrt -f 99 perf stat -ddd <benchmark>

sudo chrt -f 99 runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your /etc/security/limits.conf so that it doesn't require a privileged process to use real-time priorities).

It also makes time report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.

perf stat -ddd is even more informative than /usr/bin/time and displays such information as instructions-per-cycle, branch and cache misses, etc.

And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.

遗失的美好 2025-01-04 19:46:44

如今,在我看来,没有理由使用 time 来进行基准测试。请改用perf stat。它为您提供了更多有用的信息,并且可以在任意给定次数内重复基准测试过程并对结果进行统计,即计算方差和平均值。这比 time 更可靠,并且使用起来也更简单:

perf stat -r 10 -d <your app and arguments>

-r 10 将运行您的应用 10 次并对其进行统计。 -d 输出更多数据,例如缓存未命中。

因此,虽然 time 对于长时间运行的应用程序来说可能足够可靠,但它绝对不如 perf stat 可靠。用它来代替。

附录:如果你真的想继续使用time,至少不要使用bash内置命令,而是使用详细模式下的实际操作:

/usr/bin/time -v <some command with arguments>

输出是例如:

    Command being timed: "ls"
    User time (seconds): 0.00
    System time (seconds): 0.00
    Percent of CPU this job got: 0%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 1968
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 93
    Voluntary context switches: 1
    Involuntary context switches: 2
    Swaps: 0
    File system inputs: 8
    File system outputs: 0
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0

特别注意它如何能够测量峰值 RSS,如果您想比较补丁对峰值内存消耗的影响,这通常就足够了。即使用该值来比较之前/之后,如果 RSS 峰值显着下降,那么您做对了。

Nowadays, imo, there is no reason to use time for benchmarking purposes. Use perf stat instead. It gives you much more useful information and can repeat the benchmarking process any given number of time and do statistics on the results, i.e. calculate variance and mean value. This is much more reliable and just as simple to use as time:

perf stat -r 10 -d <your app and arguments>

The -r 10 will run your app 10 times and do statistics over it. -d outputs some more data, such as cache misses.

So while time might be reliable enough for long-running applications, it definitely is not as reliable as perf stat. Use that instead.

Addendum: If you really want to keep using time, at least don't use the bash-builtin command, but the real-deal in verbose mode:

/usr/bin/time -v <some command with arguments>

The output is then e.g.:

    Command being timed: "ls"
    User time (seconds): 0.00
    System time (seconds): 0.00
    Percent of CPU this job got: 0%
    Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
    Average shared text size (kbytes): 0
    Average unshared data size (kbytes): 0
    Average stack size (kbytes): 0
    Average total size (kbytes): 0
    Maximum resident set size (kbytes): 1968
    Average resident set size (kbytes): 0
    Major (requiring I/O) page faults: 0
    Minor (reclaiming a frame) page faults: 93
    Voluntary context switches: 1
    Involuntary context switches: 2
    Swaps: 0
    File system inputs: 8
    File system outputs: 0
    Socket messages sent: 0
    Socket messages received: 0
    Signals delivered: 0
    Page size (bytes): 4096
    Exit status: 0

Especially note how this is capable of measuring the peak RSS, which is often enough if you want to compare the effect of a patch on the peak memory consumption. I.e. use that value to compare before/after and if there is a significant decrease in the RSS peak, then you did something right.

遗心遗梦遗幸福 2025-01-04 19:46:44

是的,时间足够准确。而且您只需要运行程序十几次(前提是运行持续时间超过一秒,或者一秒的很大一部分 - 即至少超过 200 毫秒)。当然,对于大多数运行(除了第一次),文件系统会很热(即小文件已经缓存在 RAM 中),因此请考虑到这一点。

您希望 time-d 运行至少持续零点几秒的原因是时间测量的准确性和粒度。不要指望低于百分之一秒的准确度。 (您需要一些特殊的内核选项才能使其达到一毫秒)

在应用程序内部,您可以使用 时钟clock_gettimegettimeofday,
getrusagetimes (它们肯定有 Python 等效项)。

不要忘记阅读 time(7) 手册页。

Yes, time is accurate enough. And you'll need to run only a dozen of times your programs (provided the run lasts more than a second, or a significant fraction of a second - ie more than 200 milliseconds at least). Of course, the file system would be hot (i.e. small files would already be cached in RAM) for most runs (except the first), so take that into account.

the reason you want to have the time-d run to last a few tenths of seconds at least is the accuracy and granularity of the time measurement. Don't expect less than hundredth of second of accuracy. (you need some special kernel option to have it one millisecond)

From inside the application, you could use clock, clock_gettime, gettimeofday,
getrusage, times (they surely have a Python equivalent).

Don't forget to read the time(7) man page.

浮云落日 2025-01-04 19:46:44

是的。 time 命令给出了经过的时间以及消耗的 CPU。后者可能是您应该关注的,除非您正在执行大量 I/O。如果经过的时间很重要,请确保系统在运行测试时没有其他重要活动。

Yes. The time command gives both elapsed time as well as consumed CPU. The latter is probably what you should focus on, unless you're doing a lot of I/O. If elapsed time is important, make sure the system doesn't have other significant activity while running your test.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文