它还使 time 报告所有可用的统计信息,包括基准测试发生的上下文切换次数,通常应为 0,否则您可能需要重新运行基准测试。
perf stat -ddd 比 /usr/bin/time 提供更多信息,并显示每周期指令、分支和缓存未命中等信息
。最好禁用 CPU 频率缩放和提升,以便 CPU 频率在基准测试期间保持恒定,以获得一致的结果。
time produces good enough times for benchmarks that run over one second otherwise the time it took exec()ing a process may be large compared to its run-time.
However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:
sudo chrt -f 99 runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your /etc/security/limits.conf so that it doesn't require a privileged process to use real-time priorities).
It also makes time report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.
perf stat -ddd is even more informative than /usr/bin/time and displays such information as instructions-per-cycle, branch and cache misses, etc.
And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.
如今,在我看来,没有理由使用 time 来进行基准测试。请改用perf stat。它为您提供了更多有用的信息,并且可以在任意给定次数内重复基准测试过程并对结果进行统计,即计算方差和平均值。这比 time 更可靠,并且使用起来也更简单:
perf stat -r 10 -d <your app and arguments>
-r 10 将运行您的应用 10 次并对其进行统计。 -d 输出更多数据,例如缓存未命中。
因此,虽然 time 对于长时间运行的应用程序来说可能足够可靠,但它绝对不如 perf stat 可靠。用它来代替。
附录:如果你真的想继续使用time,至少不要使用bash内置命令,而是使用详细模式下的实际操作:
/usr/bin/time -v <some command with arguments>
输出是例如:
Command being timed: "ls"
User time (seconds): 0.00
System time (seconds): 0.00
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1968
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 93
Voluntary context switches: 1
Involuntary context switches: 2
Swaps: 0
File system inputs: 8
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Nowadays, imo, there is no reason to use time for benchmarking purposes. Use perf stat instead. It gives you much more useful information and can repeat the benchmarking process any given number of time and do statistics on the results, i.e. calculate variance and mean value. This is much more reliable and just as simple to use as time:
perf stat -r 10 -d <your app and arguments>
The -r 10 will run your app 10 times and do statistics over it. -d outputs some more data, such as cache misses.
So while time might be reliable enough for long-running applications, it definitely is not as reliable as perf stat. Use that instead.
Addendum: If you really want to keep using time, at least don't use the bash-builtin command, but the real-deal in verbose mode:
/usr/bin/time -v <some command with arguments>
The output is then e.g.:
Command being timed: "ls"
User time (seconds): 0.00
System time (seconds): 0.00
Percent of CPU this job got: 0%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 1968
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 93
Voluntary context switches: 1
Involuntary context switches: 2
Swaps: 0
File system inputs: 8
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
Especially note how this is capable of measuring the peak RSS, which is often enough if you want to compare the effect of a patch on the peak memory consumption. I.e. use that value to compare before/after and if there is a significant decrease in the RSS peak, then you did something right.
Yes, time is accurate enough. And you'll need to run only a dozen of times your programs (provided the run lasts more than a second, or a significant fraction of a second - ie more than 200 milliseconds at least). Of course, the file system would be hot (i.e. small files would already be cached in RAM) for most runs (except the first), so take that into account.
the reason you want to have the time-d run to last a few tenths of seconds at least is the accuracy and granularity of the time measurement. Don't expect less than hundredth of second of accuracy. (you need some special kernel option to have it one millisecond)
是的。 time 命令给出了经过的时间以及消耗的 CPU。后者可能是您应该关注的,除非您正在执行大量 I/O。如果经过的时间很重要,请确保系统在运行测试时没有其他重要活动。
Yes. The time command gives both elapsed time as well as consumed CPU. The latter is probably what you should focus on, unless you're doing a lot of I/O. If elapsed time is important, make sure the system doesn't have other significant activity while running your test.
发布评论
评论(4)
time
为运行超过一秒的基准测试提供了足够好的时间,否则exec()
进程所花费的时间与其运行时间相比可能会很大。但是,在进行基准测试时,您应该注意上下文切换。也就是说,另一个进程可能正在使用 CPU,从而与您的基准竞争 CPU 并增加其运行时间。为了避免与其他进程争用,您应该运行如下所示的基准测试:
或者
sudo chrt -f 99 在 FIFO 实时类中以优先级 99 运行您的基准测试,这使您的进程成为最高优先级进程并避免上下文切换(您可以更改
/etc/security/limits.conf
,这样就不需要特权进程来使用实时优先级)。它还使
time
报告所有可用的统计信息,包括基准测试发生的上下文切换次数,通常应为 0,否则您可能需要重新运行基准测试。perf stat -ddd
比/usr/bin/time
提供更多信息,并显示每周期指令、分支和缓存未命中等信息。最好禁用 CPU 频率缩放和提升,以便 CPU 频率在基准测试期间保持恒定,以获得一致的结果。
time
produces good enough times for benchmarks that run over one second otherwise the time it tookexec()
ing a process may be large compared to its run-time.However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:
Or
sudo chrt -f 99
runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your/etc/security/limits.conf
so that it doesn't require a privileged process to use real-time priorities).It also makes
time
report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.perf stat -ddd
is even more informative than/usr/bin/time
and displays such information as instructions-per-cycle, branch and cache misses, etc.And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.
如今,在我看来,没有理由使用
time
来进行基准测试。请改用perf stat
。它为您提供了更多有用的信息,并且可以在任意给定次数内重复基准测试过程并对结果进行统计,即计算方差和平均值。这比time
更可靠,并且使用起来也更简单:-r 10
将运行您的应用 10 次并对其进行统计。-d
输出更多数据,例如缓存未命中。因此,虽然
time
对于长时间运行的应用程序来说可能足够可靠,但它绝对不如perf stat
可靠。用它来代替。附录:如果你真的想继续使用
time
,至少不要使用bash内置命令,而是使用详细模式下的实际操作:输出是例如:
特别注意它如何能够测量峰值 RSS,如果您想比较补丁对峰值内存消耗的影响,这通常就足够了。即使用该值来比较之前/之后,如果 RSS 峰值显着下降,那么您做对了。
Nowadays, imo, there is no reason to use
time
for benchmarking purposes. Useperf stat
instead. It gives you much more useful information and can repeat the benchmarking process any given number of time and do statistics on the results, i.e. calculate variance and mean value. This is much more reliable and just as simple to use astime
:The
-r 10
will run your app 10 times and do statistics over it.-d
outputs some more data, such as cache misses.So while
time
might be reliable enough for long-running applications, it definitely is not as reliable asperf stat
. Use that instead.Addendum: If you really want to keep using
time
, at least don't use the bash-builtin command, but the real-deal in verbose mode:The output is then e.g.:
Especially note how this is capable of measuring the peak RSS, which is often enough if you want to compare the effect of a patch on the peak memory consumption. I.e. use that value to compare before/after and if there is a significant decrease in the RSS peak, then you did something right.
是的,
时间
足够准确。而且您只需要运行程序十几次(前提是运行持续时间超过一秒,或者一秒的很大一部分 - 即至少超过 200 毫秒)。当然,对于大多数运行(除了第一次),文件系统会很热(即小文件已经缓存在 RAM 中),因此请考虑到这一点。您希望
time
-d 运行至少持续零点几秒的原因是时间测量的准确性和粒度。不要指望低于百分之一秒的准确度。 (您需要一些特殊的内核选项才能使其达到一毫秒)在应用程序内部,您可以使用 时钟,clock_gettime,gettimeofday,
getrusage,times (它们肯定有 Python 等效项)。
不要忘记阅读 time(7) 手册页。
Yes,
time
is accurate enough. And you'll need to run only a dozen of times your programs (provided the run lasts more than a second, or a significant fraction of a second - ie more than 200 milliseconds at least). Of course, the file system would be hot (i.e. small files would already be cached in RAM) for most runs (except the first), so take that into account.the reason you want to have the
time
-d run to last a few tenths of seconds at least is the accuracy and granularity of the time measurement. Don't expect less than hundredth of second of accuracy. (you need some special kernel option to have it one millisecond)From inside the application, you could use clock, clock_gettime, gettimeofday,
getrusage, times (they surely have a Python equivalent).
Don't forget to read the time(7) man page.
是的。 time 命令给出了经过的时间以及消耗的 CPU。后者可能是您应该关注的,除非您正在执行大量 I/O。如果经过的时间很重要,请确保系统在运行测试时没有其他重要活动。
Yes. The time command gives both elapsed time as well as consumed CPU. The latter is probably what you should focus on, unless you're doing a lot of I/O. If elapsed time is important, make sure the system doesn't have other significant activity while running your test.