Ruby 基准违规来源

Question

运行此代码：

require 'benchmark'

Benchmark.bm do |x|
  x.report("1+1") {15_000_000.times {1+1}}
  x.report("1+1") {15_000_000.times {1+1}}
  x.report("1+1") {15_000_000.times {1+1}}
  x.report("1+1") {15_000_000.times {1+1}}
  x.report("1+1") {15_000_000.times {1+1}}
end

输出以下结果：

       user     system      total        real
1+1  2.188000   0.000000   2.188000 (  2.250000)
1+1  2.250000   0.000000   2.250000 (  2.265625)
1+1  2.234000   0.000000   2.234000 (  2.250000)
1+1  2.203000   0.000000   2.203000 (  2.250000)
1+1  2.266000   0.000000   2.266000 (  2.281250)

猜测变化是系统环境的结果，但想确认情况确实如此。

Answer 1

“猜测变化是系统环境的结果”，你是对的。

基准不可能始终准确。你没有一台完美的常规机器来总是同时运行某些东西。如果两个数字太接近，则将基准中的两个数字视为相同，如本例所示。

Answer 2

我尝试使用 eval 部分展开循环，虽然它使速度更快，但它使执行时间不太一致！

$VERBOSE &&= false # You do not want 15 thousand "warning: useless use of + in void context" warnings
# large_number = 15_000_000 # Too large! Caused eval to take too long, so I gave up
somewhat_large_number = 15_000
unrolled = "def do_addition\n" + ("1+1\n" * somewhat_large_number) + "end\n" ; nil
eval(unrolled)

require 'benchmark'

Benchmark.bm do |x|
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
  x.report("1+1 partially unrolled") { i = 0; while i < 1000; do_addition; i += 1; end}
end

（

      user     system      total        real
1+1 partially unrolled  0.750000   0.000000   0.750000 (  0.765586)
1+1 partially unrolled  0.765000   0.000000   0.765000 (  0.765586)
1+1 partially unrolled  0.688000   0.000000   0.688000 (  0.703089)
1+1 partially unrolled  0.797000   0.000000   0.797000 (  0.796834)
1+1 partially unrolled  0.750000   0.000000   0.750000 (  0.749962)
1+1 partially unrolled  0.781000   0.000000   0.781000 (  0.781210)
1+1 partially unrolled  0.719000   0.000000   0.719000 (  0.718713)
1+1 partially unrolled  0.750000   0.000000   0.750000 (  0.749962)
1+1 partially unrolled  0.765000   0.000000   0.765000 (  0.765585)
1+1 partially unrolled  0.781000   0.000000   0.781000 (  0.781210)

为了比较的目的，您在我的计算机上的基准给出了

      user     system      total        real
1+1  2.406000   0.000000   2.406000 (  2.406497)
1+1  2.407000   0.000000   2.407000 (  2.484629)
1+1  2.500000   0.000000   2.500000 (  2.734655)
1+1  2.515000   0.000000   2.515000 (  2.765908)
1+1  2.703000   0.000000   2.703000 (  4.391075)

最后一行的实时变化，但不是用户或总计）