有效的代码检测？
我经常阅读有关某些新框架及其“基准”的声明。我的问题是一般性问题，但针对具体问题：
开发人员应采取什么方法来有效地检测代码以衡量性能？
在阅读基准测试和性能测试时，需要注意哪些可能并不代表真实结果的危险信号？
在阅读基准测试和性能
All too often I read statements about some new framework and their "benchmarks." My question is a general one but to the specific points of:
What approach should a developer take to effectively instrument code to measure performance?
When reading about benchmarks and performance testing, what are some red-flags to watch out for that might not represent real results?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
衡量性能的方法有两种：使用代码检测和使用采样。
我过去使用的商业分析器（Hi-Prof、Rational Quantify、AQTime）使用了代码检测（其中一些还可以使用采样），根据我的经验，这给出了最好、最详细的结果。特别是 Rational Quantity 允许您放大结果、关注子树、删除完整的调用树以模拟改进，...
这些仪器分析器的缺点是：
设置有时还会扭曲低级函数（如内存分配、关键部分等）报告的时间......
我使用的免费分析器（Very Sleepy，Luke Stackwalker）使用采样，这意味着进行快速性能测试要容易得多看看问题出在哪里。这些免费分析器不具备商业分析器的全部功能（尽管我自己为 Very Sleepy 提交了“关注子树”功能），但由于它们速度很快，因此非常有用。
目前，我个人最喜欢的是《非常困》（Very Sleepy），卢克·斯塔克沃克（Luke StackWalker）位居第二。
在这两种情况（检测和采样）中，我的经验是：
There are two methods of measuring performance: using code instrumentation and using sampling.
The commercial profilers (Hi-Prof, Rational Quantify, AQTime) I used in the past used code instrumentation (some of them could also use sampling) and in my experience, this gives the best, most detailed result. Especially Rational Quantity allow you to zoom in on results, focus on sub trees, remove complete call trees to simulate an improvement, ...
The downside of these instrumenting profilers is that they:
The instrumentation also sometimes skews the times reported for low-level functions like memory allocations, critical sections, ...
The free profilers (Very Sleepy, Luke Stackwalker) that I use use sampling, which means that it is much easier to do a quick performance test and see where the problem lies. These free profilers don't have the full functionality of the commercial profilers (although I submitted the "focus on subtree" functionality for Very Sleepy myself), but since they are fast, they can be very useful.
At this time, my personal favorite is Very Sleepy, with Luke StackWalker coming second.
In both cases (instrumenting and sampling), my experience is that:
这取决于你想做什么。
1）如果您想维护一般计时信息，以便对回归保持警惕，则可以使用各种仪器分析器。确保它们测量各种时间，而不仅仅是 CPU 时间。
2) 如果你想找到让软件更快的方法，那是一个截然不同的问题。
您应该将重点放在查找上，而不是测量上。
重要的是，它应该在挂钟时间上采样，而不是 CPU 时间，因为 I/O 造成的时间损失与运算造成的时间损失一样可能。这排除了一些分析器。
它应该仅在您关心时才能够采样，例如在等待用户输入时不能采样。这也排除了一些分析器。
最后，也是非常重要的，是您得到的摘要。
获取每行时间的百分比至关重要。
某行所用时间的百分比是包含该行的堆栈样本的百分比。
即使使用调用图，也不要满足于仅函数计时。
这排除了更多的分析器。
（忘记“自我时间”，忘记调用计数。这些很少有用，而且常常会产生误导。）
您所追求的是发现问题的准确性，而不是测量问题的准确性。这是非常重要的一点。 （您不需要大量样本，尽管它没有什么害处。危害在于您的头脑，让您考虑测量，而不是它在做什么。）
一个很好的工具这是 RotateRight 的 Zoom 分析器。就我个人而言，我依赖手动采样。
It depends what you're trying to do.
1) If you want to maintain general timing information, so you can be alert to regressions, various instrumenting profilers are the way to go. Make sure they measure all kinds of time, not just CPU time.
2) If you want to find ways to make the software faster, that is a distinctly different problem.
You should put the emphasis on the find, not on the measure.
For this, you need something that samples the call stack, not just the program counter (over multiple threads, if necessary). That rules out profilers like gprof.
Importantly, it should sample on wall-clock time, not CPU time, because you are every bit as likely to lose time due to I/O as due to crunching. This rules out some profilers.
It should be able to take samples only when you care, such as not when waiting for user input. This also rules out some profilers.
Finally, and very important, is the summary you get.
It is essential to get per-line percent of time.
The percent of time used by a line is the percent of stack samples containing the line.
Don't settle for function-only timings, even with a call graph.
This rules out still more profilers.
(Forget about "self time", and forget about invocation counts. Those are seldom useful and often misleading.)
Accuracy of finding the problems is what you're after, not accuracy of measuring them. That is a very important point. (You don't need a large number of samples, though it does no harm. The harm is in your head, making you think about measuring, rather than what is it doing.)
One good tool for this is RotateRight's Zoom profiler. Personally I rely on manual sampling.