性能测试
我们正在使用 VMWare 和 NUnit 开发自动化回归测试。 我们已将测试分为多个步骤，现在我希望看到每个步骤都经过性能回归检查。 像 NUnit 那样简单地计时测试似乎并不可靠。 我计算出的接受系数约为 15%，但我们的步骤有时可能会有所不同，超过 35%。 在这样一个依赖资源的测试环境中，是否有一致的方法来测试性能？ “智能”计时系统是我唯一的选择吗？
We are developing automated regression tests using VMWare and NUnit. We have divided tests into steps and now I would like to see each step be examined for performance regression. Simply timing the tests, as NUnit does, does not seem reliable. I have figured in a acceptance factor of about 15% but our steps can differ sometimes to over 35%. In such a resource dependent test environment is there any consistent way of testing performance? Is a "smart" timing system my only option?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
对于此类性能测试，没有任何系统可以为您提供简单的通过/失败结果。 在现实生活中，改变你的系统可能会让某些事情变得更快，而另一些事情则变得更慢，所以这通常不是“更好”和“不是更好”之间的选择，而是不同类型更好之间的选择。 （当然，您希望避免情况变得更糟。）
我过去为此所做的只是随着时间的推移保留统计数据。 每次运行测试时，将结果以及修订号和测试时间放入 SQL 数据库中。 然后，您可以随时随地绘制它们（最好是在一个小网络小程序中，以便团队中的每个人都可以查看它们）并查看您的性能是否呈上升或下降趋势，或者自特定修订以来性能是否一直在下降。
不过，这里的关键是它必须是一个图。 这样人眼就可以观察并发现趋势。 你可以花一整周的时间尝试提出一种人工智能算法来对数据进行数值分析，但它永远无法击败人类的模式识别能力。
For this sort of performance testing, there's no such thing as a system that will give you a simple pass/fail result. In real life, changing your system is likely to make some things faster and some other things slower, so it's usually not a choice between "better" and "not better", it's a choice between different kinds of better. (Of course, you want to avoid cases where it's strictly worse.)
What I've done for this in the past is to just keep statistics over time. Every time you run your tests, drop the results in a SQL database with the revision number and the test timings. Then you can graph them whenever and however you want (ideally in a little web applet so everyone on the team can review them) and see if your performance is trending up or down, or if performance has been sucking ever since a particular revision.
The key thing here, though, is that it needs to be a graph. That way human eyes can look at it and find the trends. You could spend all week trying to come up with an AI algorithm to analyse the data numerically, but it would never beat a human's pattern-recognition ability.
您可能会研究 Ants Profiler 等工具的可用功能，因为它确实提供了方法执行/运行时间，但我不确定它在重复测试方面提供了什么。
You might look into the features available with a tool such as Ants Profiler as it does give method executing/run times, but I'm not sure what it offers in terms of repeated testing.
关于性能测试，我一直对使用 vmware 或其他虚拟化进程持怀疑态度。 我们过去处理这个问题的方法是让构建的一部分在静态机器上安装最新版本并运行测试。 您应该在虚拟化之外看到更一致的结果。
With respect to performance testing I've been very skeptical of using vmware or other virtualization processes. The way we have handled this in the past is to have part of the build install the latest version on a static machine and run the tests. You should see more consistent results outside of the virtualization.