设计压力测试框架
我对一个非常大的产品（考虑操作系统的大小）进行了某种集成/压力测试，最近我和我的团队一直在讨论如何更好地组织我们的测试工作负载。 到目前为止，我们已经满足于将所有（自定义）工作负载应用程序放在一系列批处理类型作业中，每个作业代表一次压力测试运行。 现在，我们的测试运行平均涉及 13 个系统上运行的 100 个以上工作负载，我们认为是时候构建一些更高级的东西了。
我见过很多关于单元测试框架的内容，但对于更高级别的压力类型测试却很少。 有谁知道解决管理大量工作负载问题的常见（或不常见）方法？
现在，我们希望保留每个单独工作负载的数据库，并提供一个前端，根据我们在某一天需要的压力类型，将它们混合并匹配到测试包中，但我们没有任何示例做更高级事情的最佳方法，例如对每个单独的工作负载对系统造成的压力进行排名。
我的大型产品压力测试同事在做什么？ 对我们来说，一些手写的剧本已经不够了。
I do a sort of integration/stress test on a very large product (think operating-system size), and recently my team and I have been discussing ways to better organize our test workloads. Up until now, we've been content to have all of our (custom) workload applications in a series of batch-type jobs each of which represent a single stress-test run. Now that we're at a point where the average test run involves upwards of 100 workloads running across 13 systems, we think it's time to build something a little more advanced.
I've seen a lot out there about unit testing frameworks, but very little for higher-level stress type tests. Does anyone know of a common (or uncommon) way that the problem of managing large numbers of workloads is solved?
Right now we would like to keep a database of each individual workload and provide a front-end to mix and match them into test packages depending on what kind of stress we need on a given day, but we don't have any examples of the best way to do more advanced things like ranking the stress that each individual workload places on a system.
What are my fellow stress testers on large products doing? For us, a few handrolled scripts just won't cut it anymore.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我自己的经验最初是在 IIS 平台上使用 WCAT，后来是使用 JMeter 和 Selenium。
WCAT 和 JMeter 都允许您以路线方式浏览网站、填写表格等，并将该过程记录为脚本。 然后可以单独回放该脚本或模拟多个客户端和多个线程。 随机播放以模拟不稳定和不可预测的使用等。
一旦您知道要做什么，就可以编辑脚本，或者可以手动编写脚本。 WCAT 还允许您回放日志文件，并允许您模拟真实世界的使用情况。
以上两者均安装在PC或服务器上。
Selenium 是一个 FireFox 插件，但以类似的方式记录和回放脚本并允许扩展。
比较困难的是找出您正在测试的场景，然后设计适合它们的测试。 此外，还需要考虑与数据库和其他外部资源的交互。需要花费大量时间查看日志文件，因此良好的图形输出至关重要。
My own experience has been initially on IIS platform with WCAT and latterly with JMeter and Selenium.
WCAT and JMeter both allow you to walk through a web site as a route, completing forms etc. and record the process as a script. The script can then be played back singly or simulating multiple clients and multiple threads. Randomising the play back to simulate lumpy and unpredictable use etc.
The scripts can be edited and or can be written by hand once you know whee you are going. WCAT will let you play back from the log files as well allowing you simulate real world usage.
Both of the above are installed on a PC or server.
Selenium is a FireFox add in but works in a similar way recording and playing back scripts and allowing for scaling up.
Somewhat harder is working out the scenarios you are testing for and then designing the tests to fit them. Also Interaction with the database and other external resouces need to be factored in. Expcet to spend a lot of time looking at log files, so good graphical output is essential.
对我来说最困难的事情是从负载下的服务器收集和组织性能指标。 性能监视器是我的主要工具。 当 Visual Studio Tester 版本出现时，我惊讶地发现使用性能计数器是多么容易。 他们为 Web 服务器、SQL 服务器、ASP.NET 应用程序等预先打包了一个计数器列表。我了解到了一堆我什至不知道其存在的性能计数器。 此外，您也可以收集自己的计数器。 您可以在每次运行后存储指标。 您还可以连接到生产服务器并查看它们今天的感受。 然后我实时看到所有这些图形，我感到自己充满力量！ :) 如果您需要更多负载，您可以获取 VS Load Agents 并创建负载生成设备（或者我应该将其称为僵尸网络）。 与市场上的众多产品相比，它的价格相对便宜。 MS 许可证适用于每个处理器，但不适用于每个并发请求。 这意味着您可以产生硬件能够处理的尽可能多的负载。 平均而言，我能够在双核、2GB 内存计算机上获得大约 3,000 个并发 Web 请求。 此外，您可以将性能测试合并到您的构建中。
当然，它也适用于 Windows。 此外，该工具大约 6000 美元的价格可能有点高，再加上每个额外的负载代理的相同金额。
The most difficult thing for me was to collect and organized performance metrics from servers under load. Performance Monitor was my main tool. When Visual Studio Tester edition came up I was astonished how easy it was to work with performance counters. They pre-packaged a list of counters for a web server, SQL server, ASP.NET application, etc. I learned about a bunch of performance counters I did not know even exist. In addition you can collect your own counters too. You can store metrics after each run. You can also connect to production servers and see how they are feeling today. Then I see all those graphics in real time I feel empowered! :) If you need more load you can get VS Load Agents and create load generating rig (or should I call it botnet). Comparing to over products on the market it is relatively inexpensive. MS license goes per processor but not per concurrent request. That means you can produce as much load as your hardware can handle. On average, I was able to get about 3,000 concurrent web requests on dual-core, 2GB memory computer. In addition you can incorporate performance tests into your builds.
Of course it goes for Windows. Besides the price tag of about $6K for the tool could be a bit high, plus the same amount of money per additional load agent.