.NET Web 应用程序的冒烟测试
我不敢相信我是第一个经历这个思考过程的人，所以我想知道是否有人可以帮助我。
目前的情况：开发人员编写网站，运营人员部署它。部署后，开发人员会对其进行冒烟测试，以确保部署顺利进行。
对我来说，这感觉不对，它本质上意味着需要两个人来部署应用程序；在我们的例子中，这两个人位于地球的两端，时区发挥作用，造成严重破坏。但事实是，开发人员知道最小的测试集是什么，并且可能会随着时间的推移而改变（特别是对于我们应用程序的 Web 服务部分）。恕我直言，运营人员（他们自己也会这么说）是需要遵循一组指令的按钮推动者。
手动解决方案是我们记录测试用例，并在每次部署时遵循该文档进行操作。这听起来很痛苦，而且他们可能会将不同的版本部署到不同的环境（特别是 UAT 和生产环境），并且可能需要针对每个环境使用不同的指令集。
最重要的是，我们近期的计划之一是拥有一个自动化的日常部署环境，因此我们必须指示计算机如何部署给定版本的应用程序。我非常想添加有关如何对应用程序进行冒烟测试的说明。
现在，开发人员更擅长为计算机记录指令，而不是为人类记录指令，因此显而易见的解决方案似乎是使用 nUnit 的组合（我知道这些本身不是单元测试，但它是专门构建的测试） runner）以及 Watin 或 Selenium API 来运行明显的浏览器步骤并调用 Web 服务，并向运营人员解释如何运行这些单元测试。我能做到；我已经基本上做到了。
但如果我能让这个过程更简单不是更好吗？
此时，操作人员和计算机必须知道哪组测试与应用程序的哪个版本相关，并告诉 nUnit 运行程序它应该指向哪个基本 URL（例如，www.example.com = v3.1）。 2 或 test.example.com = v3.3)。
如果测试运行器本身有一种方法可以给它一个基本 URL，并让它下载一个 zip 文件，解压它并在运行它在其中找到的任何测试装置之前自动编辑配置文件，那不是更好吗？
有没有一个开源应用程序可以做到这一点？有必要吗？有没有使用 nUnit 以外的解决方案，也许是 Fitnesse？
根据记录，我首先考虑基于 .NET 的工具，因为大多数开发人员主要是 .NET 开发人员，但我们并没有与它结合。如果存在这样的工具，使用其他语言来编写测试，只要有一个可以在 Windows 上运行的测试运行器，我们就会很乐意适应。
I cannot believe I'm the first person to go through this thought process, so I'm wondering if anyone can help me out with it.
Current situation: developers write a web site, operations deploy it. Once deployed, a developer Smoke Tests it, to make sure the deployment went smoothly.
To me this feels wrong, it essentially means it takes two people to deploy an application; in our case those two people are on opposite sides of the planet and timezones come into play, causing havoc. But the fact remains that developers know what the minimum set of tests is and that may change over time (particularly for the web service portion of our app). Operations, with all due respect to them (and they would say this themselves), are button-pushers who need a set of instructions to follow.
The manual solution is that we document the test cases and operations follow that document each time they deploy. That sounds painful, plus they may be deploying different versions to different environments (specifically UAT and Production) and may need a different set of instructions for each.
On top of this, one of our near-future plans is to have an automated daily deploy environment, so then we'll have to instruct a computer as to how to deploy a given version of our app. I would dearly like to add to that instructions for how to smoke test the app.
Now developers are better at documenting instructions for computers than they are for people, so the obvious solution seems to be to use a combination of nUnit (I know these aren't unit tests per se, but it is a built-for-purpose test runner) and either the Watin or Selenium APIs to run through the obvious browser steps and call to the web service and explain to the Operations guys how to run those unit tests. I can do that; I have mostly done it already.
But wouldn't it be nice if I could make that process simpler still?
At this point, the Operations guys and the computer are going to have to know which set of tests relate to which version of the app and tell the nUnit runner which base URL it should point to (say, www.example.com = v3.2 or test.example.com = v3.3).
Wouldn't it be nicer if the test runner itself had a way of giving it a base URL and letting it download say a zip file, unpack it and edit a configuration file automatically before running any test fixtures it found in there?
Is there an open source app that would do that? Is there a need for one? Is there a solution using something other than nUnit, maybe Fitnesse?
For the record, I'm looking at .NET-based tools first because most of the developers are primarily .NET developers, but we're not married to it. If such a tool exists using other languages to write the tests, we'll happily adapt, as long as there is a test runner that works on Windows.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（7）
我曾担任 ASP.NET 应用程序的冒烟测试编写者。我们使用 QuickTest Pro，测试运行的自动化是通过 质量中心 (它被称为测试总监。）。这涉及编写数百个测试脚本来自动实现 Web 浏览器与 Web 应用程序的交互。这些测试用于在我们的生产服务器上推出构建之前对其进行验证。 Quality Center 允许您定义测试机器“池”，以便您以多线程方式运行大量测试脚本。
更简单的冒烟测试是记录应用程序产生的所有错误/异常并针对系统运行蜘蛛。这不会获得非常“深”的代码覆盖率，但冒烟测试并不意味着深度代码覆盖率。此错误日志记录应该是生产应用程序的一部分，以便在出现错误时进行处理。 Bug 总是会从裂缝中溜走，可悲的是，最好的测试人员将是你的用户。
I worked in a smoke test writer for an asp.net application. We used QuickTest Pro, the automation of test runs was done with Quality Center (it was called Test Director.). This involved writing hundreds of test scripts that automate a web browser interacting with the web application. These tests where used validate a build before rolling it out on our production servers. Quality Center allows you to define a "pool" of test machines to allow you to run a large list of test scripts in a multi-threaded manner.
A more simplistic smoke test would be to log all errors/exceptions that the application produces and run a spider against the system. This will not obtain very "deep" code coverage, but smoke tests aren't meant for deep code coverage. This error logging should be apart of the production application to deal with errors as they come up. Bugs will always slip though the cracks and sadly enough the best testers will be your users.
我过去曾使用 Selenium 为 Web 部署进行此类冒烟测试。您可以编写一套测试脚本，然后在不同环境中的同一站点上运行。
I've used Selenium in the past to do these sort of smoke tests for web deployments. You can write a suite of test scripts and then run then against the same site in different environments.
我还对此序列进行了一些思考，并建议采用声明式方法进行部署和验证，请参阅此处了解我的想法，
http://jimblogdog.blogspot.co.uk/2010/10/introducingdeclarative-deployment.html
我还为我的开源项目 Wolfpack 创建了一些插件来自动化此操作整个过程。本质上，您将“部署冒烟测试”打包为 NuGet 包并将其发布到您的私有 NuGet 源。 Wolfpack 将自动检测包的新版本并下载它，以及 NUnit.Runner NuGet 包并解压所有文件。然后，它将使用 NUnit 控制台运行程序静默运行您的测试，并将结果解析为警报，您可以通过电子邮件、咆哮、hipchat 等方式接收该警报。
http://wolfpack.codeplex.com/
http://wolfpackcontrib .codeplex.com/wikipage?title=NUnitDeploymentPublisher
I have also put some thought into this sequence and have proposed taking a declarative approach to deployment and verification, see here for my thoughts,
http://jimblogdog.blogspot.co.uk/2010/10/introducingdeclarative-deployment.html
I have also created some plugins to my open source project Wolfpack to automate this entire process. Essentially you package your "deployment smoke tests" as a NuGet package and publish it to your private NuGet feed. Wolfpack will automatically detect the new version of the package and download it, along with the NUnit.Runner NuGet package and unpack all the files. It will then silently run your tests using the NUnit console runner and parse the results into an alert that you can receive either by email, growl, hipchat etc.
http://wolfpack.codeplex.com/
http://wolfpackcontrib.codeplex.com/wikipage?title=NUnitDeploymentPublisher
Telerik 有一些免费和非免费的 UI 测试工具，任何可能对此有所帮助的人都可以自动运行这些工具。
Telerik has some free and not-free UI testing tools that can be ran in an automated way by anybody that might help with this too.
我不知道您正在使用哪个 VCS，但您可以编写一个解决方案，通过中间服务从 VCS 中提取特定于版本的配置文件。
您可以编写一个 powershell 脚本或应用程序，从 Web 服务或 Web 应用程序下载配置文件，并将测试 URL 作为参数传递。服务器或应用程序将在能够访问 VCS 的计算机上运行，因此它可以返回文件内容。一旦检索到，脚本或应用程序就可以启动测试。
I don't know which VCS you're using, but you could write a solution that pulls a version-specific configuration file from the VCS through an intermediary service.
You could write an powershell script or an application that would download the config file from a web service or web app, passing the test URL as a parameter. The servers or app would be running on a machine with access to VCS, so it could return the file contents. Once retrieved, the script or app could then initiate the tests.
通常，您的 nUnit 测试就足够了，如果它们全部通过，则代码库应该可以正常工作。如果您在通过 nUnit 测试的情况下部署代码，但在网站上遇到失败，那么出于同样的原因，您需要添加一个也失败的额外 nUnit。然后，当您修复代码以使 nUnit 通过时，您就知道您已经解决了已部署代码所存在的问题。因此，大多数自动构建系统可以配置为首先自动运行所有 nUnit 测试，然后如果任何测试失败，则构建“失败”。
Typically, your nUnit tests are sufficient that if they all pass, the code base should be working fine. If you deploy the code, with passing nUnit tests, and encounter a failure on the website, then you need to add an additional nUnit that fails as well, for the same reason. Then, when you fix your code such that the nUnit is passing, you know that you have fixed the issue that the deployed code has. For this reason, most automatic build systems can be configured to automatically run all the nUnit tests first, and then 'fail' the build if any of the tests fail.
在花费了大量时间试图找到一个更简单的解决方案之后，我们最终教导运营团队如何使用 NUnit 的 Gui 运行程序。这比预期的要容易并且工作正常。
After much time wasted trying to make up an easier solution, we eventually tought the ops team how to use NUnit's Gui runner. This was easier than expected and is working fine.