跨多台机器运行集成测试
我有一个 n 层应用程序，不同层运行在不同的机器上。我需要在这些不同的机器上执行编码集成测试，最好使用 MSTest 框架，因为其他所有东西都是用它编写的。
存在一个编排问题，在机器“B”上的测试可以运行之前，机器“A”需要设置正确。开箱即用的 VS 似乎无法处理这种情况，所以我需要其他东西。
从表面上看，TFS 实验室管理器似乎是我管理机器、部署代码和运行适当测试所需要的，但问题是代码并不在那里 - 我们使用 Perforce 进行源代码控制。
那么
a) 是否有人成功使用 TFS 实验室管理器使用来自另一个源代码控制系统的代码进行测试部署？如果是这样，你是如何设置的？
或者
b) 是否有任何替代测试框架可以让我将代码部署到多台机器、执行编码测试并收集成功/失败结果？
I have an n-tier application with the different tiers running on different machines. I need to execute coded integration tests on those different machines, preferably using the MSTest framework since that's what everything else is written in.
There's an orchestration problem, in that before the tests on machine "B" can run, machine "A" needs to be set up correctly. VS out of the box doesn't seem to handle that scenario so I need something else.
On the surface TFS lab manager would seem to be what I need to manage the machines, deploy the code and run the appropriate tests but the wrinkle is that the code doesn't live there - we use Perforce for source control.
So
a) Has anyone successfully used TFS lab manager for test deployment using code that came out of another source control system? If so, how did you set it up?
or
b) Are there any alternative testing frameworks that will let me deploy code to multiple machines, execute coded tests and gather success/failure results?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
a) 我还没有使用不同的源代码控制来完成此操作，但我认为使用 TFS Lab Manager 绝对可以实现。 TFS 工作流程和测试控制器/代理模型基于实验室管理器中的 Windows 工作流工作，因此您需要修改构建的工作流（使用 XAML 编辑器或 VS 中的工作流 UI）并将任何 TFS 源代码控制调用替换为您自己的任务集以获取/版本/签入代码。事实上，TFS 实验室工作流程与构建工作流程分离，只能从 UNC 路径或其他路径中获取构建的二进制文件。如果您选择后者，您可以使用 MSBuild 和其他一些驱动程序（例如 Jenkins）构建代码，然后使用 TFS Lab Manager 在每一层上正确执行测试。
b) 相反，您可以完全使用 Jenkins 和 VMWare 插件来启动和关闭虚拟机以进行测试。您可以使用 Powershell 插件和 powershell 远程处理或其他一些 MSTest 插件来执行远程测试。这个选择对我来说似乎更难，但我认为两者都可以。
a) I haven't done this with different source control, but I think it's definitely possible with TFS Lab Manager. The TFS Workflow & Test Controller/Agent model works based on Windows Workflow in Lab manager, so you would want to modify the workflow for the build (using the XAML editor or Workflow UI in VS) and replace any TFS source control calls to your own task set to fetch/version/checkin-out code. In fact the TFS Lab workflow is divorced from the build workflow and can just pick up built binaries from a UNC path or something. If you chose the latter, you could build the code with MSBuild and some other driver like Jenkins and then use TFS Lab Manager to execute the tests on each tier correctly.
b) Conversely, you could use Jenkins entirely and the VMWare plug-ins to spin up and down VMs for testing. You could use the Powershell plug-in and powershell remoting or some other MSTest plugin to execute the remote tests. This option seems harder to me, but I think both could work.
根据您的评论
我希望任何 CI 平台都能以带标签的构建作为输入进行操作。
实际上，关于你的（两个）问题：我认为，除非你投入精力升级到稳定的构建环境，否则你注定要手动执行此操作/通过自编写脚本。
拥有构建服务器确实是好事，如果您想建立一个现代的开发环境，基本上离不开它。
如果您被 TFS Lab Manager 所吸引，那么使用 TFS-Build 是必须的。如果您沿着这条路走下去，您应该考虑将所有内容切换到 TFS - 包括源代码控制。
我想说，使用 Perforce 作为源代码控制&用于构建/测试的 TFS 绝对是可能的，但正确设置构建可能会很麻烦（实际上，与源代码控制相关的所有开箱即用的内容都需要重构为 p4 友好的调用）。
Based on your comments
I would expect any CI platform to operate with a labelled build as input.
Actually on your (both) questions: I think that until you invest the effort to upgrade to a stable Build environment you are doomed to do this by hand/by self-brewed scripts.
Having a build server is a really good thing and, if you are into setting up a modern development environment, you basically can't do without it.
If you 're attracted by TFS Lab Manager, going with TFS-Build is a must. If you go down this path, you should consider toggling everything into TFS - including source control.
I 'd say that using Perforce as source control & TFS for Build/Test is absolutely possible, yet it can be a headache to set up the build properly (literally everything that comes out of the box regarding source control needs to be refactored to p4-friendly calls).
我将用我实际所做的事情来回答我自己的问题：放弃实验室的想法，因为对于手头的工作来说，工作量太大了！
起初看起来很有希望，我确信只要付出足够的努力就有可能实现，但正如其他答案指出的那样，这意味着对实验室工作流程进行大量编辑以实现我的目标 - 远远超出了我所能证明的范围。不仅代码不是用 TFS 构建的，而且我在尝试告诉实验室/测试经理如何在哪里运行哪些测试时也遇到了问题。
最后，我使用 powershell 脚本拼凑了一个中途之家解决方案来设置环境（恢复 hyper-v 快照、复制一些文件并运行一些远程安装）和 VS 中内置的标准 MSTest 框架，用于针对环境。
I'm going to answer my own question with what I actually did: abandoned the lab idea as being just too much work for the job at hand!
It looked promising at first and I'm sure it is possible with sufficient effort but as the other answers noted it would mean substantial edits to the lab workflow to achieve my goal - way more than I can justify. Not only was there a problem with the code not being built with TFS but I also ran into problems trying to tell lab/test manager how to run which tests where.
In the end I cobbled a halfway house solution together using a powershell script to set up the environment (revert hyper-v snapshots, copy a few files around and run some remote installs) and the standard MSTest framework built into VS for executing tests against the environment.