持续集成和自动化测试策略
我在一家使用持续集成 (TeamCity) 的公司工作。每次有人在 CI 软件中进行检查时，都会启动构建并运行所有单元/自动化测试。问题是我们有超过 7000 个单元测试 + 756 个自动化测试（用于测试 JavaScript，因为我们有一个非常复杂的 UI 逻辑来进行计算等）。正如您可以想象的那样，每次有人在所有过程中进行检查时，都需要 2 个多小时才能完成所有步骤（构建单元测试自动化测试），因此我需要等待那么长时间才能获得结果来了解是否我的签入可能破坏了自动化测试或单元测试。最糟糕的情况是，当超过一个人签入某些内容时，TeamCity 就会开始排队构建，在我获得有效结果（已更新）之前，我最多可以等待半天！我们应该采取什么策略来加快这一进程？即使只进行一点更改，运行所有自动化测试是否是最佳实践？
I’m working in a company that uses Continuous Integration (TeamCity). Every time someone does a check in the CI software starts a build and runs all the Unit/Automated Test . The problem is that we got more than 7000 unit tests + 756 automated tests (used to test the JavaScript as we got a very complex UI logic for making calculation etc). As you can imaging every time someone does a check in the all process takes more than 2 hours to go through all the steps (build-unitest-automated test) so that I need to wait that much before I can get a result to understand if my check in has broken perhaps an automated test or a unit test. Worst situation is when more than one people check in something so that TeamCity start queue up the build and before I can get a valid result (udated) I can wait up to half day ! what strategy should we adopt to speed up a bit this process? Is it a best practice run all the automated tests even against a little change?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
我会考虑以两种方式分解你的测试套件 - 目标是让你和你的团队可以签入，去喝杯咖啡，并在回到办公桌前从团队城市获得一些有意义的反馈。
决定每次提交时您真正想要测试的内容，将剩余的测试移至按预定时间间隔运行的套件（每小时、每晚 - 无论适合您的任何内容）。
如果同意运行每个提交的测试集仍然很大 - 打破设置并分布在并行运行的多个节点上。
您可能还想增强您的 CI 机器，具体取决于您的东西的性质，将测试的工作目录放在 tmpfs（RAM 磁盘）中。
I would look at breaking up your test suite in two ways - with the goal of making it so you and your team can check in, go get a cup of coffee and have some meaningful feedback from team city when you get back to your desk.
decide what you really want to test on every commit, move the remaining tests to a suite that runs at a scheduled interval (hourly, nightly - whatever works for you).
If the set of tests agreed upon to run every commit is still large - break that set up and distribute across multiple nodes running in parallel.
You may also want to beef up your CI machine, depending on the nature of your stuff have the working directory for the tests live in tmpfs (RAM disk).
我将谈论理论上，我还没有将其付诸实践，但 CI 的目标是在夏末之前启动并运行。
从我所看到的那些最受我尊重的开发人员的陈述来看，我所看到的关于测试策略的 CI 最常见元素是将测试分为长期运行和短期运行。
然后，您需要配置标准检查来启动短期运行测试，以对解决方案进行基本验证。然后，在夜间构建和部署构建中，您唯一需要运行完整的测试套件来验证回归测试。
旁白/替代答案：由于我还没有为自己设置 CI，所以我从来没有理解 TeamCity 的业务模型，他们根据构建代理进行定价。现在我明白为什么如果您的测试套件需要那么长时间，多个构建代理就真的开始变得重要了，能够同时运行 5 个构建变得更加重要。因此，一种选择可能是花更多的钱，暂时在弹孔上贴上创可贴。
I'm going to talk in theory, I have yet to put it into practice but CI is on my goals to have up and humming by the end of the summer.
From statements I've seen made by the people that have earned the most respect from me in developers the most common element for CI I've seen in regards to the testing strategy is to split your tests into Long Running and Short Running.
Then you would want to configure that standard check ins kick off the short running test for basic validation of the solution. Then on the nightly builds, and for deployment builds is the only time you NEED to run the full test suite to give your validation of regression tests.
Aside/Alternate answer: Seeing as I haven't setup CI for myself yet, I had never understood the TeamCity business model that they making the pricing based on build agents. Now I understand why multiple build agents really start to matter if your test suite takes that long, being able to run 5 builds at once becomes much more important. So one option could be to just spend more money and stick a band-aid on the bullet hole for now.
持续集成最适合与分布式版本控制系统（例如 Git 或 Mercurial。
每个开发人员都可以经常签入其本地存储库，而无需一直触发整个集成和 UI 测试仪式。
一旦某个功能在本地完成，就可以将其签入中央存储库。因此，只有在添加新功能和/或修复时，CI 服务器才会运行所有耗时的测试。
Continuous Integration works best with a distributed version control system like Git or Mercurial.
Every developer can check in often into their local repository without triggering the whole integration and UI testing ceremony all the time.
Once a feature is finished locally, it can be checked in to the central repository. Thus the CI server runs all the time-consuming tests only when new features and/or fixes have been added.
您是否考虑过使用预先测试的提交？如果您运行远程运行构建（未提交到 VCS），则可以确定您没有破坏 VCS 中的任何内容（只是因为您尚未提交）。您可以毫无问题地继续工作。如果构建成功，您可以提交更改（即使您在同一文件中进行了一些其他更改） - 如果您通过 TeamCity 的插件提交，它将准确提交您发送到服务器进行测试的代码。
这样，您不必等到 TeamCity 的构建完成才能继续使用它。
免责声明：我是 TeamCity 开发人员。
Have you considered using pre-tested commits? If you run a remote run build (without committing in to VCS), you can be sure that you didn't break anything in VCS (just because you didn't commit yet). And you can continue working without problems. If the build is successful, you can commit your change (even if you made some other changes in the same files) - if you commit via TeamCity's plugin it will commit exactly the code you sent to the server for the testing.
This way, you don't have to wait until TeamCity's build has finished to continue working with it.
Disclaimer: I'm a TeamCity developer.