每夜构建与持续集成：长时间运行的自动化测试
我们有一个大型自动化集成测试套件的“问题”。虽然我们的构建时间是合理的（< 1 小时），但测试通常需要 > 1 小时。 6小时完成。
虽然在我们的构建运行中测试如此大的功能块很棒，但这显然是实现 CI 的障碍，我发现这对于将源代码树保持在“始终可构建”状态非常有帮助。
我回顾了诸如 这个，详细说明了区别。
这引出了几个问题：
CI 是否规定或推荐单元测试自动化与集成测试自动化？我过去听说过“仅限单元”，但在快速搜索中没有找到任何此类声明（或基本原理）。
对于组合构建+自动化测试时间/比率来说，为团队提供有效的 CI 的良好“最佳实践”是什么？我的直觉告诉我这应该是<最坏的情况是 2 小时，并且可能 < 2 小时。 1小时才真正有效。理论上，我们可以将测试分解为并行运行，并且可能在 2 小时内运行它们，但这仍然是 3 小时的运行。
从长时间运行的夜间构建 + 集成测试转向 CI 的最佳方法是什么？我正在考虑仅进行一些骨架单元测试的 CI 构建，并结合继续进行集成测试的夜间构建。
也欢迎任何工具推荐（仅限 Windows 的 C#/C++ 代码库）
We have the "problem" of a large automated suite of integration tests. While our build times are reasonable (< 1 hour), the tests typically take > 6 hours to complete.
While it's great to have this large chunk of functionality tested in our build runs, it obviously is a barrier to implementing CI, which I've found to be a very helpful for keeping source trees in a "always buildable" state.
I've reviewed threads of discussion like this one, which elaborate on the distinctions.
This leads me to a few questions:
Does CI dictate or recommend Unit vs. Integration testing automation? I've heard Unit-only in the past, but am not finding any such statements (or rationale) for this in a quick search.
What is a good "best practice" for combined build + automated test times/ratios to have effective CI for a team? My gut tells me that this should be < 2 hours as a worst case, and probably < 1 hour to be really effective. In theory, we could break up the tests to run in parallel and probably get them running in under 2 hours, but this would still be a 3 hour run.
What's the best way forward from long-running Nightly Builds + Integration Tests to CI? I'm thinking of a CI build with a few skeletal Unit Tests only, in combination with nightly builds that continue with the integration tests.
Any tooling recommendations are also welcome (Windows-only C#/C++ codebase)
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
为什么需要 6 小时？你有多少次测试？单元测试与集成测试的比例是多少？您可能有更多的集成测试，或者您的单元测试并不是真正的单元。您的单元测试是否涉及数据库？这可能就是问题所在。
6个小时是一个很长的时间。上面的文章有一些提示。
Why does it takes 6 hours? How many tests do you have? What are the ratio of the unit-test compared to integrated ones? You probrably have many more integrated tests or your unit-test are not really unit. Are your unit tests touching the DB? This may be the problem.
6 hours is a long long time. The article above has some tips.
这里有一些事情。
一般来说，您将有许多构建，其中一个是编译和构建的。运行单元测试，一种执行此操作并运行本地验收测试，另一种运行集成测试。
您肯定不需要需要一个可以完成所有工作的单个构建。
对我来说，你的构建时间听起来相当长 - 请记住，这里的重点是在出现问题时提供快速反馈。我对您的项目了解不多 - 但我认为您应该将编译和单元测试构建时间缩短到两到三分钟以内。除了非常大的项目之外，这在所有项目中都是完全可以实现的，因此如果您的单元测试需要很长时间，那么是时候开始问为什么了。
6个小时也是很长的时间了。你确定你的测试正在测试正确的东西吗？您是否进行了太多范围广泛的测试？您是否到处使用“sleep()”来弥补您在测试代码中没有很好地建模异步的事实？
您可能应该阅读 Jez Humbles 的《持续交付》一书，并看看《Growing Object Oriented Software》，了解如何编写单元/集成测试。 GOOS使用Java作为实现语言，但所有概念都是相同的。
There are a few things here.
In general you will have a number of builds, one that compiles & runs unit tests, one that does that and runs local acceptance tests, and one that runs integration tests.
You definately don't need a single build that does everything.
Your build times to me sound pretty long - remember that the point here is to give quick feedback that something has gone awry. I don't know much about your project - but i would think that you should look to get your compile and unit test build down to under two to three minutes. This is perfectly achievable, in all but very large projects, so if your unit tests take along time, then its time to start asking why.
6 hours is also a very long time. are you sure that your tests are testing the right stuff? do you have too many wide scope tests? are you using "sleep()" everywhere to makeup for the fact that you haven't modeled asynchrony well in your test code?
You should probably get hold of Jez Humbles book "Continuous Delivery", and take a look at Growing Object Oriented Software as how to write unit / integration tests. GOOS uses Java as an implementation language, but all the concepts are the same.