是否应该在每次构建时执行代码覆盖率？
我是棕地应用程序开发的忠实粉丝。毫无疑问，这是一本很棒的书，我会向所有开发人员推荐它。我来这里是因为我明白了书中关于代码覆盖率的要点。在我的新商店，我们使用 Team City 进行自动化构建/持续集成，构建完成大约需要 40 分钟。 《Brownfield》一书讨论了无摩擦开发以及我们希望如何减轻开发人员必须承受的常见负担。这是我在第 130 页读到的内容。
“代码覆盖率：两个进程的价格？ 正如您从清单 5.2 中的示例目标中看到的，您最终得到两个输出文件： 一份包含测试结果，一份包含代码覆盖率结果。这是因为你 实际上正在执行此任务期间的测试。
如果您正在运行，从技术上讲，您不需要在单独的任务中执行测试 代码覆盖率任务。出于这个原因，许多团队将替代自动化的 测试任务的代码覆盖率任务，本质上是在 CI 流程。 CI服务器将编译代码、测试代码并生成代码覆盖率 每次签到的统计数据。
尽管这种方法在概念上没有任何问题，但请注意一些问题 缺点。首先，生成代码覆盖率统计数据会产生开销。什么时候 有很多测试，这种开销可能足以导致摩擦 长时间运行的自动构建脚本的形式。请记住，主要构建 脚本应该尽可能快地运行，以鼓励团队成员经常运行它。如果 运行时间太长，您可能会发现开发人员正在寻找解决方法。
由于这些原因，我们建议单独执行代码覆盖率任务 构建脚本的默认任务。它应该定期运行，也许作为构建文件中的单独计划任务每两周甚至每月执行一次，但我们 不认为该指标有足够的好处来保证额外的开销 它在每次签入时执行。“
这与我当前商店的做法相反，我们每次构建都执行 NCover。我想去找我的领导并要求我们不要这样做，但我能做的最好的就是告诉他”这就是《布朗菲尔德》书中所说的“。我认为这还不够好。所以我依靠你们向我提供有关这个主题的个人经验和建议。谢谢。
I'm a huge fan of Brownfield Application Development. A great book no doubt and I'd recommend it to all devs out there. I'm here because I got to the point in the book about code coverage. At my new shop, we're using Team City for automated builds/continuous integration and it takes about 40 minutes for the build to complete. The Brownfield book talks all about frictionless development and how we want to ease the common burdens that developers have to endure. Here's what I read on page 130..
"Code coverage: Two processes for the price of one?
As you can see from the sample target in listing 5.2, you end up with two output files:
one with the test results and one with the code coverage results. This is because you
actually are executing your tests during this task.
You don’t technically need to execute your tests in a separate task if you’re running
the code coverage task. For this reason, many teams will substitute an automated
code coverage task for their testing task, essentially performing both actions in the
CI process. The CI server will compile the code, test it, and generate code coverage
stats on every check-in.
Although there’s nothing conceptually wrong with this approach, be aware of some
downsides. First, there’s overhead to generating code coverage statistics. When
there are a lot of tests, this overhead could be significant enough to cause friction in
the form of a longer-running automated build script. Remember that the main build
script should run as fast as possible to encourage team members to run it often. If
it takes too long to run, you may find developers looking for workarounds.
For these reasons, we recommend executing the code coverage task separately from
the build script’s default task. It should be run at regular intervals, perhaps as a separate scheduled task in your build file that executes biweekly or even monthly, but we
don’t feel there’s enough benefit to the metric to warrant the extra overhead of having
it execute on every check-in."
This is contrary to the practice at my current shop were we execute NCover per build. I want to go to my lead and request we not do this, but the best I can do is tell him "this is what the Brownfield book says". I don't think that's good enough. So I'm relying on you guys to fill me in with your personal experiences and advice on this topic. Thanks.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
持续集成/自动化构建系统中总是存在两个相互竞争的利益：
您将始终需要在这些相互竞争的利益之间进行权衡并找到平衡。我通常尝试将构建时间控制在 10 分钟以内，并且如果需要超过 20 分钟才能提供有关构建稳定性的任何有意义的反馈，则会认为构建系统已损坏。但这并不需要是测试每种情况的完整构建；稍后可能会运行其他测试或在其他计算机上并行运行以进一步测试系统。
如果您发现构建时间为 40 分钟，我建议您尽快执行以下操作之一：
如果可能的话，我会100％推荐第一个解决方案。然而，有时硬件无法立即可用，我们必须做出牺牲。
代码覆盖率是一个相对稳定的指标，因为代码覆盖率数字在一天内急剧恶化的情况相对较少。因此，如果代码覆盖率需要很长时间才能执行，那么它在每次构建时都发生并不重要。但您仍然应该尝试至少每晚获取一次代码覆盖率数字。夜间构建可以允许花费更长的时间，因为（大概）不会有人等待它们，但它们仍然会定期提供有关项目状态的反馈，并确保不会引入很多不可预见的问题。
也就是说，如果您能够让硬件进行更多分布式或并行构建/测试，您绝对应该走这条路 - 它将确保您的开发人员尽快知道他们是否破坏了某些内容或在系统中引入了问题。构建系统的快速反馈所带来的生产力的提高将很快收回硬件成本。
另外，如果您的构建机器不是持续工作（即有很多时间空闲），那么我建议将其设置为执行以下操作：
这样，您就可以获得快速反馈，而且只要构建机器有能力，就可以为每个构建进行更多扩展的测试。
There are always two competing interests in continuous integration / automated build systems:
You will always need to make tradeoffs and find a balance between these competing interests. I usually try to keep my build times under 10 minutes, and will consider build systems broken if it takes more than about 20 minutes to give any sort of meaningful feedback about the build's stability. But this doesn't need to be a complete build that tests every case; there may be additional tests that are run later or in parallel on other machines to further test the system.
If you are seeing build times of 40 minutes, I would recommend you do one of the following as soon as possible:
I would 100% recommend the first solution if at all possible. However, sometimes the hardware isn't available right away and we have to make sacrifices.
Code coverage is a relatively stable metric, in that it is relatively rare that your code coverage numbers would get dramatically worse within a single day. So if the code coverage is taking a long time to perform, then it's not really critical that it occurs on every build. But you should still try to get code coverage numbers at least once a night. Nightly builds can be allowed to take a bit longer, since there (presumably) won't be anybody waiting on them, but they still provide regular feedback about your project's status and ensure there aren't lots of unforeseen problems being introduced.
That said, if you are able to get the hardware to do more distributed or parallel building/testing, you should definitely go that route - it will ensure that your developers know as soon as possible if they broke something or introduced a problem in the system. The cost of the hardware will quickly pay itself back in the increased productivity that occurs from the rapid feedback of the build system.
Also, if your build machine is not constantly working (i.e. there is a lot of time when it is idle), then I would recommend setting it up to do the following:
That way, you get the quick feedback, but also get the more extended tests for every build, so long as the build machine has the capacity for it.
我不会对如何解决这个问题做出任何假设——你有点本末倒置了。您抱怨构建时间太长，所以这就是我要求解决的问题，而没有关于如何去做的先入为主的观念。对于这个问题还有许多其他潜在的解决方案（更快的机器、不同的流程等），明智的做法是不要排除其中任何一个。
最终，这是一个问题：您的管理层是否重视构建系统的输出，足以证明其所花费的时间是合理的。 （以及您可能采取的任何补救时间消耗的措施是否具有可接受的输出保真度）。
I wouldn't make any presumptions about how to fix this - you're putting the cart before the horse a bit here. You have a complaint that the build takes too long, so that's the issue I would ask to resolve, without preconceived notions about how to do it. There are many other potential solutions to this problem (faster machines, different processes, etc.) and you would be wise not to exclude any of them.
Ultimately this is a question of whether your management values the output of the build system enough to justify the time it takes. (And whether any action you might take to remedy the time consumption has acceptable fidelity in output).
这是每个团队和每个环境的决定。您应该首先确定构建持续时间的阈值，然后在确定后将运行时间较长的进程分解为发生频率较低的进程（最好在 CI 中每天不少于 1 或 2 次）。
This is a per team and per environment decision. You should first determine your threshold for build duration, and then factor out longer running processes into less-frequent occurrences (ideally no fewer than 1 or 2 times a day in CI) once that has been determined.
反对意见似乎是执行所有测试并收集代码覆盖率是昂贵的，并且您不想（好吧，有人不想）为每个构建支付这个价格。
我无法想象为什么您（或某人）不想总是知道承保状态是什么。
如果构建机器没有其他事情可做，那么它是否也这样做也没关系。
如果你的构建机器太忙于构建，也许你通过要求它为太多的主机提供服务而使其过载，或者你正在做太多的构建（为什么这么多的改变？嗯，也许测试不是很好！） 。
如果问题是测试本身确实需要很长时间，您也许可以找到一种优化测试的方法。特别是，您不需要对未更改的代码部分重新运行测试。弄清楚如何做到这一点（并信任它）可能是一个挑战。
一些测试覆盖率工具（例如我们的）使您能够跟踪哪些测试覆盖了代码的哪一部分，并且，如果代码发生更改，则需要重新运行哪些测试。通过一些额外的脚本，您可以简单地重新运行首先受影响的测试；这使您能够尽早/快速获得完整的测试结果，而无需运行所有测试。然后，如果构建出现问题，您会尽快发现。
[如果您很偏执并且不真正信任增量测试过程，您可以运行它们以获得早期反馈，然后继续再次运行所有测试，为您提供完整的结果。]
The objection appears to be that executing all the tests, and collecting code coverage, is expensive, and you don't (well, someone doesn't) want to pay that price for each build.
I cannot imagine why on earth you (or that someone) would not want to always know what the coverage status was.
If the build machine has nothing else to do, then it doesn't matter if it does this too.
If your build machine is too busy doing builds, maybe you've overloaded it by asking it to serve too many masters, or you are doing too many builds (why so many changes? hmm, maybe the tests aren't very good !).
If the problem is that the tests themselves really do take a long time, you can perhaps find a way to optimize the tests. In particular, you shouldn't need to re-run tests for the part of the code that didn't change. Figuring out how to do this (and trusting it) might be a challenge.
Some test coverage tools (such as ours) enable you to track what tests cover which part of the code, and, given a code change, which tests need to be re-run. With some additional scripting, you can simply re-run the tests that are affected first; this enables you to get what amounts to full test results early/fast without running all the tests. Then if there are issues with the build you find out as soon as possible.
[If you are paranoid and don't really trust the incremental testing process, you can run them for the early feedback, and then go on to run all the tests again, giving you full results.]