对于单元测试来说，慢到什么程度算太慢？
Michael Feathers 在有效处理遗留代码中的第 13-14 页提到：
需要 1/10 的单元测试 第二个运行是一个缓慢的单元测试...... 如果[单元测试]运行得不快，他们 不是单元测试。
我可以理解为什么如果有 30,000 个测试，那么 1/10 秒就太慢了，因为运行需要接近一个小时。然而，这是否意味着 1/11 秒更好呢？不，不是真的（因为只快了 5 分钟）。因此，严格的快速规则可能并不完美。
因此，当考虑单元测试有多慢才算太慢时，也许我应该重新表述这个问题。 开发人员等待单元测试套件完成多长时间才算太长？
举一个测试速度的例子。看一下几个 MSTest 单元测试持续时间计时：
0.2637638 seconds
0.0589954
0.0272193
0.0209824
0.0199389
0.0088322
0.0033815
0.0028137
0.0027601
0.0008775
0.0008171
0.0007351
0.0007147
0.0005898
0.0004937
0.0004624
0.00045
0.0004397
0.0004385
0.0004376
0.0003329
所有 21 个单元测试的平均值为 0.019785 秒。请注意，最慢的测试是由于它使用 Microsoft Moles 来模拟/隔离文件系统。
因此，在这个示例中，如果我的单元测试套件增长到 10,000 个测试，则运行时间可能超过 3 分钟。
Michael Feathers, in Working Effectively With Legacy Code, on pages 13-14 mentions:
A unit test that takes 1/10th of a
second to run is a slow unit test...
If [unit tests] don't run fast, they
aren't unit tests.
I can understand why 1/10th a second is too slow if one has 30,000 tests, as it would take close to an hour to run. However, does this mean 1/11th of a second is any better? No, not really (as it's only 5 minutes faster). So a hard fast rule probably isn't perfect.
Thus when considering how slow is too slow for a unit tests, perhaps I should rephrase the question. How long is too long for a developer to wait for the unit test suite to complete?
To give an example of test speeds. Take a look at several MSTest unit test duration timings:
0.2637638 seconds
0.0589954
0.0272193
0.0209824
0.0199389
0.0088322
0.0033815
0.0028137
0.0027601
0.0008775
0.0008171
0.0007351
0.0007147
0.0005898
0.0004937
0.0004624
0.00045
0.0004397
0.0004385
0.0004376
0.0003329
The average for all 21 of these unit tests comes to 0.019785 seconds. Note the slowest test is due to it using Microsoft Moles to mock/isolate the file system.
So with this example, if my unit test suite grows to 10,000 tests, it could take over 3 minutes to run.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（6）
我研究过一个这样的项目，其中单元测试的数量使得系统需要很长时间才能测试所有内容。 “太长”意味着您基本上没有将其作为正常开发例程的一部分。
然而，他们所做的是将单元测试分为两部分。关键测试和“其他一切”。
关键测试只需要几秒钟就可以运行，并且只测试系统最关键的部分，其中“关键”这里的意思是“如果这里出了问题，一切都会出错”。
导致整个运行时间过长的测试被归入“其他所有”部分，并且仅在构建服务器上运行。
每当有人将代码提交到源代码控制存储库时，关键测试将再次首先运行，然后在未来几分钟内安排“完整运行”。如果在该时间间隔内没有人签入代码，则会运行完整的测试。当然，他们用的时间不是 30 分钟，而是 8-10 分钟。
这是使用 TeamCity 完成的，因此即使一个构建代理忙于完整的单元测试套件，其他构建代理仍然可以获取正常提交并根据需要经常运行关键单元测试。
I've looked at one such project where the number of unit tests made the system take too long to test everything. "Too long" meaning that you basically didn't do that as part of your normal development routine.
However, what they had done was to categorize the unit tests into two parts. Critical tests, and "everything else".
Critical tests took just a few seconds to run, and tested only the most critical parts of the system, where "critical" here meant "if something is wrong here, everything is going to be wrong".
Tests that made the entire run take too long was relegated to the "everything else" section, and was only run on the build server.
Whenever someone committed code to the source control repository, the critical tests would again run first, and then a "full run" was scheduled a few minutes into the future. If nobody checked in code during that interval, the full tests was run. Granted, they didn't take 30 minutes, more like 8-10.
This was done using TeamCity, so even if one build agent was busy with the full unit test suit, the other build agents could still pick up normal commits and run the critical unit tests as often as needed.
我只参与过测试套件运行至少十分钟的项目。较大的，则更像是几个小时。我们忍气吞声，等待着，因为他们几乎肯定会在你扔给他们的任何东西中至少找到一个问题。这些项目是如此庞大和繁琐。
我想知道这些项目是什么，可以在几秒钟内进行全面测试。
（当项目的单元测试需要几个小时时，完成工作的秘诀是同时处理四到五件事。您在测试套件中添加一组补丁，然后进行任务切换，然后通过当你完成你切换到的事情时，也许你的结果已经回来了。）
I've only ever worked on projects where the test suite took at least ten minutes to run. The bigger ones, it was more like hours. And we sucked it up and waited, because they were pretty much guaranteed to find at least one problem in anything you threw at them. The projects were that big and hairy.
I wanna know what these projects are that can be tested comprehensively in seconds.
(The secret to getting things done when your project's unit tests take hours is to have four or five things you're working on at the same time. You throw one set of patches at the test suite and you task-switch, and by the time you're done with the thing you switched to, maybe your results have come back.)
我的单元测试需要几秒钟才能执行。我有一个方法可以执行非常复杂的计算和数十亿次操作。当我们重构这个棘手且超级快速的方法时，我们使用一些已知的良好值作为单元测试的基础（我们必须优化其中的废话，因为正如我所说，它正在执行数十亿次计算）。
规则并不适用于每个领域/问题空间。
我们不能将此方法“划分”为可以进行单元测试的更小的方法：它是一个微小但非常复杂的方法（使用无法快速重新创建的极其巨大的预计算表等） 。
我们对该方法进行了单元测试。它们是单元测试。他们需要几秒钟的时间来执行。这是一件好事[TM]。
现在，我当然不怀疑您使用像 JUnit 这样的单元测试库来进行非单元测试的事情：例如，我们还使用 JUnit 来测试复杂的多线程场景。这些不是“单元测试”，但你敢打赌 JUnit 仍然统治着这一天:)
I've got unit tests that takes a few seconds to execute. I've got a method which does very complicated computing and billions and billions of operations. There are a few know good values that we use as the basis for unit testing when we refactor this tricky and uber-fast method (which we must optimize the crap out of it because, as I said, it is performing billions and billions of computations).
Rules don't adapt to every domain / problem space.
We can't "divide" this method into smaller methods that we could unit test: it is a tiny but very complicated method (making use of insanely huge precomputed tables that can't be re-created fast enough on the fly etc.).
We have unit tests for that method. They are unit tests. They takes seconds to execute. It is a Good Thing [TM].
Now of course I don't dispute that you use unit testing libraries like JUnit for things that aren't unit testing: for example we also use JUnit to test complex multi-threaded scenario. These ones aren't "unit test" but you bet that JUnit still rules the day :)
编辑请参阅我对另一个答案的评论（链接）。请注意，关于单元测试有很多来回讨论，因此在您决定赞成或反对这个答案之前，请阅读该答案的所有评论。
接下来，使用像 Might-Moose 这样的工具（Mighty-Moose 已被放弃，但还有其他工具），每次签入文件时，它仅运行受代码更改影响的测试（而不是整个测试库）。
EDIT See my comment to another answer (Link). Please note that there was a lot of back and forth about Unit Testing so before you decide to upvote or downvote this answer, please read all the comments on that answer.
Next, use a tool like Might-Moose (Mighty-Moose was abandoned but there are other tools) that only runs the tests affected by your code change (instead of your entire test library) every time you check-in a file.
对于开发人员来说，等待单元测试套件完成多长时间才算太长？
这实际上取决于开发人员愿意等待更改反馈的时间。我想说，如果您开始谈论几分钟，那就太慢了，您可能应该将测试套件分解为单独的测试项目并单独运行它们。
How long is too long for a developer to wait for the unit test suite to complete?
It really depends how long the devs are happy to wait for feedback of their change. I'd say if you start talking minutes than it's too slow and you should probably break up the test suite into individual test projects and run them separately.
那么你的问题是什么？ :-) 我同意，这里真正的衡量标准是开发人员必须等待完整运行单元测试的时间。太长了，他们会在提交代码之前开始偷工减料。我希望看到完整的提交构建只需不到一两分钟，但这并不总是可能的。在我的工作中，提交构建过去需要 8 分钟，人们在提交之前只开始运行其中的一小部分 - 所以我们购买了更强大的机器:-)
So what's your question? :-) I agree, the true metric here is how long developers have to wait for a complete run of the unit tests. Too long and they'll start cutting corners before committing code. I'd like to see a complete commit build take less than a minute or two, but that's not always possible. At my work, a commit build used to take 8 minutes and people just started only running small parts of it before committing - so we bought more powerful machines :-)