First, testing is like security -- you can never be 100% sure you've got it, but each layer adds more confidence and a framework for more easily fixing the problems that remain.
Second, you can break tests into subroutines which themselves can then be tested. When you have 20 similar tests, making a (tested) subroutine means your main test is 20 simple invocations of the subroutine which is much more likely to be correct.
Third, some would argue that TDD addresses this concern. That is, if you just write 20 tests and they pass, you're not completely confident that they are actually testing anything. But if each test you wrote initially failed, and then you fixed it, then you're much more confident that it's really testing your code. IMHO this back-and-forth takes more time than it's worth, but it is a process that tries to address your concern.
A test being wrong is unlikely to break your production code. At least, not any worse than having no test at all. So it's not a "point of failure": the tests don't have to be correct in order for the product to actually work. They might have to be correct before it's signed off as working, but the process of fixing any broken tests does not endanger your implementation code.
You can think of tests, even trivial tests like these, as being a second opinion what the code is supposed to do. One opinion is the test, the other is the implementation. If they don't agree, then you know you have a problem and you look closer.
It's also useful if someone in future wants to implement the same interface from scratch. They shouldn't have to read the first implementation in order to know what Discount means, and the tests act as an unambiguous back-up to any written description of the interface you may have.
That said, you're trading off time. If there are other tests you could be writing using the time you save skipping these trivial tests, maybe they would be more valuable. It depends on your test setup and the nature of the application, really. If the Discount is important to the app, then you're going to catch any bugs in this method in functional testing anyway. All unit testing does is let you catch them at the point you're testing this unit, when the location of the error will be immediately obvious, instead of waiting until the app is integrated together and the location of the error might be less obvious.
By the way, personally I wouldn't use 100 as the price in the test case (or rather, if I did then I'd add another test with another price). The reason is that someone in future might think that Discount is supposed to be a percentage. One purpose of trivial tests like this is to ensure that mistakes in reading the specification are corrected.
[Concerning the edit: I think it's inevitable that an incorrect specification is a point of failure. If you don't know what the app is supposed to do, then chances are it won't do it. But writing tests to reflect the spec doesn't magnify this problem, it merely fails to solve it. So you aren't adding new points of failure, you're just representing the existing faults in code instead of waffle documentation.]
All I see is more possible points of failure, with no real beneficial return, if the discount price is wrong, the test team will still find the issue, how did unit testing save any work?
Unit testing isn't really supposed to save work, it's supposed to help you find and prevent bugs. It's more work, but it's the right kind of work. It's thinking about your code at the lowest levels of granularity and writing test cases that prove that it works under expected conditions, for a given set of inputs. It's isolating variables so you can save time by looking in the right place when a bug does present itself. It's saving that suite of tests so that you can use them again and again when you have to make a change down the road.
I personally think that most methodologies are not many steps removed from cargo cult software engineering, TDD included, but you don't have to adhere to strict TDD to reap the benefits of unit testing. Keep the good parts and throw out the parts that yield little benefit.
Finally, the answer to your titular question "How do you unit test a unit test?" is that you shouldn't have to. Each unit test should be brain-dead simple. Call a method with a specific input and compare it to its expected output. If the specification for a method changes then you can expect that some of the unit tests for that method will need to change as well. That's one of the reasons that you do unit testing at such a low level of granularity, so only some of the unit tests have to change. If you find that tests for many different methods are changing for one change in a requirement, then you may not be testing at a fine enough level of granularity.
Unit tests are there so that your units (methods) do what you expect. Writing the test first forces you to think about what you expect before you write the code. Thinking before doing is always a good idea.
Unit tests should reflect the business rules. Granted, there can be errors in the code, but writing the test first allows you to write it from the perspective of the business rule before any code has been written. Writing the test afterwards, I think, is more likely to lead to the error you describe because you know how the code implements it and are tempted just to make sure that the implementation is correct -- not that the intent is correct.
Also, unit tests are only one form -- and the lowest, at that -- of tests that you should be writing. Integration tests and acceptance tests should also be written, the latter by the customer, if possible, to make sure that the system operates the way it is expected. If you find errors during this testing, go back and write unit tests (that fail) to test the change in functionality to make it work correctly, then change your code to make the test pass. Now you have regression tests that capture your bug fixes.
[EDIT]
Another thing that I have found with doing TDD. It almost forces good design by default. This is because highly coupled designs are nearly impossible to unit test in isolation. It doesn't take very long using TDD to figure out that using interfaces, inversion of control, and dependency injection -- all patterns that will improve your design and reduce coupling -- are really important for testable code.
How does one test a test? Mutation testing is a valuable technique that I have personally used to surprisingly good effect. Read the linked article for more details, and links to even more academic references, but in general it "tests your tests" by modifying your source code (changing "x += 1" to "x -= 1" for example) and then rerunning your tests, ensuring that at least one test fails. Any mutations that don't cause test failures are flagged for later investigation.
You'd be surprised at how you can have 100% line and branch coverage with a set of tests that look comprehensive, and yet you can fundamentally change or even comment out a line in your source without any of the tests complaining. Often this comes down to not testing with the right inputs to cover all boundary cases, sometimes it's more subtle, but in all cases I was impressed with how much came out of it.
When applying Test-Driven Development (TDD), one begins with a failing test. This step, that might seem unecessary, actually is here to verify the unit test is testing something. Indeed, if the test never fails, it brings no value and worse, leads to wrong confidence as you'll rely on a positive result that is not proving anything.
When following this process strictly, all ''units'' are protected by the safety net the unit tests are making, even the most mundane.
Assert.IsEqual(p.DiscountPrice,90);
There is no reason the test evolves in that direction - or I'm missing something in your reasoning. When the price is 100 and the discount 20, the discount price is 80. This is like an invariant.
Now imagine your software needs to support another kind of discount based on percentage, perhaps depending on the volume bought, your Product::DiscountPrice() method may become more complicated. And it is possible that introducing those changes breaks the simple discount rule we had initially. Then you'll see the value of this test which will detect the regression immediately.
Red - Green - Refactor - this is to remember the essence of the TDD process.
Red refers to JUnit red bar when a tests fails.
Green is the color of JUnit progress bar when all tests pass.
Refactor under green condition: remove any dupliation, improve readability.
Now to address your point about the "3-4 layers above the code", this is true in a traditional (waterfall-like) process, not when the development process is agile. And agile is the world where TDD is coming from ; TDD is the cornerstone of eXtreme Programming.
Agile is about direct communication rather than thrown-over-the-wall requirement documents.
While, I am all for unit testing, I sometimes wonder if this form of test first development is really beneficial...
Small, trivial tests like this can be the "canary in the coalmine" for your codebase, alerting of danger before it's too late. The trivial tests are useful to keep around because they help you get the interactions right.
For example, think about a trivial test put in place to probe how to use an API you're unfamiliar with. If that test has any relevance to what you're doing in the code that uses the API "for real" it's useful to keep that test around. When the API releases a new version and you need to upgrade. You now have your assumptions about how you expect the API to behave recorded in an executable format that you can use to catch regressions.
...[I]n a real process, you have 3-4 layers above your code (Business Request, Requirements Document, Architecture Document), where the actual defined business rule (Discount Price is Price - Discount) could be misdefined. If that's the situation, your unit test means nothing to you.
If you've been coding for years without writing tests it may not be immediately obvious to you that there is any value. But if you are of the mindset that the best way to work is "release early, release often" or "agile" in that you want the ability to deploy rapidly/continuously, then your test definitely means something. The only way to do this is by legitimizing every change you make to the code with a test. No matter how small the test, once you have a green test suite you're theoretically OK to deploy. See also "continuous production" and "perpetual beta."
You don't have to be "test first" to be of this mindset, either, but that generally is the most efficient way to get there. When you do TDD, you lock yourself into small two to three minute Red Green Refactor cycle. At no point are you not able to stop and leave and have a complete mess on your hands that will take an hour to debug and put back together.
Additionally, your unit test is another point of failure...
A successful test is one that demonstrates a failure in the system. A failing test will alert you to an error in the logic of the test or in the logic of your system. The goal of your tests is to break your code or prove one scenario works.
If you're writing tests after the code, you run the risk of writing a test that is "bad" because in order to see that your test truly works, you need to see it both broken and working. When you're writing tests after the code, this means you have to "spring the trap" and introduce a bug into the code to see the test fail. Most developers are not only uneasy about this, but would argue it is a waste of time.
What do we gain here?
There is definitely a benefit to doing things this way. Michael Feathers defines "legacy code" as "untested code." When you take this approach, you legitimize every change you make to your codebase. It's more rigorous than not using tests, but when it comes to maintaining a large codebase, it pays for itself.
Speaking of Feathers, there are two great resources you should check out in regard to this:
Both of these explain how to work these types of practices and disciplines into projects that aren't "Greenfield." They provide techniques for writing tests around tightly coupled components, hard wired dependencies, and things that you don't necessarily have control over. It's all about finding "seams" and testing around those.
[I]f the discount price is wrong, the test team will still find the issue, how did unit testing save any work?
Habits like these are like an investment. Returns aren't immediate; they build up over time. The alternative to not testing is essentially taking on debt of not being able to catch regressions, introduce code without fear of integration errors, or drive design decisions. The beauty is you legitimize every change introduced into your codebase.
What am I missing here? Please teach me to love TDD, as I'm having a hard time accepting it as useful so far. I want too, because I want to stay progressive, but it just doesn't make sense to me.
I look at it as a professional responsibility. It's an ideal to strive toward. But it is very hard to follow and tedious. If you care about it, and feel you shouldn't produce code that is not tested, you'll be able to find the will power to learn good testing habits. One thing that I do a lot now (as do others) is timebox myself an hour to write code without any tests at all, then have the discipline to throw it away. This may seem wasteful, but it's not really. It's not like that exercise cost a company physical materials. It helped me to understand the problem and how to write code in such a way that it is both of higher quality and testable.
My advice would ultimately be that if you really don't have a desire to be good at it, then don't do it at all. Poor tests that aren't maintained, don't perform well, etc. can be worse than not having any tests. It's hard to learn on your own, and you probably won't love it, but it is going to be next to impossible to learn if you don't have a desire to do it, or can't see enough value in it to warrant the time investment.
A couple people keep mentioned that testing helps enforce the spec. It has been my experience that the spec has been wrong as well, more often than not...
A developer's keyboard is where the rubber meets the road. If the spec is wrong and you don't raise the flag on it, then it's highly probable you'll get blamed for it. Or at least your code will. The discipline and rigor involved in testing is difficult to adhere to. It's not at all easy. It takes practice, a lot of learning and a lot of mistakes. But eventually it does pay off. On a fast-paced, quickly changing project, it's the only way you can sleep at night, no matter if it slows you down.
Another thing to think about here is that techniques that are fundamentally the same as testing have been proven to work in the past: "clean room" and "design by contract" both tend to produce the same types of "meta"-code constructs that tests do, and enforce those at different points. None of these techniques are silver bullets, and rigor is going to cost you ultimately in the scope of features you can deliver in terms of time to market. But that's not what it's about. It's about being able to maintain what you do deliver. And that's very important for most projects.
Unit testing works very similar to double entry book keeping. You state the same thing (business rule) in two quite different ways (as programmed rules in your production code, and as simple, representative examples in your tests). It's very unlikely that you make the same mistake in both, so if they both agree with each other, it's rather unlikely that you got it wrong.
How is testing going to be worth the effort? In my experience in at least four ways, at least when doing test driven development:
it helps you come up with a well decoupled design. You can only unit test code that is well decoupled;
it helps you determine when you are done. Having to specify the needed behavior in tests helps to not build functionality that you don't actually need, and determine when the functionality is complete;
it gives you a safety net for refactorings, which makes the code much more amenable to changes; and
it saves you a lot of debugging time, which is horribly costly (I've heard estimates that traditionally, developers spend up to 80% of their time debugging).
Most unit tests, test assumptions. In this case, the discount price should be the price minus the discount. If your assumptions are wrong I bet your code is also wrong. And if you make a silly mistake, the test will fail and you will correct it.
If the rules change, the test will fail and that is a good thing. So you have to change the test too in this case.
As a general rule, if a test fails right away (and you don't use test first design), either the test or the code is wrong (or both if you are having a bad day). You use common sense (and possilby the specs) to correct the offending code and rerun the test.
Like Jason said, testing is security. And yes, sometimes they introduce extra work because of faulty tests. But most of the time they are huge time savers. (And you have the perfect opportunity to punish the guy who breaks the test (we are talking rubber chicken)).
Test everything you can. Even trivial mistakes, like forgetting to convert meters to feet can have very expensive side effects. Write a test, write the code for it to check, get it to pass, move on. Who knows at some point in the future, someone may change the discount code. A test can detect the problem.
Remember that the cost of fixing defects increases (exponentially) as the defects live through the development cycle. Yes, the testing team might catch the defect, but it will (usually) take more work to isolate and fix the defect from that point than if a unit test had failed, and it will be easier to introduce other defects while fixing it if you don't have unit tests to run.
That's usually easier to see with something more than a trivial example ... and with trivial examples, well, if you somehow mess up the unit test, the person reviewing it will catch the error in the test or the error in the code, or both. (They are being reviewed, right?) As tvanfosson points out, unit testing is just one part of an SQA plan.
In a sense, unit tests are insurance. They're no guarantee that you'll catch every defect, and it may seem at times like you're spending a lot of resources on them, but when they do catch defects that you can fix, you'll be spending a lot less than if you'd had no tests at all and had to fix all defects downstream.
Even more automation can help here ! Yes, writing unit tests can be a lot of work, so use some tools to help you out. Have a look at something like Pex, from Microsoft, if you're using .Net It will automatically create suites of unit tests for you by examining your code. It will come up with tests which give good coverage, trying to cover all paths through your code.
Of course, just by looking at your code it can't know what you were actually trying to do, so it doesn't know if it's correct or not. But, it will generate interesting tests cases for you, and you can then examine them and see if it is behaving as you expect.
If you then go further and write parameterized unit tests (you can think of these as contracts, really) it will generate specific tests cases from these, and this time it can know if something's wrong, because your assertions in your tests will fail.
I've thought a bit about a good way to respond to this question, and would like to draw a parallel to the scientific method. IMO, you could rephrase this question, "How do you experiment an experiment?"
Experiments verify empirical assumptions (hypotheses) about the physical universe. Unit tests will test assumptions about the state or behavior of the code they call. We can talk about the validity of an experiment, but that's because we know, through numerous other experiments, that something doesn't fit. It doesn't have both convergent validity and empirical evidence. We don't design a new experiment to test or verify the validity of an experiment, but we may design a completely new experiment.
So like experiments, we don't describe the validity of a unit test based on whether or not it passes a unit test itself. Along with other unit tests, it describes the assumptions we make about the system it is testing. Also, like experiments, we try to remove as much complexity as we can from what we are testing. "As simple as possible, but no simpler."
Unlike experiments, we have a trick up our sleeve to verify our tests are valid other than just convergent validity. We can cleverly introduce a bug we know should be caught by the test, and see if the test does indeed fail. (If only we could do that in the real world, we'd depend much less on this convergent validity thing!) A more efficient way to do this is watch your test fail before implementing it (the red step in Red, Green, Refactor).
Even if you do not test your code, it will surely be tested in production by your users. Users are very creative in trying to crash your soft and finding even non-critical errors.
Fixing bugs in production is much more costly than resolving issues in development phase. As a side-effect, you will lose income because of an exodus of customers. You can count on 11 lost or not gained customers for 1 angry customer.
发布评论
评论（17）
首先，测试就像安全性一样——你永远无法 100% 确定你已经得到了它，但每一层都增加了更多的信心和一个框架，可以更轻松地解决仍然存在的问题。
其次，您可以将测试分解为子例程，然后可以对其本身进行测试。 当您有 20 个类似的测试时，制作一个（已测试的）子例程意味着您的主要测试是子例程的 20 次简单调用，这更有可能是正确的。
第三，有些人认为 TDD 解决了这个问题。 也就是说，如果您只编写 20 个测试并且它们通过了，那么您并不完全确信它们实际上正在测试任何内容。 但是，如果您最初编写的每个测试都失败，然后您修复了它，那么您就会更有信心它确实在测试您的代码。 恕我直言，这种来回花费的时间比其价值要多，但这是一个试图解决您的担忧的过程。
First, testing is like security -- you can never be 100% sure you've got it, but each layer adds more confidence and a framework for more easily fixing the problems that remain.
Second, you can break tests into subroutines which themselves can then be tested. When you have 20 similar tests, making a (tested) subroutine means your main test is 20 simple invocations of the subroutine which is much more likely to be correct.
Third, some would argue that TDD addresses this concern. That is, if you just write 20 tests and they pass, you're not completely confident that they are actually testing anything. But if each test you wrote initially failed, and then you fixed it, then you're much more confident that it's really testing your code. IMHO this back-and-forth takes more time than it's worth, but it is a process that tries to address your concern.
错误的测试不太可能破坏您的生产代码。 至少，不比根本没有测试更糟糕。 因此，这不是一个“故障点”：测试不一定要正确才能让产品真正工作。 它们可能必须正确才能被批准工作，但是修复任何损坏的测试的过程不会危及您的实现代码。
您可以将测试（甚至是像这样的琐碎测试）视为代码应该执行的第二个意见。 一种意见是测试，另一种意见是实施。 如果他们不同意，那么你就知道自己有问题，并且你会仔细观察。
如果将来有人想要从头开始实现相同的接口，它也很有用。 他们不必阅读第一个实现来了解折扣的含义，并且测试可以作为您可能拥有的接口的任何书面描述的明确支持。
也就是说，你正在权衡时间。 如果您可以使用跳过这些琐碎测试节省的时间来编写其他测试，也许它们会更有价值。 这实际上取决于您的测试设置和应用程序的性质。 如果折扣对应用程序很重要，那么您无论如何都会在功能测试中发现此方法中的任何错误。 所有单元测试所做的就是让您在测试该单元时捕获它们，此时错误的位置将立即显而易见，而不是等到应用程序集成在一起并且错误的位置可能< /em> 不太明显。
顺便说一句，我个人不会使用 100 作为测试用例中的价格（或者更确切地说，如果我这样做，我会添加另一个价格的测试）。 原因是将来有人可能会认为折扣应该是百分比。 像这样的琐碎测试的目的之一是确保阅读规范时的错误得到纠正。
[关于编辑：我认为错误的规范不可避免地会成为失败点。 如果您不知道该应用程序应该做什么，那么它很可能不会这样做。 但是编写测试来反映规范并不会放大这个问题，它只是无法解决它。 因此，您并没有添加新的故障点，您只是在代码中表示现有的错误，而不是在
waffle文档中。]A test being wrong is unlikely to break your production code. At least, not any worse than having no test at all. So it's not a "point of failure": the tests don't have to be correct in order for the product to actually work. They might have to be correct before it's signed off as working, but the process of fixing any broken tests does not endanger your implementation code.
You can think of tests, even trivial tests like these, as being a second opinion what the code is supposed to do. One opinion is the test, the other is the implementation. If they don't agree, then you know you have a problem and you look closer.
It's also useful if someone in future wants to implement the same interface from scratch. They shouldn't have to read the first implementation in order to know what Discount means, and the tests act as an unambiguous back-up to any written description of the interface you may have.
That said, you're trading off time. If there are other tests you could be writing using the time you save skipping these trivial tests, maybe they would be more valuable. It depends on your test setup and the nature of the application, really. If the Discount is important to the app, then you're going to catch any bugs in this method in functional testing anyway. All unit testing does is let you catch them at the point you're testing this unit, when the location of the error will be immediately obvious, instead of waiting until the app is integrated together and the location of the error might be less obvious.
By the way, personally I wouldn't use 100 as the price in the test case (or rather, if I did then I'd add another test with another price). The reason is that someone in future might think that Discount is supposed to be a percentage. One purpose of trivial tests like this is to ensure that mistakes in reading the specification are corrected.
[Concerning the edit: I think it's inevitable that an incorrect specification is a point of failure. If you don't know what the app is supposed to do, then chances are it won't do it. But writing tests to reflect the spec doesn't magnify this problem, it merely fails to solve it. So you aren't adding new points of failure, you're just representing the existing faults in code instead of
waffledocumentation.]单元测试并不是真的应该节省工作，它应该帮助您发现并防止错误。 这是更多的工作，但这是正确的工作。 它以最低的粒度级别思考您的代码，并编写测试用例来证明它对于给定的输入集可以在预期条件下工作。 它隔离变量，因此当错误确实出现时，您可以通过查找正确的位置来节省时间。 它保存这套测试，以便您在以后必须进行更改时可以一次又一次地使用它们。
我个人认为大多数方法论都没有从 cargo cult 软件工程中删除很多步骤，TDD包括在内，但您不必遵守严格的 TDD 即可获得单元测试的好处。 保留好的部分，扔掉效益不大的部分。
最后，您的标题问题“如何对单元测试进行单元测试？”的答案是您不必这样做。 每个单元测试都应该非常简单。 使用特定输入调用方法并将其与预期输出进行比较。 如果方法的规范发生变化，那么您可以预期该方法的一些单元测试也需要更改。 这就是您以如此低的粒度进行单元测试的原因之一，因此只有一些单元测试需要更改。 如果您发现许多不同方法的测试因需求中的一项更改而发生变化，那么您可能没有以足够精细的粒度级别进行测试。
Unit testing isn't really supposed to save work, it's supposed to help you find and prevent bugs. It's more work, but it's the right kind of work. It's thinking about your code at the lowest levels of granularity and writing test cases that prove that it works under expected conditions, for a given set of inputs. It's isolating variables so you can save time by looking in the right place when a bug does present itself. It's saving that suite of tests so that you can use them again and again when you have to make a change down the road.
I personally think that most methodologies are not many steps removed from cargo cult software engineering, TDD included, but you don't have to adhere to strict TDD to reap the benefits of unit testing. Keep the good parts and throw out the parts that yield little benefit.
Finally, the answer to your titular question "How do you unit test a unit test?" is that you shouldn't have to. Each unit test should be brain-dead simple. Call a method with a specific input and compare it to its expected output. If the specification for a method changes then you can expect that some of the unit tests for that method will need to change as well. That's one of the reasons that you do unit testing at such a low level of granularity, so only some of the unit tests have to change. If you find that tests for many different methods are changing for one change in a requirement, then you may not be testing at a fine enough level of granularity.
单元测试的存在是为了让您的单元（方法）按照您的预期进行。 首先编写测试迫使您在编写代码之前考虑您的期望。 行动前思考总是一个好主意。
单元测试应该反映业务规则。 当然，代码中可能存在错误，但首先编写测试可以让您在编写任何代码之前从业务规则的角度编写测试。 我认为，事后编写测试更有可能导致您所描述的错误，因为您知道代码如何实现它，并且只是想确保实现是正确的 - 而不是意图是正确的。
此外，单元测试只是您应该编写的测试的一种形式（也是最低级的形式）。 如果可能的话，还应该编写集成测试和验收测试，后者由客户编写，以确保系统按预期方式运行。 如果您在此测试期间发现错误，请返回并编写单元测试（失败）来测试功能更改以使其正常工作，然后更改代码以使测试通过。 现在您已经有了捕获错误修复的回归测试。
[编辑]
我在 TDD 中发现的另一件事。 它几乎默认强制要求良好的设计。 这是因为高度耦合的设计几乎不可能单独进行单元测试。 使用 TDD 不需要很长时间就能发现使用接口、控制反转和依赖项注入（所有将改进设计并减少耦合的模式）对于可测试代码确实非常重要。
Unit tests are there so that your units (methods) do what you expect. Writing the test first forces you to think about what you expect before you write the code. Thinking before doing is always a good idea.
Unit tests should reflect the business rules. Granted, there can be errors in the code, but writing the test first allows you to write it from the perspective of the business rule before any code has been written. Writing the test afterwards, I think, is more likely to lead to the error you describe because you know how the code implements it and are tempted just to make sure that the implementation is correct -- not that the intent is correct.
Also, unit tests are only one form -- and the lowest, at that -- of tests that you should be writing. Integration tests and acceptance tests should also be written, the latter by the customer, if possible, to make sure that the system operates the way it is expected. If you find errors during this testing, go back and write unit tests (that fail) to test the change in functionality to make it work correctly, then change your code to make the test pass. Now you have regression tests that capture your bug fixes.
[EDIT]
Another thing that I have found with doing TDD. It almost forces good design by default. This is because highly coupled designs are nearly impossible to unit test in isolation. It doesn't take very long using TDD to figure out that using interfaces, inversion of control, and dependency injection -- all patterns that will improve your design and reduce coupling -- are really important for testable code.
如何测试一项测试？ 突变测试是一项很有价值的技术，我个人使用过它，效果出奇的好。 阅读链接的文章了解更多详细信息，并链接到更多学术参考文献，但一般来说，它通过修改源代码（例如将“x += 1”更改为“x -= 1”）来“测试您的测试”，然后重新运行测试，确保至少有一项测试失败。 任何不会导致测试失败的突变都会被标记以供以后调查。
您会惊讶地发现，通过一组看起来很全面的测试，您可以实现 100% 的行和分支覆盖，但您可以从根本上更改甚至注释掉源代码中的一行，而不会出现任何测试抱怨。 通常这可以归结为没有使用正确的输入进行测试来覆盖所有边界情况，有时它更微妙，但在所有情况下，我对它的结果印象深刻。
How does one test a test? Mutation testing is a valuable technique that I have personally used to surprisingly good effect. Read the linked article for more details, and links to even more academic references, but in general it "tests your tests" by modifying your source code (changing "x += 1" to "x -= 1" for example) and then rerunning your tests, ensuring that at least one test fails. Any mutations that don't cause test failures are flagged for later investigation.
You'd be surprised at how you can have 100% line and branch coverage with a set of tests that look comprehensive, and yet you can fundamentally change or even comment out a line in your source without any of the tests complaining. Often this comes down to not testing with the right inputs to cover all boundary cases, sometimes it's more subtle, but in all cases I was impressed with how much came out of it.
应用测试驱动开发 (TDD) 时，首先会进行失败测试。 这一步看似不必要，实际上是为了验证单元测试正在测试某些内容。 事实上，如果测试永远不会失败，它就不会带来任何价值，更糟糕的是，会导致错误的信心，因为你会依赖一个不能证明任何事情的积极结果。
当严格遵循这个过程时，所有“单元”都受到单元测试所建立的安全网的保护，即使是最普通的单元。
测试没有理由朝那个方向发展——或者我在你的推理中遗漏了一些东西。 当价格为100，折扣20时，折扣价为80。这就像一个不变量。
现在假设您的软件需要支持另一种基于百分比的折扣，也许取决于购买的数量，您的 Product::DiscountPrice() 方法可能会变得更加复杂。 引入这些变化可能会打破我们最初的简单折扣规则。 然后您将看到此测试的值，它将立即检测回归。
红-绿-重构——这是要记住TDD过程的本质。
红色是指测试失败时JUnit的红色条。
绿色是所有测试通过时JUnit进度条的颜色。
绿色条件下重构：去除重复，提高可读性。
现在，解决您关于“代码上方 3-4 层”的观点，这在传统（瀑布式）流程中是正确的，而不是在敏捷开发流程中。 敏捷是 TDD 的发源地； TDD 是极限编程的基石。
敏捷是关于直接沟通，而不是扔掉墙上的需求文档。
When applying Test-Driven Development (TDD), one begins with a failing test. This step, that might seem unecessary, actually is here to verify the unit test is testing something. Indeed, if the test never fails, it brings no value and worse, leads to wrong confidence as you'll rely on a positive result that is not proving anything.
When following this process strictly, all ''units'' are protected by the safety net the unit tests are making, even the most mundane.
There is no reason the test evolves in that direction - or I'm missing something in your reasoning. When the price is 100 and the discount 20, the discount price is 80. This is like an invariant.
Now imagine your software needs to support another kind of discount based on percentage, perhaps depending on the volume bought, your Product::DiscountPrice() method may become more complicated. And it is possible that introducing those changes breaks the simple discount rule we had initially. Then you'll see the value of this test which will detect the regression immediately.
Red - Green - Refactor - this is to remember the essence of the TDD process.
Red refers to JUnit red bar when a tests fails.
Green is the color of JUnit progress bar when all tests pass.
Refactor under green condition: remove any dupliation, improve readability.
Now to address your point about the "3-4 layers above the code", this is true in a traditional (waterfall-like) process, not when the development process is agile. And agile is the world where TDD is coming from ; TDD is the cornerstone of eXtreme Programming.
Agile is about direct communication rather than thrown-over-the-wall requirement documents.
像这样的小而琐碎的测试可以成为您的代码库的“煤矿里的金丝雀”，在为时已晚之前发出危险警报。 保留这些琐碎的测试很有用，因为它们可以帮助您获得正确的交互。
例如，考虑一个简单的测试来探究如何使用您不熟悉的 API。 如果该测试与您在“真正”使用 API 的代码中所做的事情有任何相关性，那么保留该测试会很有用。 当API发布新版本，需要升级时。 现在，您已经对 API 的行为方式做出了假设，并以可执行格式记录下来，您可以使用该格式来捕获回归。
如果您已经编码多年但没有编写测试，那么您可能不会立即意识到它有任何价值。 但是，如果您认为最好的工作方式是“尽早发布、经常发布”或“敏捷”，因为您希望能够快速/持续部署，那么您的测试绝对有意义。 做到这一点的唯一方法是通过测试使您对代码所做的每项更改合法化。 无论测试有多小，一旦拥有绿色测试套件，理论上就可以部署。 另请参见“持续生产”和“永久测试版”。
您也不必“先测试”才能拥有这种心态，但这通常是实现这一目标的最有效方法。 当你进行 TDD 时，你会将自己锁定在两到三分钟的红绿重构周期中。 在任何时候，您都不能停下来离开，手上的东西一团糟，需要一个小时的时间来调试和重新组装。
成功的测试表明系统存在故障。 失败的测试将提醒您测试逻辑或系统逻辑中存在错误。 测试的目标是破坏代码或证明一种方案有效。
如果您在代码之后编写测试，那么您将面临编写“糟糕”测试的风险，因为为了看到您的测试真正有效，您需要看到它既损坏又有效。 当您在代码之后编写测试时，这意味着您必须“抛出陷阱”并在代码中引入错误才能看到测试失败。 大多数开发人员不仅对此感到不安，而且认为这是浪费时间。
以这种方式做事肯定有好处。 Michael Feathers 将“遗留代码”定义为“未经测试的代码”。 当您采用这种方法时，您对代码库所做的每项更改都是合法的。 这比不使用测试更严格，但是当涉及到维护大型代码库时，它是值得的。
说到 Feathers，您应该查看以下两个很棒的资源：
两者都解释了如何使用这些类型的将实践和纪律融入到非“绿地”项目中。 它们提供了围绕紧密耦合的组件、硬连线依赖项以及您不一定能够控制的事物编写测试的技术。 这一切都是为了寻找“接缝”并围绕这些接缝进行测试。
像这样的习惯就像一项投资。 回报不是立即的； 它们随着时间的推移而积累。 不进行测试的替代方案本质上是承担无法捕捉回归、无法引入代码而不担心集成错误或推动设计决策的债务。 美妙之处在于您可以使代码库中引入的每项更改都合法化。
我将其视为一种职业责任。 这是一个值得努力的理想。 但它很难遵循并且乏味。 如果您关心它，并且认为您不应该编写未经测试的代码，那么您将能够找到学习良好测试习惯的意志力。 我现在经常做的一件事（和其他人一样）是给自己一个小时的时间来编写代码，根本不进行任何测试，然后有纪律地扔掉它。 这可能看起来很浪费，但事实并非如此。 这并不是说该活动会消耗公司的物质材料。 它帮助我理解问题以及如何以更高质量和可测试的方式编写代码。
我的建议最终是，如果你真的不想擅长这件事，那就根本不要这样做。 未维护、性能不佳等不良测试可能比没有任何测试更糟糕。 自己学习很难，而且你可能不会喜欢它，但如果你没有意愿去做，或者看不到它足够的价值，那么学习几乎是不可能的。保证时间投资。
开发人员的键盘是橡胶与道路的交汇处。 如果规范是错误的，而你又没有提出警告，那么你很可能会因此受到指责。 或者至少你的代码会。 测试中涉及的纪律和严格性很难遵守。 这一点都不容易。 这需要练习、大量的学习和大量的错误。 但最终它确实得到了回报。 在快节奏、快速变化的项目中，这是您晚上睡觉的唯一方法，无论它是否会减慢您的速度。
这里要考虑的另一件事是，与测试基本相同的技术在过去已被证明是有效的：“洁净室”和“按合同设计”都倾向于产生相同类型的“元”代码构造，测试确实如此，并在不同的点执行这些测试。 这些技术都不是灵丹妙药，严格的要求最终会让您在产品上市时间方面付出代价。 但这不是重点。 这是关于能够维持您所做的交付。 这对于大多数项目来说非常重要。
Small, trivial tests like this can be the "canary in the coalmine" for your codebase, alerting of danger before it's too late. The trivial tests are useful to keep around because they help you get the interactions right.
For example, think about a trivial test put in place to probe how to use an API you're unfamiliar with. If that test has any relevance to what you're doing in the code that uses the API "for real" it's useful to keep that test around. When the API releases a new version and you need to upgrade. You now have your assumptions about how you expect the API to behave recorded in an executable format that you can use to catch regressions.
If you've been coding for years without writing tests it may not be immediately obvious to you that there is any value. But if you are of the mindset that the best way to work is "release early, release often" or "agile" in that you want the ability to deploy rapidly/continuously, then your test definitely means something. The only way to do this is by legitimizing every change you make to the code with a test. No matter how small the test, once you have a green test suite you're theoretically OK to deploy. See also "continuous production" and "perpetual beta."
You don't have to be "test first" to be of this mindset, either, but that generally is the most efficient way to get there. When you do TDD, you lock yourself into small two to three minute Red Green Refactor cycle. At no point are you not able to stop and leave and have a complete mess on your hands that will take an hour to debug and put back together.
A successful test is one that demonstrates a failure in the system. A failing test will alert you to an error in the logic of the test or in the logic of your system. The goal of your tests is to break your code or prove one scenario works.
If you're writing tests after the code, you run the risk of writing a test that is "bad" because in order to see that your test truly works, you need to see it both broken and working. When you're writing tests after the code, this means you have to "spring the trap" and introduce a bug into the code to see the test fail. Most developers are not only uneasy about this, but would argue it is a waste of time.
There is definitely a benefit to doing things this way. Michael Feathers defines "legacy code" as "untested code." When you take this approach, you legitimize every change you make to your codebase. It's more rigorous than not using tests, but when it comes to maintaining a large codebase, it pays for itself.
Speaking of Feathers, there are two great resources you should check out in regard to this:
Both of these explain how to work these types of practices and disciplines into projects that aren't "Greenfield." They provide techniques for writing tests around tightly coupled components, hard wired dependencies, and things that you don't necessarily have control over. It's all about finding "seams" and testing around those.
Habits like these are like an investment. Returns aren't immediate; they build up over time. The alternative to not testing is essentially taking on debt of not being able to catch regressions, introduce code without fear of integration errors, or drive design decisions. The beauty is you legitimize every change introduced into your codebase.
I look at it as a professional responsibility. It's an ideal to strive toward. But it is very hard to follow and tedious. If you care about it, and feel you shouldn't produce code that is not tested, you'll be able to find the will power to learn good testing habits. One thing that I do a lot now (as do others) is timebox myself an hour to write code without any tests at all, then have the discipline to throw it away. This may seem wasteful, but it's not really. It's not like that exercise cost a company physical materials. It helped me to understand the problem and how to write code in such a way that it is both of higher quality and testable.
My advice would ultimately be that if you really don't have a desire to be good at it, then don't do it at all. Poor tests that aren't maintained, don't perform well, etc. can be worse than not having any tests. It's hard to learn on your own, and you probably won't love it, but it is going to be next to impossible to learn if you don't have a desire to do it, or can't see enough value in it to warrant the time investment.
A developer's keyboard is where the rubber meets the road. If the spec is wrong and you don't raise the flag on it, then it's highly probable you'll get blamed for it. Or at least your code will. The discipline and rigor involved in testing is difficult to adhere to. It's not at all easy. It takes practice, a lot of learning and a lot of mistakes. But eventually it does pay off. On a fast-paced, quickly changing project, it's the only way you can sleep at night, no matter if it slows you down.
Another thing to think about here is that techniques that are fundamentally the same as testing have been proven to work in the past: "clean room" and "design by contract" both tend to produce the same types of "meta"-code constructs that tests do, and enforce those at different points. None of these techniques are silver bullets, and rigor is going to cost you ultimately in the scope of features you can deliver in terms of time to market. But that's not what it's about. It's about being able to maintain what you do deliver. And that's very important for most projects.
单元测试的工作原理与复式记账非常相似。 您以两种完全不同的方式陈述同一件事（业务规则）（作为生产代码中的编程规则，以及作为测试中的简单、代表性示例）。 您不太可能在两者中都犯相同的错误，因此如果它们都彼此同意，那么您就不太可能犯错。
测试如何值得付出努力？ 根据我的经验，至少有四种方式，至少在进行测试驱动开发时：
Unit testing works very similar to double entry book keeping. You state the same thing (business rule) in two quite different ways (as programmed rules in your production code, and as simple, representative examples in your tests). It's very unlikely that you make the same mistake in both, so if they both agree with each other, it's rather unlikely that you got it wrong.
How is testing going to be worth the effort? In my experience in at least four ways, at least when doing test driven development:
大多数单元测试，测试假设。 在这种情况下，折扣价格应该是价格减去折扣。 如果你的假设是错误的，我敢打赌你的代码也是错误的。 如果你犯了一个愚蠢的错误，测试就会失败，你会纠正它。
如果规则改变，测试就会失败，这是一件好事。 所以在这种情况下你也必须改变测试。
作为一般规则，如果测试立即失败（并且您不使用测试优先设计），则测试或代码都是错误的（或者如果您度过了糟糕的一天，则两者都是错误的）。 您可以使用常识（也可能根据规范）来纠正有问题的代码并重新运行测试。
正如杰森所说，测试就是安全。 是的，有时他们会因为错误的测试而引入额外的工作。 但大多数时候，它们可以节省大量时间。 （而且你有绝佳的机会惩罚那些破坏测试的人（我们说的是橡皮鸡））。
Most unit tests, test assumptions. In this case, the discount price should be the price minus the discount. If your assumptions are wrong I bet your code is also wrong. And if you make a silly mistake, the test will fail and you will correct it.
If the rules change, the test will fail and that is a good thing. So you have to change the test too in this case.
As a general rule, if a test fails right away (and you don't use test first design), either the test or the code is wrong (or both if you are having a bad day). You use common sense (and possilby the specs) to correct the offending code and rerun the test.
Like Jason said, testing is security. And yes, sometimes they introduce extra work because of faulty tests. But most of the time they are huge time savers. (And you have the perfect opportunity to punish the guy who breaks the test (we are talking rubber chicken)).
尽你所能测试一切。 即使是很小的错误，例如忘记将米转换为英尺，也会产生非常昂贵的副作用。 编写测试，编写代码进行检查，使其通过，然后继续。 谁知道将来某个时候，有人可能会更改折扣代码。 测试可以发现问题。
Test everything you can. Even trivial mistakes, like forgetting to convert meters to feet can have very expensive side effects. Write a test, write the code for it to check, get it to pass, move on. Who knows at some point in the future, someone may change the discount code. A test can detect the problem.
我认为单元测试和生产代码具有共生关系。 简而言之：一个测试另一个。 两者都考验开发人员。
I see unit tests and production code as having a symbiotic relationship. Simply put: one tests the other. And both test the developer.
请记住，随着缺陷在整个开发周期中存在，修复缺陷的成本会增加（呈指数级）。 是的，测试团队可能会发现缺陷，但与单元测试失败相比，从该点隔离和修复缺陷（通常）需要更多的工作，并且如果您在修复缺陷时更容易引入其他缺陷没有要运行的单元测试。
这通常比一个简单的例子更容易看到......并且通过简单的例子，好吧，如果你以某种方式弄乱了单元测试，审查它的人会发现测试中的错误或代码中的错误，或者两个都。 （它们正在接受审查，对吧？）正如 tvanfosson 指出的< /a>，单元测试只是 SQA 计划的一部分。
从某种意义上说，单元测试是保险。 它们不能保证您会发现所有缺陷，有时您似乎在它们上花费了大量资源，但是当它们确实发现了您可以修复的缺陷时，您花费的资源就会少得多与根本没有进行任何测试并且必须修复下游的所有缺陷相比。
Remember that the cost of fixing defects increases (exponentially) as the defects live through the development cycle. Yes, the testing team might catch the defect, but it will (usually) take more work to isolate and fix the defect from that point than if a unit test had failed, and it will be easier to introduce other defects while fixing it if you don't have unit tests to run.
That's usually easier to see with something more than a trivial example ... and with trivial examples, well, if you somehow mess up the unit test, the person reviewing it will catch the error in the test or the error in the code, or both. (They are being reviewed, right?) As tvanfosson points out, unit testing is just one part of an SQA plan.
In a sense, unit tests are insurance. They're no guarantee that you'll catch every defect, and it may seem at times like you're spending a lot of resources on them, but when they do catch defects that you can fix, you'll be spending a lot less than if you'd had no tests at all and had to fix all defects downstream.
我明白你的观点，但显然有些言过其实了。
你的论点基本上是：测试会导致失败。 因此测试很糟糕/浪费时间。
虽然在某些情况下可能是这样，但这并不是大多数。
TDD 假设：更多测试 = 更少失败。
测试更有可能发现故障点而不是引入故障点。
I see your point, but it's clearly overstated.
Your argument is basically: Tests introduce failure. Therefore tests are bad/waste of time.
While that may be true in some cases, it's hardly the majority.
TDD assumes: More Tests = Less Failure.
Tests are more likely to catch points of failure than introduce them.
更多的自动化可以在这方面提供帮助！
是的，编写单元测试可能需要大量工作，因此请使用一些工具来帮助您。
如果您使用 .Net，请看一下 Microsoft 的 Pex 之类的东西
它会通过检查您的代码自动为您创建单元测试套件。 它将提出提供良好覆盖范围的测试，尝试覆盖代码中的所有路径。
当然，仅通过查看您的代码，它无法知道您实际上想要做什么，因此它不知道它是否正确。 但是，它会为您生成有趣的测试用例，然后您可以检查它们并查看它的行为是否符合您的预期。
如果您进一步编写参数化单元测试（实际上，您可以将它们视为合同），它将从中生成特定的测试用例，这一次它可以知道是否有问题，因为您在测试中的断言将失败。
Even more automation can help here !
Yes, writing unit tests can be a lot of work, so use some tools to help you out.
Have a look at something like Pex, from Microsoft, if you're using .Net
It will automatically create suites of unit tests for you by examining your code. It will come up with tests which give good coverage, trying to cover all paths through your code.
Of course, just by looking at your code it can't know what you were actually trying to do, so it doesn't know if it's correct or not. But, it will generate interesting tests cases for you, and you can then examine them and see if it is behaving as you expect.
If you then go further and write parameterized unit tests (you can think of these as contracts, really) it will generate specific tests cases from these, and this time it can know if something's wrong, because your assertions in your tests will fail.
我想了一些回答这个问题的好方法，并想与科学方法进行比较。 IMO，你可以重新表述这个问题，“你如何进行实验？”
实验验证有关物理宇宙的经验假设（假说）。 单元测试将测试有关它们调用的代码的状态或行为的假设。 我们可以谈论实验的有效性，但那是因为我们通过大量其他实验知道有些东西不合适。 它不具有收敛有效性和经验证据。 我们不会设计一个新的实验来测试或验证一个实验的有效性，但我们可能会设计一个全新的实验。
因此，像实验一样，我们不会根据单元测试本身是否通过单元测试来描述单元测试的有效性。 与其他单元测试一起，它描述了我们对其正在测试的系统所做的假设。 此外，就像实验一样，我们尝试尽可能地消除正在测试的复杂性。 “尽可能简单，但不能更简单。”
与实验不同，我们有一个技巧来验证我们的测试是否有效，而不仅仅是收敛有效性。 我们可以巧妙地引入一个我们知道测试应该捕获的错误，并查看测试是否确实失败。 （如果我们能在现实世界中做到这一点，我们就不会那么依赖这种收敛有效性了！）更有效的方法是在实现测试之前观察测试失败（Red 中的红色步骤，绿色，重构）。
I've thought a bit about a good way to respond to this question, and would like to draw a parallel to the scientific method. IMO, you could rephrase this question, "How do you experiment an experiment?"
Experiments verify empirical assumptions (hypotheses) about the physical universe. Unit tests will test assumptions about the state or behavior of the code they call. We can talk about the validity of an experiment, but that's because we know, through numerous other experiments, that something doesn't fit. It doesn't have both convergent validity and empirical evidence. We don't design a new experiment to test or verify the validity of an experiment, but we may design a completely new experiment.
So like experiments, we don't describe the validity of a unit test based on whether or not it passes a unit test itself. Along with other unit tests, it describes the assumptions we make about the system it is testing. Also, like experiments, we try to remove as much complexity as we can from what we are testing. "As simple as possible, but no simpler."
Unlike experiments, we have a trick up our sleeve to verify our tests are valid other than just convergent validity. We can cleverly introduce a bug we know should be caught by the test, and see if the test does indeed fail. (If only we could do that in the real world, we'd depend much less on this convergent validity thing!) A more efficient way to do this is watch your test fail before implementing it (the red step in Red, Green, Refactor).
编写测试时需要使用正确的范例。
您不能总是确定，但它们可以改善整体测试。
You need to use the correct paradigm when writing tests.
You cant always be sure but they improve overall tests.
即使您不测试代码，用户也肯定会在生产中对其进行测试。 用户非常有创意地尝试让你的软件崩溃，甚至发现非严重的错误。
修复生产中的错误比解决开发阶段的问题要昂贵得多。
副作用是，您将因客户流失而损失收入。 您可以指望 1 个愤怒的客户会导致 11 个失去或未获得的客户。
Even if you do not test your code, it will surely be tested in production by your users. Users are very creative in trying to crash your soft and finding even non-critical errors.
Fixing bugs in production is much more costly than resolving issues in development phase.
As a side-effect, you will lose income because of an exodus of customers. You can count on 11 lost or not gained customers for 1 angry customer.