如何找到可能直接或间接调用给定方法的所有单元测试?
当我更改方法时,我希望知道要运行的最佳测试;必须有一个工具可以做到这一点!
由于我们有很多接口,所以我对所有在实现接口的类上至少有一个路径 var 植入方法时调用接口上的方法的单元测试感兴趣。
或者换句话说,当工具无法证明结果不受我更改的方法影响时,我想要一个所有单元测试的列表。
(我们在 .net 上使用 nUnit,并且有很多缓慢的情况)单元测试,我们需要很多年才能将所有单元测试重构得更快)
另请参阅:
How do I find all the unit tests that may directly or indirectly call a given method?
When I change a method, I wish to know the best tests to run; there must be a tool for this!
As we have lots of interfaces, I am interested in all unit tests that calls a method on an interface when there is at least one path var the implantation method on a class that implements the interface.
Or in other words, I want a list of all unit tests when the tool cannot prove the result is not affected by the method I have changed.
(We are using nUnit on .net and have lots of slow unit tests, it will be many year until we have refactored all our unit tests to be fast)
see also:
发布评论
评论(4)
您需要的是每个测试与其所执行的代码之间的关系。
这可以静态计算,但这很困难,而且我不知道有任何工具可以做到这一点。更糟糕的是,如果您有这样的工具,则用于确定测试影响的静态分析实际上可能比运行测试本身花费的时间更长,因此这看起来不是一个有吸引力的方向。
但是,这可以使用测试覆盖率工具来计算。对于每个单独的测试,
运行该测试(我们假设它通过)并收集测试覆盖率数据。我们现在有
很多对 (t_i,c_i) 表示“测试 i 具有覆盖范围 c”。
当代码库发生变化时,可以查阅测试数据覆盖率集。一个简单的检查:如果对于任何(t_i,c_i),如果c_i提到文件F,并且F已更改,则需要再次运行t_i。
给定几乎任何表示形式的测试覆盖率数据,这在抽象中很容易检测到。鉴于大多数测试覆盖率工具并没有具体告诉您它们如何
存储测试覆盖率数据,这比实践中看起来更难。
实际上,理想情况下,如果 c_i 提到 F 的任何编程元素,并且该编程元素已更改,则需要再次运行 t_i。
我们的SD 测试覆盖率工具在方法级别为 Java 和 C# 提供了这种功能。
您需要设置一些脚本来将实际测试(无论您如何打包)与收集的测试覆盖向量相关联。作为一个实际问题,这往往非常容易。
What you need is a relation between each test and the code it exercises.
This can be computed statically but it is hard and I don't know any tools that do it. Worse, if you had such a tool, the static analysis to decide what the test affected, might actually take longer than just running the test itself, so this doesn't look like an attractive direction.
However, this can be computed using a test coverage tool. For each individual test,
run that test (we presume it passes) and collect the test coverage data. We now have
a lot of pairs (t_i,c_i) for "test i has coverage c".
When the code base changes, the test data coverage sets can be consulted. A simple check: if for any (t_i,c_i), if c_i mentions file F, and F has changed, you need to run t_i again.
Given test coverage data in almost any representation this is sort of easy to detect in the abstract. Given that most test coverage tools don't specifically tell you how they
store the test coverage data, this is harder than it looks in practice.
Actually, what you want ideally is if c_i mentions any programmatic element of F, and that programmatic element has changed, you need to run t_i again.
Our SD Test Coverage tools provide this capability for Java and C#, at the level of methods.
You need to set up some scripting to associate the actual test, however you have packaged it, with collected test coverage vectors. As a practical matter this tends to be pretty easy.
您可以使用VS2010的“查看呼叫层次结构”或ReSharpers“ReSharper -> Inspect -> Incoming Calls”。您甚至可以从 ReSharper 导出结果。
如果您想使用自动化来实现您的目标,即因为这不是一次性的事情,而是您想要集成到构建过程中的事情,我建议您构建自己的解决方案。您可以在程序集上使用 Reflection 或 Mono.Cecil 并自行遍历调用层次结构。但是,这可能非常耗时,因为您需要在所有程序集上创建完整的调用层次结构树。另一种可能性是创建 Visual Studio 或 ReSharper 插件,这样您就可以从源代码访问它们创建的对象模型。
基本上,我想说的是:我不知道目前有任何现有方法可以通过自动化实现您的目标,编写您自己的解决方案将很有趣,但很棘手,而且可能很耗时,无论是在开发还是运行它时。
You can use VS2010's "View Call Hierarchy" or ReSharpers "ReSharper -> Inspect -> Incoming calls". You can even export the result from ReSharper.
If you want to use automation to achieve your goal, i.e. because this is not a one-time thing, but something you want to integrate into your build process, I suggest, you build your own solution. You could use Reflection or Mono.Cecil on the assemblies and walk the call hierarchy yourself. However, this can be quite time consuming, because you would need to create a complete call hierarchy tree over all your assemblies. Another possibility would be to create a Visual Studio or ReSharper plugin, so you can access the object model they create from your source code.
Basically, what I am saying is: I don't know of any currently existing method to achieve your goal via automation and writing your own solution will be fun but tricky and possibly time consuming, either when developing or running it.
Visual Studio 2010 具有以下功能: http://blogs.msdn.com/b/phuene/archive/2009/12/07/test-impact-analysis-in-visual-studio-2010.aspx
自从您正在使用 nunit,您也可以尝试此技术来运行 MSTest: http ://msdn.microsoft.com/en-gb/magazine/cc163643.aspx#S4
Visual Studio 2010 has this exact feature: http://blogs.msdn.com/b/phuene/archive/2009/12/07/test-impact-analysis-in-visual-studio-2010.aspx
Since you're using nunit, you could try this technique to run MSTest as well: http://msdn.microsoft.com/en-gb/magazine/cc163643.aspx#S4
在最一般的情况下,当您使用委托、lambda 表达式等时,我的直觉是这个问题相当于检查给定的程序是否会完成(停止问题),所以我不希望找到合理的答案。
如果您的真正目标是能够快速测试您的更改是否破坏了某些内容,我建议:
In the most general case, when you use delegates, lambda expressions, etc., I have the intuition that this problem is equivalent to checking if a given program will finish (the Halting Problem), so I wouldn't hope to find a reasonable answer.
If your real goal is to be able to quickly test if your change broke something, I would recommend: