From my current perspective (after 8 months of development with about 10 developers on average) I would advise against using MSTest for the following reasons
The framework in itself is quite slow. I don't mean the test code that you write - that's under your control. I mean the framework running those tests is slow, whether it's running a test suite, single tests etc.
The need to keep a Test-Metadata file which always leads to complications when several developers are working on it (recreating e.g. the metadata etc.). Every other test suite doesn't need a metadata file. It is kind of nice to organize your tests but you can achieve the same through namespaces, classes and method names.
Doing Continuous Integration, if you want to run unit tests on your build machine you will need to install Visual Studio on that machine.
In other words, if I would have to decide again 8 months ago, I would probably take NUnit. I may not have the integrated test results report, but developers would have a more seamless testing experience.
补充:我们现在还有一些测试,甚至无法说出有多少。由于 OutOfMemoryExceptions 和其他不稳定问题,不可能再从 Visual Studio 运行它们。我们从脚本运行测试。在 Visual Studio 中查看测试结果很容易,但是当解决方案打开时,VS 崩溃(每次)。因此我们需要使用文本搜索来搜索失败的测试。集成工具不再有任何优势。
另一个更新:我们现在使用 VS 2013。很多事情都改变了。自我们开始以来,他们第三次重写了 MS Test 测试运行程序。这导致了很多重大变化,但两个新版本都没有做得更好。我们很高兴没有使用 MS Test 的奇特功能,因为它们都不再受支持。实在是太可惜了。我们仍然使用脚本来构建和运行所有单元测试,因为它更方便。 Visual Studio 需要几分钟才能开始运行测试(编译后直到第一次测试开始的时间测量)。他们可能通过更新修复它,这可能是我们项目的特定问题。然而,在运行相同的测试时,Resharper 的速度要快得多。
结论:至少与Resharper结合使用时,MS Test是有用的。我希望他们最终能弄清楚测试运行程序应该如何编写,并且在我们下次更新 Visual Studio 时不会进行此类重大更改。
Here is my experience with MS Test
We are running MS Test with around 3800 Test.
It takes very long for the tests just to start executing, which is painful when running single tests.
It takes around 1GB Memory to execute the tests. No, it is not due to memory leaks in our tests. Frequently we run into OutOfMemoryExceptions.
Because it uses that much resource, we are starting to execute the tests from batch-files. So what's the whole integration good for?
It is buggy and unstable:
For instance, if you remove the [Ignore] Attribute from a test, it does not recognize it, because it caches information about tests somewhere. You need to refresh the testlist, which sometimes solves the problem, or restart VS.
It randomly does not copy reference assemblies to theout directory.
Deployment Items (additional files to be used) just don't work properly. They are ignored randomly.
There is hidden (not visible in the test code) information in vsmdi and testrunconfig files. If you don't care about it, it might not work.
Functionally it might be comparable to NUnit, but it is very expensive if you consider using VS tester edition.
Addition: We have some more tests now, can't even say how many. It is impossible to run them all anymore from Visual Studio, because of OutOfMemoryExceptions and other instability problems. We run the tests from scripts. It would be easy to view test results in Visual Studio, but when the solution is open, VS crashes (every time). So we need to search the failing tests using text search. There is no advantage of an integrated tool anymore.
Another Update: We are using VS 2013 now. A lot of things changed. They rewrote the MS Test test runner for the third time since we started. This caused a lot of breaking changes, but neither new version was doing anything better. We are glad that we didn't use the fancy features of MS Test, because they are all not supported anymore. It's really a shame. We are still using scripts to build and run all unit tests, because it is handier. Visual Studio required a few minutes to start running tests (time measures after compilation until first test starts). They probably fixes it with an update and this might be a specific problem of our project. However, Resharper is much quicker when running the same tests.
Conclusion: At least in combination with Resharper, MS Test is useful. And I hope that they finally find out how the test runner should be written and won't do this kind of breaking changes when we update Visual Studio next time.
I am not sure of others but NUnit provides nice GUI and console to run your Unit Tests and also you can generate report of the result of NUnit Test execution which would give the detail of whethere the test has failed or passed and also what time did it take for your unit test
NUnit is a unit testing framework, which is also supported by resharper. I think you are using Microsoft's unit testing framework, so NUnit is just an alternative to Microsoft's product ;)
在 NUnit 中,测试不是并行执行的。相反,似乎所有测试都在单个线程上执行。在 MSTest 中,每个测试都在单独的线程上实例化,这导致运行交错。因此,如果测试 A 的成功依赖于测试 B,则它可能会失败,因为测试 B 可能会在测试 A 运行时开始运行。
In NUnit, tests are not executed in parallel. Rather, it appears that all tests execute on a single thread. In MSTest, each test is instantiated on a separate thread, this results the runs being interleaved. Therefore, if test A depends on test B for its success, it likely will fail as test B will likely start running as test A is running.
发布评论
评论(9)
NUnit 与 MS-Test
Assert.AreEqual(expected,actual)
vsAssert.That(actual, Is.EqualTo(expected))
[TestCase]
中的 Try-Catch 来完成! NUnit 允许进行参数化测试。NUnit has few advantages over MS-Test
Assert.AreEqual(expected, actual)
vsAssert.That(actual, Is.EqualTo(expected))
[TestCase]
! NUnit allows for parameter-ized tests.从我目前的角度来看(经过 8 个月的开发,平均有 10 名开发人员),我建议反对使用 MSTest,原因如下
慢的。我的意思不是测试代码
你写的——这在你的控制之下。
我的意思是运行这些的框架
测试很慢,无论是运行
测试套件、单个测试等。
这总是会导致并发症
当几个开发人员正在工作时
在其上(重新创建例如元数据
ETC。)。其他所有测试套件都没有
需要一个元数据文件。这是一种
很高兴组织你的测试,但是你
可以通过以下方式实现相同的目的
命名空间、类和方法名称。
换句话说,如果 8 个月前我必须再次做出决定,我可能会选择 NUnit。我可能没有集成的测试结果报告,但开发人员将拥有更无缝的测试体验。
From my current perspective (after 8 months of development with about 10 developers on average) I would advise against using MSTest for the following reasons
slow. I don't mean the test code that
you write - that's under your control.
I mean the framework running those
tests is slow, whether it's running a
test suite, single tests etc.
which always leads to complications
when several developers are working
on it (recreating e.g. the metadata
etc.). Every other test suite doesn't
need a metadata file. It is kind of
nice to organize your tests but you
can achieve the same through
namespaces, classes and method names.
In other words, if I would have to decide again 8 months ago, I would probably take NUnit. I may not have the integrated test results report, but developers would have a more seamless testing experience.
这是我使用 MS Test 的经验
补充:我们现在还有一些测试,甚至无法说出有多少。由于 OutOfMemoryExceptions 和其他不稳定问题,不可能再从 Visual Studio 运行它们。我们从脚本运行测试。在 Visual Studio 中查看测试结果很容易,但是当解决方案打开时,VS 崩溃(每次)。因此我们需要使用文本搜索来搜索失败的测试。集成工具不再有任何优势。
另一个更新:我们现在使用 VS 2013。很多事情都改变了。自我们开始以来,他们第三次重写了 MS Test 测试运行程序。这导致了很多重大变化,但两个新版本都没有做得更好。我们很高兴没有使用 MS Test 的奇特功能,因为它们都不再受支持。实在是太可惜了。我们仍然使用脚本来构建和运行所有单元测试,因为它更方便。 Visual Studio 需要几分钟才能开始运行测试(编译后直到第一次测试开始的时间测量)。他们可能通过更新修复它,这可能是我们项目的特定问题。然而,在运行相同的测试时,Resharper 的速度要快得多。
结论:至少与Resharper结合使用时,MS Test是有用的。我希望他们最终能弄清楚测试运行程序应该如何编写,并且在我们下次更新 Visual Studio 时不会进行此类重大更改。
Here is my experience with MS Test
Addition: We have some more tests now, can't even say how many. It is impossible to run them all anymore from Visual Studio, because of OutOfMemoryExceptions and other instability problems. We run the tests from scripts. It would be easy to view test results in Visual Studio, but when the solution is open, VS crashes (every time). So we need to search the failing tests using text search. There is no advantage of an integrated tool anymore.
Another Update: We are using VS 2013 now. A lot of things changed. They rewrote the MS Test test runner for the third time since we started. This caused a lot of breaking changes, but neither new version was doing anything better. We are glad that we didn't use the fancy features of MS Test, because they are all not supported anymore. It's really a shame. We are still using scripts to build and run all unit tests, because it is handier. Visual Studio required a few minutes to start running tests (time measures after compilation until first test starts). They probably fixes it with an update and this might be a specific problem of our project. However, Resharper is much quicker when running the same tests.
Conclusion: At least in combination with Resharper, MS Test is useful. And I hope that they finally find out how the test runner should be written and won't do this kind of breaking changes when we update Visual Studio next time.
NUnit 可以与 Visual Studio 结合使用。它是一个框架而不是一个单独的程序。所以你可以关心一下它是否适合你:)。
(来源:codeplex.com)
“安装插件后,您将在工具菜单下找到一个新的子菜单。”
有关导入它的更多信息,请参阅 http://nunitit.codeplex.com/。
另外,通过SO的搜索可以找到很多东西。 这个例如,主题列出了 NUnit 相对于 MS 标准测试的优势。
NUnit can be used in combination with visual studio. It is a framework not a separate program. So you could care an see if it suits you :).
(source: codeplex.com)
"After installing the plugin you'll find a new submenu under the tools menu."
See http://nunitit.codeplex.com/ for more information on importing it.
Also, a lot can be found using the search of SO. This topic lists advantages of NUnit over MS standard testing for instance.
MS-Test 相对于 NUnit 的最大优势是 MS-Test 可以使用反射生成模拟对象。
我发现它非常有用
Biggest advantage MS-Test over NUnit is MS-Test can generate mock objects using Reflection.
I found it very usefull
NUnit 可与 VS 标准版配合使用。
NUnit works with the Standard edition of VS.
我不确定其他人,但 NUnit 提供了很好的 GUI 和控制台来运行您的单元测试,您还可以生成 NUnit 测试执行结果的报告,该报告将提供测试是否失败或通过以及测试时间的详细信息参加单元测试
I am not sure of others but NUnit provides nice GUI and console to run your Unit Tests and also you can generate report of the result of NUnit Test execution which would give the detail of whethere the test has failed or passed and also what time did it take for your unit test
NUnit 是一个单元测试框架,resharper 也支持它。我认为你使用的是微软的单元测试框架,所以NUnit只是微软产品的替代品;)
这里是NUnit主页的链接: http://nunit.org
NUnit is a unit testing framework, which is also supported by resharper. I think you are using Microsoft's unit testing framework, so NUnit is just an alternative to Microsoft's product ;)
Here is the link to the homepage of NUnit: http://nunit.org
在 NUnit 中,测试不是并行执行的。相反,似乎所有测试都在单个线程上执行。在 MSTest 中,每个测试都在单独的线程上实例化,这导致运行交错。因此,如果测试 A 的成功依赖于测试 B,则它可能会失败,因为测试 B 可能会在测试 A 运行时开始运行。
In NUnit, tests are not executed in parallel. Rather, it appears that all tests execute on a single thread. In MSTest, each test is instantiated on a separate thread, this results the runs being interleaved. Therefore, if test A depends on test B for its success, it likely will fail as test B will likely start running as test A is running.