使用 NUnit 的 TFS Build 2010 代码覆盖率

发布于 2024-11-27 12:26:49 字数 393 浏览 2 评论 0原文

我想知道你们中是否有人有在运行 NUnit 测试时在 TFS Build Server 2010 中生成代码覆盖率报告的经验。

我知道可以使用打包的替代方案(MSTest + 在 testrunco​​nfig 文件上启用覆盖)轻松完成此操作,但使用 NUnit 时会涉及更多一些事情。我在这里和那里找到了一些指向 NCover 的信息,但它似乎已经过时了。我想知道是否还有其他替代方案以及是否有人实际实施了这一点。

以下是有关我们的环境/需求的更多信息: - TFS 构建服务器 2010 - 测试位于普通类库中(不是测试库 - 即没有关联的 testrunco​​nfig 文件),并且在 NUnit 中实现。我们没有 MSTest。 - 我们有兴趣将覆盖率报告作为每个构建的一部分,并在可能的情况下为通过/失败标准设置覆盖率阈值要求。

I was wondering if any of you guys had any experience generating code coverage reports in TFS Build Server 2010 while running NUnit tests.

I know it can be easily done with the packaged alternative (MSTest + enabling coverage on the testrunconfig file), but things are a little more involved when using NUnit. I've found some info here and there pointing to NCover, but it seems outdated. I wonder if there are other alternatives and whether someone has actually implemented this or not.

Here's more info about our environment/needs:
- TFS Build Server 2010
- Tests are in plain class libraries (not Test libraries - i.e., no testrunconfig files associated), and are implemented in NUnit. We have no MSTests.
- We are interested in running coverage reports as part of each build and if possible setting coverage threshold requirements for pass/fail criteria.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

赏烟花じ飞满天 2024-12-04 12:26:49

我们已经使用 NUnit-NCover 完成了这项工作,并对我们的结果非常满意。
NUnit 执行之后是 NUnitTfs 执行,以便在构建日志中发布我们的测试结果。然后 NCover 启动,生成我们的代码覆盖率结果。

一个主要的缺点是,设置正确调用 NCover 的参数并不是一件容易的事。但自从我安装了它之后,我就再也不需要维护它了。

有两件事可能会带来缺点:

  • NUnitTfs 不能很好地与 NCover 配合使用(至少我找不到在同一步骤中执行两者的方法,所以(因为 NCover 调用 NUnit)我必须运行单元测试两次:(1) 获取测试结果,(2) 获取 Nover 的覆盖率结果,这自然会使我的构建持续更长时间
  • 设置正确调用的参数 。 NCover 并不是微不足道的。但是自从我安装了它之后,我就再也不需要维护它了

无论如何,生成的报告(尤其是趋势方面)对于监控我们的代码如何随时间演变非常有用。特别是如果您正在开发一个平台(而不是短期项目),趋势报告就非常有价值

编辑

我会尝试快速地介绍一下。我是如何实现这一点的肮脏方式,我希望它有用。目前,我们的构建服务器上有 NOver 3.4.12。
关于 NUnit 程序集的简单命名约定是,如果我们有一个生产程序集“123.dll”,则存在另一个名为“123_nunit.dll”的程序集来实现其测试。因此,每个构建都有几个令人感兴趣的 *_nunit.dll 程序集。

构建过程模板中“如果不禁用测试”下的部分是为了实现我们的目标而重新设计的部分,特别是名为“为测试程序集运行 MSTest”的部分。整个实现在这里,经过一些清理以使流程更容易明白了(图片太大,无法直接插入此处)。

首先,在构建过程模板和构建过程模板中实现了一些附加参数。然后可以在每个构建定义中进行设置:
在此处输入图像描述

然后我们在“中形成 NUnit 参数”制定 nunitCommandLine":

String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL))

然后在“调用 NUnit”中使用
在此处输入图像描述

以防万一这成功 &我们已经为此构建设置了覆盖率,我们将转到“生成 NCover NCCOV”(此特定程序集的覆盖率文件)。为此,我们使用以下参数调用 NCover.Console.exe:

String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""",
              NUnitPath,
              Path.GetFileName(nunitDLL),
              Path.GetDirectoryName(nunitDLL),
              Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)),
              Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"),
              Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""),
              BuildDetail.BuildNumber)

所有这些都在 foreach 循环“对于所有 nunit dll”中运行。当我们退出循环时,我们进入“最终 NCover 活动”&首先是“合并 NCCovs”部分,其中再次执行 NCover.Console.exe - 这次使用不同的参数:

String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ",
              Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
              BuildDetail.BuildNumber,
              NCoverDropLocation,
              BuildDetail.BuildDefinition.TeamProject
              )

运行后,我们已达到此版本的所有 NCCOV 文件合并为一个名为的 NCCOV 文件的程度在构建+趋势文件(在其整个生命周期中监视构建)已使用当前构建的元素进行更新之后。

我们现在只需生成最终的 HTML 报告,这是在“生成最终 NCover 代表”中完成的,其中我们使用以下参数调用 NCover.reporting:

String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ",
              Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
              BuildDetail.BuildNumber,
              PathForNCoverResults,
              NCoverDropLocation,
              BuildDetail.BuildDefinition.TeamProject,
              BuildType
              )

We 've done it with NUnit-NCover and are pretty happy with our results.
NUnit execution is followed by NUnitTfs execution in order to get our testing results published in the Build Log. Then NCover kicks in, generating our code coverage results.

One major thing that poses as a disadvantage is fact that setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it.

Two things could pose as disadvantages:

  • NUnitTfs doesn't work well with NCover (at least I couldn't find a way to execute both in the same step, so (since NCover invokes NUnit) I have to run Unit tests twice: (1) to get the test results and (2) to get coverage results over NCover. Naturally, that makes my builds last longer.
  • Setting up the arguments for properly invoking NCover wasn't trivial. But since I installed it, I never had to maintain it .

In any case, the resulting reporting (especially the Trend aspect) is very useful in monitoring how our code evolves within time. Especially if you 're working on a Platform (as opposed to short-timed Projects), Trend reports are of great value.

EDIT
I 'll try to present in a quick & dirty manner how I 've implemented this, I hope it can be useful. We currently have NCover 3.4.12 on our build server.
Our simple naming convention regarding our NUnit assemblies is that if we have a production assembly "123.dll", then another assembly named "123_nunit.dll" exists that implements its tests. So, each build has several *_nunit.dll assemblies that are of interest.

The part in the build process template under "If not disable tests" is the one that has been reworked in order to achieve our goals, in particular the section that was named "Run MSTest for Test Assemblies". The whole implementation is here, after some cleanups to make the flow easier to be understood (pic was too large to be directly inserted here).

At first, some additional Arguments are implemented in the Build Process Template & are then available to be set in each build definition:
enter image description here

We then form the NUnit args in "Formulate nunitCommandLine":

String.Format("{0} /xml={1}\\{2}.xml", nunitDLL, TestResultsDirectory, Path.GetFileNameWithoutExtension(nunitDLL))

This is then used in the "Invoke NUnit"
enter image description here

In case this succeeds & we have set coverage for this build we move to "Generate NCover NCCOV" (the coverage file for this particular assembly). For this we invoke NCover.Console.exe with the following as Args:

String.Format("""{0}"" ""{1}"" //w ""{2}"" //x ""{3}\{4}"" //literal //ias {5} //onlywithsource //p ""{6}""",
              NUnitPath,
              Path.GetFileName(nunitDLL),
              Path.GetDirectoryName(nunitDLL),
              Path.GetDirectoryName(Path.GetDirectoryName(nunitDLL)),
              Path.GetFileName(nunitDLL).Replace("_nunit.dll", ".nccov"),
              Path.GetFileNameWithoutExtension(nunitDLL).Replace("_nunit", ""),
              BuildDetail.BuildNumber)

All these run in the foreach loop "For all nunit dlls". When we exit the loop, we enter "Final NCover Activities" & at first the part "Merge NCCovs", where NCover.Console.exe is executed again - this time with different args:

String.Format("""{0}\*.nccov"" //s ""{0}\{1}.nccov"" //at ""{2}\{3}\{3}.trend"" //p {1} ",
              Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
              BuildDetail.BuildNumber,
              NCoverDropLocation,
              BuildDetail.BuildDefinition.TeamProject
              )

When this has run, we have reached the point where all NCCOV files of this build are merged into one NCCOV-file named after the build + the Trend file (that monitors the build throughout its life) has been updated with the elements of this current build.

We now have to only generate the final HTML report, this is done in "Generate final NCover rep" where we invoke NCover.reporting with the following args:

String.Format(" ""{0}\{1}.nccov"" //or FullCoverageReport //op ""{2}\{1}_NCoverReport.html"" //p ""{1}"" //at ""{3}\{4}\{4}_{5}.trend"" ",
              Path.GetDirectoryName(Path.GetDirectoryName(testAssemblies(0))),
              BuildDetail.BuildNumber,
              PathForNCoverResults,
              NCoverDropLocation,
              BuildDetail.BuildDefinition.TeamProject,
              BuildType
              )
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文