我有一个非常大的代码库,其中包含大量的单元测试(使用 CppUnit)。 我需要计算出这些测试执行了多少百分比的代码,并且(理想情况下)生成某种报告,告诉我在每个库或每个文件的基础上,有多少代码已被执行。
关键在于:它必须完全无人值守地运行(最终在持续集成构建中),并且必须是跨平台的(至少是 WIN32 和 *nix)。
谁能推荐一个或一组工具来帮助我做到这一点? 我无法放弃 CppUnit(我也不想这样做——它太糟糕了),但除此之外,我渴望听到您可能提出的任何建议。
干杯,
I have a very large code base that contains extensive unit tests (using CppUnit). I need to work out what percentage of the code is exercised by these tests, and (ideally) generate some sort of report that tells me on a per-library or per-file basis, how much of the code was exercised.
Here's the kicker: this has to run completely unnatended (eventually inside a continuous integration build), and has to be cross platform (well, WIN32 and *nix at least).
Can anyone suggest a tool, or set of tools that can help me do this? I can't change away from CppUnit (nor would I want to - it kicks ass), but otherwise I'm eager to hear any recommendations you might have.
Cheers,
发布评论
评论(5)
如果您可以使用 GNU GCC 作为编译器,则 gcov 工具运行良好。 整个过程完全自动化非常容易。
If you can use GNU GCC as your complier, then the gcov tool works well. It's very easy to fully automate the whole process.
查看我们的 SD C++ 测试覆盖率工具。 对于 GCC 和 MSVC6 都可以获得它。
它具有低开销的探测数据收集、覆盖在代码上的覆盖率数据的良好显示,以及跨方法/类/文件/目录级别的覆盖率汇总的完整报告生成。
编辑:2015 年 8 月:现在通过 Visual Studio 2015 支持 GCC5 和各种 MS 方言。要在 Linux 下使用这些工具,您需要 Wine,但这些工具提供 Linux 原生 sh 脚本和基于 Linux/Java 的 UI,所以该工具感觉就像那里的原生 Linux 工具一样。
Check out our SD C++ Test Coverage tool. It can be obtained for GCC, and for MSVC6.
It has low overhead probe data collection, a nice display of coverage data overlayed on your code, and complete report generation with rollups on coverage across the method/class/file/directory levels.
EDIT: Aug 2015: Now supports GCC5 and various MS dialects through Visual Studio 2015. To use these tools under Linux, you need Wine, but there the tools provide Linux-native sh scripting and a Linux/Java based UI, so the tool feels like a native Linux tool there.
我想我应该指定编译器 - 我们在 Linux 上使用 gcc,在 WIn32 上使用 MSVC 6(是的,我知道,它很旧,但它(大部分)对我们有用)。
因此,gcov 不适用于我们的 Win32 构建,而 Bullseye 不适用于我们的 Linux 构建。
再说一遍,也许我只需要在一个操作系统中进行覆盖......
I guess I should have specified the compiler - we're using gcc for Linux, and MSVC 6 (yeah I know, it's old, but it works (mostly) for us) for WIn32.
For that reasons, gcov won't work for our Win32 builds, and Bullseye won't work for our Linux builds.
Then again maybe I only need coverage in one OS...
如果您使用 GCC 工具链,gcov 将为您提供源、功能和分支覆盖率统计信息。 gcov 适用于 MinGW 和 Cygwin。 这将使您能够获取覆盖率统计数据,并发出经过检测的源代码,使您能够可视化未执行的代码。
然而,如果你真的想用漂亮的报告来完成它,可以将 gcov 与 结合使用lcov 是要走的路。 lcov 将为您提供范围为文件和目录的条形报告、功能覆盖率统计数据以及用于显示覆盖率的颜色编码源文件浏览(绿色表示已执行,红色表示未执行...)。
lcov 在 Linux 上很容易,但在 Cygwin 上可能需要一些 Perl 破解。 我个人在 Windows 上执行脚本(lcov 是用 perl 实现的)时遇到了一些问题。 我已经得到了一个可以使用的破解版本,但请预先警告。
另一种方法是在 Windows 上执行 gcov 发出,并在 Linux 上执行 lcov 后处理,它肯定可以开箱即用。
If you are using the GCC toolchain, gcov is going to get you source, functional, and branch coverage statistics. gcov works fine for MinGW and Cygwin. This will allow you to get coverage statistics as well as emitting instrumented source code that allows you to visualize unexecuted code.
However, if you really want to hit it out of the park with pretty reports, using gcov in conjunction with lcov is the way to go. lcov will give you bar reports scoped to files and directories, functional coverage statistics, and color coded source file browsing to show coverage (green means executed, red means not...).
lcov is easy on Linux, but may require some perl hacking on Cygwin. I personally have had some problems executing the scripts (lcov is implemented in perl) on Windows. I've gotten a hacked up version to work, but be forewarned.
Another approach is doing the gcov emit on windows, and doing the lcov post processing on Linux, where it will surely work out of the box.
我应该使用哪个工具?
这篇文章介绍了另一位开发人员寻找 C++ 代码覆盖工具的挫败感。 作者最终的解决方案是Bullseye Coverage。
靶心覆盖功能:
至于挂钩持续集成,这取决于您使用的 CI 解决方案,但您可能可以将仪器/覆盖率测量步骤挂钩到用于自动化测试的 make 文件中。
测试 Linux 与 Windows?
只要所有测试在两种环境中都能正确运行,您就应该能够很好地测量其中一种环境的覆盖率。 (尽管 Bullseye 似乎支持两个平台)。 但为什么不在这两种环境中进行持续集成构建呢? 如果您在两种环境中向客户交付,那么您需要在这两种环境中进行测试。
因此,听起来您可能需要设置两台连续构建服务器,一台用于 Linux 构建,一台用于 Windows 构建。 也许这可以通过一些虚拟化软件轻松实现,例如 vmware 或 virtualbox。 您可能不需要在两个操作系统上运行代码覆盖率指标,但您绝对应该在两个操作系统上运行单元测试。
Which tool should I use?
This article describes another developers frustrations searching for C++ code coverage tools. The author's final solution was Bullseye Coverage.
Bullseye Coverage features:
As for hooking into your continuous integration, it depends on which CI solution you use, but you can likely hook the instrumentation / coverage measurement steps into the make file you use for automated testing.
Testing Linux vs Windows?
So long as all your tests run correctly in both environments, you should be fine measuring coverage on one or the other. (Though Bullseye appears to support both platforms). But why aren't you doing continuous integration builds in both environments?? If you deliver to clients in both environments then you need to be testing in both.
For that reason, it sounds like you might need to have two continuous build servers set up, one for a linux build and one for a windows build. Perhaps this can be easily accomplished with some virtualization software like vmware or virtualbox. You may not need to run code coverage metrics on both OSs, but you should definitely be running your unit tests on both.