开发基于叮当的工具时,如何加快编译时间?
我正在开发基于叮当的工具 (即,a 但是,重新编译源文件的时间比我想要的要长。 具体来说,对于下面的小示例程序,大约需要7个 秒可以编译这个文件,我只#included
abar 最低限度。一旦我开始拉动AST匹配者,编译时间就会攀升至 20秒以上。在开发周期中,这是很多摩擦, 特别是在学习新的API并因此修复大量时 语法错误。
我发现了一些幻灯片 建立clang/llvm效率 蒂尔曼·谢勒(Tilmann Scheller)在Eurollvm 2015上的演讲中,该演讲表现不佳 通过更改使用哪个编译器的改进(小于2倍) (具体地,使用clang
编译已编译的 GCC
)。我忽略了链接改进,因为链接是 就我而言,可以忍受的快速。
我尝试使用 gcc预编译标头 (PCH),但结果混合在一起(下面的详细信息)。同时使用PCH汇编 大约一半,首先产生PCH需要更多 是普通汇编的两倍,产生一个巨大的PCH文件 (300-500 MB),有点麻烦(我必须重组 #includes
在我的项目中,管理其他依赖项和文件, 并仔细监视该过程,以确保使用PCH)。
理想情况下,我想在一秒钟内获得编译时间。毕竟, 我的源文件仅为100行代码(LOC)。现在, 预处理器输出约为200k,并且单词模板
出现 在其中5000多个行中,我完全知道编译器正在做 不仅仅是明显的。但是我的代码不是直接使用的 不仅仅是其中的一小部分。
作为比较的点,当使用python绑定时,它需要 0.05秒至 run 一个相对最小的复杂性的脚本。我知道 libclang
, Clang的C绑定,但我不想在C中写下我的工具。
此外,libclang(对于C和Python)仅揭示了C ++ API中可用的一小部分信息。由于这个问题最初是编写的,所以我花了几个月的时间沿着libclang路线走了,最终由于缺少的信息而放弃了它是无法使用的(示例是获得二进制表达式的操作员,尽管现在已经解决了该表达式,但是还有很多其他表达式)。
对不同变化的一些测量,使用GCC-9.3的中位数测量中值测量:
选项 | 无PCH | GEN PCH | 使用PCH | PCH尺寸 |
---|---|---|---|---|
G ++ | 6.8 S | 14 S | 2.5 S 2.5 S | 342 MB |
G ++ -O2 -O2 | 6.9 S | 14 S | 2.5 S | 345 MB |
G ++ -G | 8.6 S | 18 S | 4.1 S | 472 MB |
G ++ -G -O2 | 8.9 S | 18 S | 478 S | 478 |
MB代码> g ++ -g 配置是我最关心的一种。
@HolyBlackCat建议以clang
作为编译器进行尝试。以下是clang+llvm -14.0.0的时间:
选项 | 无PCH | GEN PCH | 使用PCH | PCH尺寸 |
---|---|---|---|---|
clang | 5.6 S | 5.1 S 2.3 S | 2.3 S | 62 MB |
clang -O2 | 5.6 S S | 5.2 s | 2.5 S | 62 MB |
clang -G | 5.6 S | 5.2 S | 2.4 S | 62 MB |
clang -G -O2 | 5.7 S | 5.1 S 5.1 S | 2.6 S 2.6 S | 62 MB |
肯定是对<<<<代码> G ++ ,并且很可能是我将要做的事情,尽管它的目标不足。
为了完整性,我还测量了GCC-122.1,并发现所有测量时间的GCC-9.3都比GCC-9.3差2%。
我的设置:
- 使用
clang+llvm-14.0.0
从 github.com 。 - X86_64上使用Linux Mint 20.1上的GCC-9.3.0编译。
- 四核Intel I5-6600K CPU @ 3.50HGz。
- 8 GB RAM。
- Linux在VMware Workstation 12中运行。(VM不是 问题。 VM中的许多C ++项目都非常迅速地编译 比Windows主机下的汇编快。)
- 主机是Windows 10,具有32 GB物理RAM。
汇编的源文件:
// print-tu.cc
// Print contents of a Translation Unit.
// clang
#include "clang/AST/ASTConsumer.h" // clang::ASTConsumer
#include "clang/AST/ASTContext.h" // clang::ASTContext
#include "clang/AST/DeclBase.h" // clang::Decl
#include "clang/AST/DeclGroup.h" // clang::DeclGroupRef
#include "clang/Basic/SourceLocation.h" // clang::SourceLocation
#include "clang/Frontend/CompilerInstance.h" // clang::CompilerInstance
#include "clang/Frontend/CompilerInvocation.h" // clang::CompilerInvocation
#include "clang/Frontend/FrontendAction.h" // clang::FrontendAction, clang::ASTFrontendAction
#include "clang/Tooling/CommonOptionsParser.h" // clang::tooling::CommonOptionsParser
#include "clang/Tooling/Tooling.h" // clang::tooling::ClangTool, clang::tooling::newFrontendActionFactory
// llvm
#include "llvm/ADT/StringRef.h" // llvm::StringRef
#include "llvm/Support/CommandLine.h" // llvm::cl::extrahelp
// libc++
#include <iostream> // std::cout, etc.
// libc
#include <assert.h> // assert
using clang::tooling::CommonOptionsParser;
using clang::tooling::ClangTool;
using clang::tooling::newFrontendActionFactory;
using clang::ASTConsumer;
using clang::ASTContext;
using clang::ASTFrontendAction;
using clang::CompilerInstance;
using clang::CompilerInvocation;
using clang::Decl;
using clang::DeclGroupRef;
using clang::DiagnosticConsumer;
using clang::FileManager;
using clang::FrontendAction;
using clang::PCHContainerOperations;
using clang::SourceLocation;
using clang::TargetOptions;
using llvm::StringRef;
using std::cout;
// Apply a custom category to all command-line options so that they are the
// only ones displayed.
static llvm::cl::OptionCategory MyToolCategory("my-tool options");
// CommonOptionsParser declares HelpMessage with a description of the common
// command-line options related to the compilation database and input files.
// It's nice to have this help message in all tools.
static llvm::cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
// A help message for this specific tool can be added afterwards.
static llvm::cl::extrahelp MoreHelp("\nMore help text...\n");
// Implement ASTConsumer
class MyASTConsumer : public ASTConsumer {
public: // data
// The context for the TU, as established by 'Initialize'.
ASTContext *m_astContext;
public: // methods
MyASTConsumer()
: m_astContext(NULL)
{}
// This is called at the start of the TU.
virtual void Initialize(ASTContext &ctx) override
{
m_astContext = &ctx;
}
// This is called at the end of the TU.
virtual void HandleTranslationUnit(ASTContext &ctx) override
{
cout << "in HandleTranslationUnit\n";
assert(m_astContext == &ctx);
}
virtual bool HandleTopLevelDecl(DeclGroupRef declGroup) override
{
cout << "in HandleTopLevelDecl\n";
assert(m_astContext);
for (Decl const *decl : declGroup) {
SourceLocation loc = decl->getLocation();
cout << " decl at "
<< loc.printToString(m_astContext->getSourceManager()) << '\n';
cout << " kind name: " << decl->getDeclKindName() << '\n';
}
return true;
}
};
// Implement the FrontendAction interface.
//
// Inheriting from ASTFrontendAction provides definitions for
// 'ExecuteAction' and 'usesPreprocessorOnly'.
class MyFrontendAction : public ASTFrontendAction {
public:
virtual std::unique_ptr<ASTConsumer> CreateASTConsumer(
CompilerInstance &ci,
StringRef inFile) override
{
cout << "in CreateASTConsumer\n";
TargetOptions const &targetOptions = ci.getTargetOpts();
cout << " target options triple: " << targetOptions.Triple << '\n';
cout << " inFile: " << inFile.str() << '\n';
return std::unique_ptr<ASTConsumer>(new MyASTConsumer());
}
};
int main(int argc, const char **argv)
{
auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
if (!ExpectedParser) {
// Fail gracefully for unsupported options.
llvm::errs() << ExpectedParser.takeError();
return 2;
}
CommonOptionsParser& OptionsParser = ExpectedParser.get();
ClangTool Tool(OptionsParser.getCompilations(),
OptionsParser.getSourcePathList());
// Arrange to run 'MyFrontendAction' on each TU.
return Tool.run(newFrontendActionFactory<MyFrontendAction>().get());
}
// EOF
编译命令(没有-G
或-O2
):
g++ -c -o print-tu.o print-tu.cc $(llvm-config --cxxflags)
I'm developing a Clang-based tool
(i.e., a ClangTool
),
but the time to recompile a source file is longer than I would like.
Specifically, for the small example program below, it takes about 7
seconds to compile this one file, and I've only #included
a bare
minimum. Once I start pulling in AST matchers, compile times climb to
20 seconds and more. That's a lot of friction in the development cycle,
especially when learning a new API and hence fixing a large number of
syntax errors.
I found some slides called
Building Clang/LLVM efficiently
by Tilmann Scheller from a talk at EuroLLVM 2015 which show modest
improvement (less than a factor of 2) by changing which compiler is used
(specifically, compiling with a clang
that has been compiled bygcc
). I'm ignoring the linking improvements because linking is
tolerably fast in my case.
I tried using GCC precompiled headers
(PCH), but the results are mixed (details below). While compilation with PCH takes
about half as long, generating the PCH in the first place takes more
than twice as long as ordinary compilation, produces a huge PCH file
(300-500 MB), and is somewhat of a hassle to use (I have to reorganize#includes
across my project, manage additional dependencies and files,
and carefully monitor the process to be sure the PCHs are being used).
Ideally, I would like to get compile times under one second. After all,
my source file is only about 100 lines of code (LOC). Now, the
preprocessor output has about 200K LOC, and the word template
appears
in over 5000 of those lines, so I'm fully aware the compiler is doing
more than is immediately apparent. But my code is not directly using
more than a tiny fraction of that.
As a point of comparison, when using the Python bindings, it takes about
0.05 seconds to run a script of comparably minimal complexity. I am aware of libclang
,
the C bindings for Clang, but I don't want to write my tool in C.
Furthermore, libclang (for both C and Python) only exposes a small fraction of the information available in the C++ API. Since this question was originally written, I spent a couple months going down the libclang route only to eventually give up on it as unusable due to the missing information (an example is getting the operator of a binary expression, although that one has now been addressed, but there are many others).
Some measurements of different variations, median-of-5 measurement with GCC-9.3:
Options | No PCH | Gen PCH | Use PCH | PCH Size |
---|---|---|---|---|
g++ | 6.8 s | 14 s | 2.5 s | 342 MB |
g++ -O2 | 6.9 s | 14 s | 2.5 s | 345 MB |
g++ -g | 8.6 s | 18 s | 4.1 s | 472 MB |
g++ -g -O2 | 8.9 s | 18 s | 4.3 s | 478 MB |
The g++ -g
configuration is the one I care most about.
@HolyBlackCat suggested trying with clang
as the compiler. Here are the times for Clang+LLVM-14.0.0:
Options | No PCH | Gen PCH | Use PCH | PCH Size |
---|---|---|---|---|
clang | 5.6 s | 5.1 s | 2.3 s | 62 MB |
clang -O2 | 5.6 s | 5.2 s | 2.5 s | 62 MB |
clang -g | 5.6 s | 5.2 s | 2.4 s | 62 MB |
clang -g -O2 | 5.7 s | 5.1 s | 2.6 s | 62 MB |
That's certainly an improvement over g++
, and may well be what I do going forward, although it falls short of my sub-second goal.
For completeness, I also measured GCC-12.1, and found its to be about 2% worse than GCC-9.3 for all measured times.
My setup:
- Using binary distribution of
clang+llvm-14.0.0
downloaded from github.com. - Compiling with GCC-9.3.0 on Linux Mint 20.1 for x86_64.
- Quad-core Intel i5-6600K CPU @ 3.50HGz.
- 8 GB RAM.
- Linux is running inside VMware Workstation 12. (The VM is not the
problem. Many C++ projects compile very quickly in the VM, even
faster than compilation under the Windows host.) - Host is Windows 10 with 32 GB physical RAM.
Source file to compile:
// print-tu.cc
// Print contents of a Translation Unit.
// clang
#include "clang/AST/ASTConsumer.h" // clang::ASTConsumer
#include "clang/AST/ASTContext.h" // clang::ASTContext
#include "clang/AST/DeclBase.h" // clang::Decl
#include "clang/AST/DeclGroup.h" // clang::DeclGroupRef
#include "clang/Basic/SourceLocation.h" // clang::SourceLocation
#include "clang/Frontend/CompilerInstance.h" // clang::CompilerInstance
#include "clang/Frontend/CompilerInvocation.h" // clang::CompilerInvocation
#include "clang/Frontend/FrontendAction.h" // clang::FrontendAction, clang::ASTFrontendAction
#include "clang/Tooling/CommonOptionsParser.h" // clang::tooling::CommonOptionsParser
#include "clang/Tooling/Tooling.h" // clang::tooling::ClangTool, clang::tooling::newFrontendActionFactory
// llvm
#include "llvm/ADT/StringRef.h" // llvm::StringRef
#include "llvm/Support/CommandLine.h" // llvm::cl::extrahelp
// libc++
#include <iostream> // std::cout, etc.
// libc
#include <assert.h> // assert
using clang::tooling::CommonOptionsParser;
using clang::tooling::ClangTool;
using clang::tooling::newFrontendActionFactory;
using clang::ASTConsumer;
using clang::ASTContext;
using clang::ASTFrontendAction;
using clang::CompilerInstance;
using clang::CompilerInvocation;
using clang::Decl;
using clang::DeclGroupRef;
using clang::DiagnosticConsumer;
using clang::FileManager;
using clang::FrontendAction;
using clang::PCHContainerOperations;
using clang::SourceLocation;
using clang::TargetOptions;
using llvm::StringRef;
using std::cout;
// Apply a custom category to all command-line options so that they are the
// only ones displayed.
static llvm::cl::OptionCategory MyToolCategory("my-tool options");
// CommonOptionsParser declares HelpMessage with a description of the common
// command-line options related to the compilation database and input files.
// It's nice to have this help message in all tools.
static llvm::cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
// A help message for this specific tool can be added afterwards.
static llvm::cl::extrahelp MoreHelp("\nMore help text...\n");
// Implement ASTConsumer
class MyASTConsumer : public ASTConsumer {
public: // data
// The context for the TU, as established by 'Initialize'.
ASTContext *m_astContext;
public: // methods
MyASTConsumer()
: m_astContext(NULL)
{}
// This is called at the start of the TU.
virtual void Initialize(ASTContext &ctx) override
{
m_astContext = &ctx;
}
// This is called at the end of the TU.
virtual void HandleTranslationUnit(ASTContext &ctx) override
{
cout << "in HandleTranslationUnit\n";
assert(m_astContext == &ctx);
}
virtual bool HandleTopLevelDecl(DeclGroupRef declGroup) override
{
cout << "in HandleTopLevelDecl\n";
assert(m_astContext);
for (Decl const *decl : declGroup) {
SourceLocation loc = decl->getLocation();
cout << " decl at "
<< loc.printToString(m_astContext->getSourceManager()) << '\n';
cout << " kind name: " << decl->getDeclKindName() << '\n';
}
return true;
}
};
// Implement the FrontendAction interface.
//
// Inheriting from ASTFrontendAction provides definitions for
// 'ExecuteAction' and 'usesPreprocessorOnly'.
class MyFrontendAction : public ASTFrontendAction {
public:
virtual std::unique_ptr<ASTConsumer> CreateASTConsumer(
CompilerInstance &ci,
StringRef inFile) override
{
cout << "in CreateASTConsumer\n";
TargetOptions const &targetOptions = ci.getTargetOpts();
cout << " target options triple: " << targetOptions.Triple << '\n';
cout << " inFile: " << inFile.str() << '\n';
return std::unique_ptr<ASTConsumer>(new MyASTConsumer());
}
};
int main(int argc, const char **argv)
{
auto ExpectedParser = CommonOptionsParser::create(argc, argv, MyToolCategory);
if (!ExpectedParser) {
// Fail gracefully for unsupported options.
llvm::errs() << ExpectedParser.takeError();
return 2;
}
CommonOptionsParser& OptionsParser = ExpectedParser.get();
ClangTool Tool(OptionsParser.getCompilations(),
OptionsParser.getSourcePathList());
// Arrange to run 'MyFrontendAction' on each TU.
return Tool.run(newFrontendActionFactory<MyFrontendAction>().get());
}
// EOF
Compilation command (without -g
or -O2
):
g++ -c -o print-tu.o print-tu.cc $(llvm-config --cxxflags)
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论