我们的构建速度太慢了。 它在 Linux 上使用嵌套的 gnu makefile。 它从同一源树为三个不同的目标创建三个构建。 它使用符号链接依次指向三个并行目录树中的每一个。 我们可以通过在子目录中使用 make 来进行部分构建，这可以节省时间，但如果我们的工作跨越多个目录，我们必须至少为三个目标之一构建，这至少需要 45 分钟。 仅构建子目录可能“仅”需要 5-10 分钟。
您知道有哪些需要快速检查的事情可能会导致此构建系统陷入困境吗？ 例如，是否有比符号链接更快的替代方案？
另外：我看过关于递归makefile的论文。 有谁直接知道扁平化目前拥有许多 makefile（大约 800 个）和超过 450 万行源代码的 Makefile 系统会产生什么影响？ 目前，人们喜欢通过在该目录中使用 make 来构建当前的子目录或进程（嵌入式 linux 目标）。
我刚刚了解到，直到最近，构建的时间是原来的两倍 (wince)，此时发布工程师部署了 ccache。
Our build is dog slow. It uses nested gnu makefiles on linux. It creates three builds for three different targets from the same source tree. It uses symlinks to point to each of the three parallel directory trees in turn. We can do partial builds by using make inside subdirectories, which saves time, but if our work spans multiple directories we must build for at least one of the three targets and that takes a minimum of 45 minutes. A subdirectory only build may take "only" 5-10 minutes.
Do you know of any quick things to check that may be bogging down this build system? For example, is there a faster alternative to symlinks?
Addition: I've seen the paper regarding recursive makefiles. Does anyone know firsthand what would be the effects of flatting a Makefile system which currently has many makefiles (around 800) and over 4.5 million source lines of code? People currently enjoy being able to build just their current subdirectory or process (embedded linux target) by using make in that directory.
I've just learned the build was, until recently, twice as long (wince), at which point the release engineer deployed ccache.
发布评论
评论（5）
加快构建速度的注意事项：
构建往往受 I/O 限制，因此将 I/O 分布在多个驱动器/控制器或机器上。 例如，将源放在一个物理驱动器上，将目标（构建输出）放在另一个物理驱动器上，并将这两个驱动器与包含构建工具（.NET、Java、Ant 等）的物理驱动器分开。 ）以及包含操作系统的物理驱动器。
构建通常可以异步完成，因此建立并使用单独的构建机器（持续集成服务器）。 特别是使用它来计算指标、生成文档、生成候选版本以及其他需要花费太长时间或在开发人员工作站上不需要的任何事情。
构建工具通常涉及大量的进程启动/关闭开销，因此请选择能够最小化该开销的工具、脚本和过程。 对于像 make 和 Ant 这样倾向于将其他工具作为子进程调用的工具尤其如此。 例如，我正在转向基于 Python 的构建系统，这样我就可以从单个进程中完成大部分构建处理，但在必要时仍然能够轻松生成其他进程。
构建工具通常支持跳过构建步骤，因为检测到它们不会执行任何操作（不编译，因为自上次编译以来源代码尚未更改）。 但是，默认情况下，对跳过构建步骤的支持通常不处于活动状态 - 您可能需要专门调用它。 例如，编译器通常会自动执行此操作，但代码生成则不会。 当然，这样做的必然结果是尽可能使用增量构建（在工作站上开发时，只要它“表现良好”）。
构建脚本可能会快速而轻松地变得非常复杂，因此请花些时间使它们变得简单。 首先，将构建分成单独的项目，每个项目仅构建一个“工件”，例如 JAR、DLL 或 EXE。 这使您只需不调用当前不需要的长构建即可节省大量时间。 其次，简化每个项目的构建，永远不要涉及另一个项目——始终通过构建工件而不是源来建立项目依赖关系。 这将使每个项目都是独立的，并且您可以使用“超级”脚本随意构建项目的各种子集。
最后，将您的构建基础设施视为自己的真实项目 - 记录针对它的错误和功能请求，对其进行版本控制，执行官方发布，并继续无限期地完善它。 将其视为必不可少的产品，而不是事后的想法：您的构建是任何健康项目的命脉，如果您关心它，它可以拯救您的面包。
Considerations for speeding up your build:
Builds tend to be I/O bound, so distribute the I/O across multiple drives/controllers or machines. For example, put the source on one physical drive and put the target (the build output) on a different physical drive, and separate both of those drives from the physical drive that contains your build tools (.NET, Java, Ant, etc.) and from the physical drive that contains your OS.
Builds often can be done asynchronously, so establish and use separate build machines (continuous integration server). Use this particularly for calculating metrics, generating docs, producing release candidates, and anything else that takes too long or is not needed on a developer's workstation.
Build tools often involve lots of process startup/shutdown overhead, so choose tools, scripts, and procedures that minimize that overhead. This is particularly true of tools like make and Ant that tend to invoke other tools as subprocesses. For example, I am moving to a Python-based build system so that I can do most of my build processing from a single process, yet still be able to easily spawn other processes when I must.
Build tools often support skipping build steps based on detecting that they will do nothing (don't compile because the source has not been changed since the last compile). However, that support for skipping build steps is often not active by default--you may need to specifically invoke it. For example, compilers usually do this automatically, but code generation does not. Of course, a corollary to this is to use incremental builds when you can (while developing on your workstation, as long as it "behaves").
Build scripts can quickly and easily get very complex, so take the time to make them simple. First, separate the builds into separate projects that each build only one "artifact", such as a JAR, DLL, or EXE. This allows you to save lots of time by merely not invoking a long build that you don't need at the moment. Second, simplify each and every project build by NEVER reaching into another project--always make your project dependencies via the build artifact rather than the source. This will make each project stand-alone, and you can use "super" scripts to build various subsets of the projects at will.
Finally, treat your build infrastructure as a real project of its own--log bugs and feature requests against it, version it, perform official releases, and continue to refine it indefinitely. Treat it like an essential product rather than as an afterthought: your build is your lifeblood for any healthy project, it can save your buns if you care for it.
我不确定为什么在你的情况下需要符号链接。 如果您从同一源构建多个目标，您可以尝试将中间文件和目标文件放在不同的目录中。
另外，您可以尝试使用类似 makefile ://www.perforce.com/jam/jam.html" rel="nofollow noreferrer">果酱。 如果您有多个 CPU，可以尝试
make -jn，其中 n 是 CPU 数量 + 1。I'm not sure why symlinks are needed in your case. If you are building multiple targets from the same source, you might try putting your intermediate and target files in separate directories.
Also, instead of nested makefiles, you could try to use something like jam. If you have a multiple CPUs, you can try
make -jn, where n is the number of CPUs + 1.如果您有不止一台计算机可供编译，您还可以使用 distcc。 这会将构建过程分散到多台计算机上，与 make -j 2 结合使用可以进一步加快速度。
根据项目的大小，瓶颈实际上可能与编译器有关（即您启用 -O2 或 -O3） ，除了缩小源代码或删除未使用的 #include 文件之外，可能没有什么可以做的。
If you have more than one computer available for compilation, you can also use distcc. This spreads the build process over multiple computers which should speed things up further in combination with make -j 2.
Depending on the size of the project, the bottleneck may actually be with the compiler (i.e. you're enabling -O2 or -O3), and there might not be much that can be done aside from shrinking the source code or removing unused #include files.
提及嵌套 makefile 表明“递归”中建议的方法认为有害” 可能会有所帮助。 从摘要来看：
（单个逻辑 makefile 仍然可以由多个物理文件组成。）
The mention of nested makefiles suggests that the approach suggested in "Recursive Make Considered Harmful" could help. From the abstract:
(The single logical makefile can still be made up of multiple physical files.)
我们用冰淇淋。 不是为了在构建时消磨时间，而是为了加快构建过程。 它是一个分布式编译环境，利用办公室所有 PC 的空闲 CPU 时间。 我们的设置非常复杂，因为我们有一个交叉编译环境，但如果您不进行交叉编译，它可能会简单得多。
http://en.opensuse.org/Icecream
We use icecream. Not to kill time while it builds, but to speed up the build process. It's a distributed compiling environnment that uses the spare CPU time of all the PCs in the office. Our setup is quite complex as we have a cross-compiling environment, but it could be much simpler if you aren't cross-compiling.
http://en.opensuse.org/Icecream