Windows 上的 Make(并行作业)

发布于 2024-08-06 12:01:21 字数 196 浏览 1 评论 0原文

什么设置适用于 Windows 上的 GNU make 并行作业 (-j)?

我尝试使用 MinGW make 3.81 将 shell 设置为 cmd.exe,这可以创建多个进程,但 make 失败并显示“等待作业”消息。

这可以工作吗?最好的设置是什么? (MinGW / Cygwin / ???) 有人能给我指出一个可以测试的工作示例吗?

What setup works for GNU make parallel jobs (-j) on Windows?

I have tried setting the shell to cmd.exe using MinGW make 3.81, this works in creating the multiple processes but make fails with the "waiting for job" message.

Can this work and what is the best setup? (MinGW / Cygwin / ???)
Can someone point me to a working example to test against?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(5

私野 2024-08-13 12:01:21

可能对 CMake 用户有帮助的提示

当 makefile 像 cmake-makefiles 通常那样递归调用 make 时,-jN 选项不起作用。
但是有一个补救措施可以再次起作用,因为在生成的 makefile 中,CMake 通过这样的语法进行调用。

$(MAKE) -f CMakeFiles\Makefile2 <subproject>

这意味着您可以修改变量 MAKE:

mingw32-make "MAKE=mingw32-make -j3"

现在每次启动 subproject-make 时,它​​都会再次获得“-j3”选项。
但请注意,这实际上不会将并行编译的数量限制为 3,正如您所期望的那样。如果您有 3 个以上的项目,并且在同一层次结构上彼此不依赖,则所有 3 个项目将并行构建,并且每个项目都会启动 3 个编译步骤。导致 9 个并行编译步骤。

当我们再次仔细查看由 cmake 生成的顶部 Makefile 时,我们会发现目标“all”本质上只启动一个子 make。

$(MAKE) -f CMakeFiles\Makefile2 all

因此,我们可以通过调用删除一层子项目并行性,

mingw32-make "MAKE=mingw32-make -j3" -f CMakeFiles\Makefile2 all

但这会牺牲进度报告。

尽管如此,我希望这会有所帮助。

A tip that might help CMake users

The -jN option does not work when a makefile calls make recursively as cmake-makefiles typically do.
But there is a remedy that again works because in the generated makefiles CMake calls via such a syntax.

$(MAKE) -f CMakeFiles\Makefile2 <subproject>

This means you can modify the variable MAKE:

mingw32-make "MAKE=mingw32-make -j3"

Now every time a subproject-make is started, it again gets the "-j3" option.
But note that this effectively does not limit the number of parallel compiles to 3 as you might expect. If you have more than 3 projects that don't depend on each other on the same hierarchy then all 3 projects will build parallel and each of them launches 3 compile steps. Resulting in 9 parallel compile steps.

When we take another close look into the top Makefile generated by cmake then we see that the target "all" essentially only starts a sub make.

$(MAKE) -f CMakeFiles\Makefile2 all

So we can remove one layer of subproject parallelism by calling

mingw32-make "MAKE=mingw32-make -j3" -f CMakeFiles\Makefile2 all

But this sacrifices the progress reporting.

I hope this helps nevertheless.

只是一片海 2024-08-13 12:01:21

据我所知,GNU Make 的并行作业依赖于底层平台来提供(CPU)负载信息。不幸的是,这是以与 Windows 不兼容的方式完成的,并且只有“make -j”命令才能执行任何缩放操作。该命令(没有最大作业数)可能非常危险,因为它会像一只疯狂的饥饿狗一样吃掉内存(我在 Qt 上尝试过,它在 QtWebkit 上崩溃了,QtWebkit 是 Qt 中最大的项目,4GB RAM,Win7)。

再想一想,我感觉 make -j3 比 make 运行得更快,但根据我在网络上找到的内容(论坛、手册...),它似乎是 POSIX/linux 特定的功能。

顺便说一句,我很惊讶你的问题被否决了。

As Far as I can understand, GNU Make's parallel jobs depend on the underlying platform to supply (CPU) load information. This is unfortunately done in a non-Windows-compatible way, and only the "make -j" command does anything scalingwise. That command (without max number of jobs) is potentially quite dangerous, as it will eat memory like a rabid starving dog (I tried it on Qt, and it crashed on QtWebkit, the largest project in Qt, 4GB RAM, Win7).

On second thought, I get the feeling make -j3 runs faster than make, but in light of what I can find on the web (forums, manual...) it seems very POSIX/linux specific functionality.

BTW, I'm surprised your question was voted down.

陈甜 2024-08-13 12:01:21

我发现这篇 Danny Thorpe 的博客文章解释了以下问题Windows 上的并行 make 构建。基本上它建议的是首先设置以下环境变量:

set SHELL=cmd.exe

这样,我就能够运行 make -j 样式命令,但不幸的是不是受控的 make -jN样式命令。

I found this Danny Thorpe's blog entry that explains the problems with parallel make build on Windows. Basically what it suggests is to set the following environment variable first:

set SHELL=cmd.exe

With that, I was able to run the make -j style command, but unfortunately not the controlled make -jN style command.

青芜 2024-08-13 12:01:21

我已经解决了这个问题,所以我把解决方案分享给大家。

您可以尝试MinGW-TDM(主页:http://tdm-gcc.tdragon.net/ )或 MinGW-Equation(下载页面:http://www.equation.com/ servlet/equation.cmd?fa=fortran)。

例如,将 32 位版本安装到 D:\MinGW\MinGW32 (如果您希望使用 64 位编译二进制文件,则将 64 位版本安装到 D:\MinGW\MinGW64)

创建一个批处理文件,如下所示,然后运行它在您的构建目录中:

Set MinGW_bin=D:\MinGW\MinGW32\bin
Set MSys_bin=D:\MinGW\msys\bin
Set Path=%MSys_bin%;%MinGW_bin%
mingw32-make SHELL=CMD.exe -j6 -f makefile

您可以从 http://sourceforge.net/ 下载 MSYS apps/trac/mingw-w64/wiki/MSYS。它有许多类似 UNIX 的实用程序,需要构建许多开源库、应用程序。

其中 -j6 是运行 6 个并行编译作业的选项。

提示:您可以设置选项 -jN,其中 N = CPU 核心(线程)数量的 1.5 倍。就我而言,我的 CPU 有 4 个线程,因此我将此值设置为 6。

注意:您可以运行

make.exe -jN -f 生成文件

不带 SHELL=CMD.exe 选项,您的编译是并行运行的,但在某些情况下不兼容(其中 make.exe 来自 MSYS 目录)。

问候,

I've resolved this problem so I share the solution to everyone.

You can try MinGW-TDM (Home page: http://tdm-gcc.tdragon.net/) or MinGW-Equation (Download page: http://www.equation.com/servlet/equation.cmd?fa=fortran).

Install 32 bit version to D:\MinGW\MinGW32 for example (and 64 bit version to D:\MinGW\MinGW64 if you wish to use 64 bits compilation binaries)

Create a batch file as following then run it at your build directory:

Set MinGW_bin=D:\MinGW\MinGW32\bin
Set MSys_bin=D:\MinGW\msys\bin
Set Path=%MSys_bin%;%MinGW_bin%
mingw32-make SHELL=CMD.exe -j6 -f makefile

You can download MSYS from http://sourceforge.net/apps/trac/mingw-w64/wiki/MSYS. It has many UNIX like utilities that required building many open source libraries, applications.

Where -j6 is the option to run 6 parallel compiling jobs.

TIP: you can set the option -jN where N = your numbers of CPU cores (threads) multiple by 1.5. In my case, my CPU has 4 threads so I set this value to 6.

Note: you can run

make.exe -jN -f makefile

without SHELL=CMD.exe option and your compilation is run parallel but it not compatible in some case (where make.exe is come from MSYS directory).

Regards,

じ违心 2024-08-13 12:01:21

我在 Cygwin 下使用 make -jn 从未遇到过任何问题。它运作得相当好。我经常将它与 Microsoft 的 cl.exe 编译器一起使用。它对我来说开箱即用。非常像 UNIX,这是一件好事™。构建可以很好地移植到 Linux 和 Mac 的脚本。彻底推荐。

自从我遇到 5 个反斜杠太少,但 6 个反斜杠又太多的地方以来,我一直不喜欢 MinGW make。叹! (与 MinGW hacks 一起使用 make AFAIK 中允许反斜杠分隔的路径名。)

I've never had any promblems using make -jn under Cygwin. It works rather well. I regularly use it with Microsoft's cl.exe compiler. It just works out of the box for me. Very UNIX like, which is a Good Thing™. Build scripts port nicely to Linux and the Mac. Thoroughly recommended.

Never liked MinGW make since I got to a place where 5 back-slashes were too few, yet six was too many. Sigh! (To do with MinGW hacks to make backslash separated path names allowable in make AFAIK.)

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文