NUMA 机器上并行 MATLAB 的性能问题
我在 Windows 7 下的 64 核 NUMA 机器上的 MATLAB 中运行内存密集型并行计算，每个插槽 8 个核心。我正在使用并行计算工具箱来做到这一点。我注意到一个非常奇怪的 cpu 负载模式：然后运行 36 个并行 MATLAB，第一个插槽上的内核已满载，第二个插槽也几乎满载，第三个插槽约为 50% 等等。最后一个套接字通常几乎完全空闲并且不执行任何操作。同时运行超过 12 个并行工作线程似乎会对所有工作线程的性能产生非常不利的影响。
我尝试尝试 CPU 关联性，将不同的工作线程固定到不同的内核。虽然它有助于简单的测试（即 CPU 负载模式在所有核心上变得统一），但它对我们现实生活中的内存密集型计算没有帮助。
我怀疑问题出在内存局部性上。即所有内存都分配在第一个和第二个套接字上。这可以解释奇怪的CPU负载：操作系统厌倦了运行更接近数据的计算线程。但我既不知道如何直接确认这个怀疑，也不知道如何解决它（如果它是真的）。
如果这很重要的话，我在所有并行工作线程中使用 maxNumCompThreads(4) 。超线程已关闭。
I'm running memory-intensive parallel computations in MATLAB on a 64-core NUMA machine under Windows 7, 8 cores per socket. I'm using parallel computing toolbox to do that. I've noticed a very strange cpu load pattern: then running say 36 parallel MATLABs, the cores on the 1st socket are fully loaded, 2nd socket is almost fully loaded too, third socket is about 50% and so on. The last socket is usually almost completely free and doing nothing. Running more than 12 parallel workers simultaneously seem to very adversely affect performance of all workers.
I tried to experiment with cpu affinity, pinning different workers to different cores. While it helps in simple tests (i.e. cpu load pattern becomes uniform across all cores), it doesn't help in our real-life memory-intensive computations.
I suspect the problem is with memory locality. I.e. all memory is allocated on 1st and 2nd sockets. This would explain strange cpu load: OS tires to run computational threads closer to the data. But I don't know neither how to confirm this suspicion directly, nor how to fix it, if it's true.
I use maxNumCompThreads(4) in all my parallel workers, if that's important. Hyperthreading is off.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
您应该只能使用并行计算工具箱运行 12 个本地工作线程。请参阅数据表。
You should only be able to run 12 local workers using Parallel Computing Toolbox. See the data sheet.
请注意，在 R2014a 中，取消了对本地工人数量的限制。请参阅发行说明。
Please note that in R2014a the limit on the number of local workers was removed. See the release notes.