CPU跳跃如何影响Python的计算速度？

发布于 2025-02-08 01:13:10 字数 841 浏览 1 评论 0原文

系统：Debian 10在Google Cloud Platform中以DLVM，N1-HighMem-16机器类型运行。

我在Python中进行了一些大量计算，并且我正在使用多处理模块使用更多的CPU。我注意到，在观看HTOP中的计算旋转时，闲置处理器的频率很高。也就是说，有时CPU 0会闲置，然后会变得忙碌。稍后的某个时候，它可能会再次闲置。这使我相信积极的工作可能也在跳来跳去。当然，恒定的CPU跳正在减慢计算？有没有办法防止恒定的CPU跳跃？还是一个坏主意？

这是您可以运行的Python代码的MWE，可以练习多处理足够长的时间以检测我在谈论的跳跃类型：

import multiprocessing
import numpy as np
import time


def function_calculation():

    data = np.zeros(1000000)
    t1 = time.perf_counter()
    direction = 1
    while time.perf_counter() - t1 < 600:  # 10 minutes.
        data = data + direction
        direction = direction * (-1)


num_processors = multiprocessing.cpu_count() - 1  # leave one out for system stuff.
for proc in range(num_processors):
    p = multiprocessing.Process(target=function_calculation)
    p.start()

原文

System: Debian 10 running in the Google Cloud Platform as a DLVM, n1-highmem-16 machine type.

I'm doing some heavy calculations in Python, and I'm using the multiprocessing module to use a lot more of my CPUs. I've noticed, while watching my calculations spin in htop, that the idle processors jump around fairly frequently. That is, sometimes CPU 0 will be idle, and then it will become busy. Sometime later it might become idle again. This leads me to believe that the active jobs are probably jumping around as well. Surely the constant CPU hopping is slowing down the calculations? Is there a way to prevent the constant CPU hopping? Or is that a bad idea?

Here is a MWE of Python code you can run that will exercise multiprocessing long enough to detect the kind of hopping I'm talking about:

import multiprocessing
import numpy as np
import time


def function_calculation():

    data = np.zeros(1000000)
    t1 = time.perf_counter()
    direction = 1
    while time.perf_counter() - t1 < 600:  # 10 minutes.
        data = data + direction
        direction = direction * (-1)


num_processors = multiprocessing.cpu_count() - 1  # leave one out for system stuff.
for proc in range(num_processors):
    p = multiprocessing.Process(target=function_calculation)
    p.start()

分享到QQ

分享到微博