我正在尝试并行运行一个函数，但它被卡住并无限运行，直到我强行中断该进程

Question

当我使用 for 循环顺序运行此函数时，它工作得很好，但是与多处理并行执行它会使其连续运行而不会停止。我已经尝试过 Pool 和 Process 方法，但都给出了相同的问题，找不到我混淆的地方。

import numpy as np
import pandas as pd
import time
import multiprocessing as mp
import io

from google.colab import drive
drive.mount('/content/drive')
from google.colab import files
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

up = files.upload()
data = pd.read_csv(io.BytesIO(up['heart.csv']))

#### SEQUENTIAL PROGRAM - This part runs smoothly
            
s1 = time.time()
            
def kmeans(n):
  sc = StandardScaler()
  X = sc.fit_transform(pd.get_dummies(data, drop_first = True))
  ss = []
  km = KMeans(n_clusters=n, random_state=1)
  km = km.fit(X)
  ss.append(km.inertia_)
  print(f"Process {n} is executing")

  return ss
            
for i in range(1, 50):
  kmeans(i)
            
s2 = time.time()
            
print(f"took {s2-s1}s to finish")
        
### Using Multiprocessing- Process() - Runs continuously without stopping
        
tic2 = time.time()
        
process_list = []
for i in range(1, 50):
  p =  mp.Process(target= kmeans, args = [i])
  p.start()
  process_list.append(p)
        
for process in process_list:
  process.join()
        
 toc2 = time.time()
        
 print('Done in {:.4f} seconds'.format(toc2-tic2))
    
    ### Using Multiprocessing- Pool() - Runs continuously without stopping
    
 tic1 = time.time()
    
 pool = mp.Pool()
 pool.map(kmeans, range(1,50))
 pool.close()
    
 toc1 = time.time()
    
 print('Done in {:.4f} seconds'.format(toc1-tic1))

Answer 1

除了无意义的数据之外，代码上的这个微小变化可以完美运行（如果您忽略合成数据引起的异常），

也许这将帮助您了解哪里出了问题。

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
import pandas as pd
from concurrent.futures import ProcessPoolExecutor
import numpy as np
import warnings

def kmeans(n):
    data = ['John', 'Paul', 'George', 'Ringo']
    ss = []
    print(f"Process {n} is executing")
    try:
        sc = StandardScaler()
        X = sc.fit_transform(pd.get_dummies(data, drop_first=True))
        km = KMeans(n_clusters=n, random_state=1)
        km = km.fit(X)
        ss.append(km.inertia_)
    except Exception:
        pass
    print(f"Process {n} is ending")
    return ss


def main():
    warnings.filterwarnings('ignore')
    with ProcessPoolExecutor() as executor:
        executor.map(kmeans, list(range(50)))


if __name__ == '__main__':
    main()