scipy.optimize.curve_fit的错误处理,从' Multiprocess'模块

发布于 2025-02-09 05:25:31 字数 4125 浏览 2 评论 0原文

我的目标是在图像中的一堆列上平行一堆1D高斯拟合。我已经使用了模块多进程而不是多处理,因为我无法找到一种通过数组传递的方法,因为如果没有它,函数的参数就会腌制。现在工作正常。

但是,一旦我在腌制功能内部使用curve_fit(在这种情况下为傻...),我就会有问题。如果我评论curve_fit部分并简单地返回初始猜测参数,则一切正常。但是我无法像在循环中处理数据时那样处理curve_fit runtimeerror,因此:

                    try:
                        params, _ = curve_fit(gauss1d,x,a,p0=guess)
                        params[3]=np.abs(params[3])+0.5
                    except RuntimeError:
                        params = [1,1,1,1]

相反,如果拟合不收敛,请引发此错误:

文件“ Blablabla \ lib \ lib \ lib \ site-packages \ supterprocess \ multiprocess \ \ pool..py”,第771行, 提高自我

valueerror:对物体太深而无法获得所需的数组,

这是由于这个小家伙造成的: 文件“ Blablabla \ lib \ lib \ site-packages \ scipy \ equipize \ minpack.py,第423行,最少 retval = _minpack._lmdif(func,x0,args,full_output,ftol,ftol,xtol, MinPack.Error:功能调用结果不是正确的浮子数组。

我相信我正在达到最大功能并获得ValueError,但是由于某种原因,我的除外不再在泳池内处理此功能。我可能会追逐红鲱鱼。

下面的完整代码。添加其他例外(例如ValueError)没有任何更改。所有输入都赞赏。

 def processframe(framey):
        import numpy as np
        import scipy.ndimage as nd
        from scipy.optimize import curve_fit
        sizey = framey.shape
        paramsy = np.zeros([sizey[1],4])
        columnsy = np.split(framey,sizey[1],axis=1)
        
        if __name__ == '__main__':
            import numpy as np
            def thefit(k):
                def gfit1d(a): #simple 1d gaussian fit.  Fast.  Effective.  Uses 1d gauss stats to set initial parameters
                    def gguess1d(a): #This assumes a single gaussian 1d distribution on approximately the scale of a bright image (0 to 255), and returns stats for a 1d distribution, using physics rules (width is 1/e^2 radius)
                        x=np.linspace(1,np.amax(a.shape),np.amax(a.shape),endpoint=True)
                        fata = nd.gaussian_filter1d(a,2)
                        GuessY0 = np.amin(a)
                        GuessA = np.amax(a)-GuessY0
                        X0A = np.amax(fata)
                        X0Y0 = np.amin(fata)
                        X0form = np.multiply(fata+X0Y0,(fata+X0Y0>(X0A+X0Y0)/3))
                        GuessX0 = np.sum(np.multiply(X0form,x))/(1+np.sum(X0form))
                        GuessSigma = np.sqrt(np.sum(np.square(np.multiply(x-GuessX0,np.multiply(a-GuessY0,a-GuessY0>2))))/(1+np.sum(np.multiply(a-GuessY0,a-GuessY0>2))))/4
                        solution=np.asarray([GuessY0, GuessA, GuessX0, GuessSigma])
                        return np.nan_to_num(np.real(solution.flatten()))

                    therange = np.amax(a)-np.amin(a)
                    size = np.amax(a.shape)
                    x=np.linspace(1,size,size,endpoint=True)
                    guess =np.nan_to_num(np.real( gguess1d(a))).flatten()
                   
                    def gauss1d(x,aa,bb,cc,dd):
                        return aa+bb*np.exp(-2*(x-cc)**2/(np.abs(dd)+0.5)**2)+2000*(cc<1)*(1-cc)**2+2000*(cc>size)*(cc-size)**2+2000*(np.abs(aa-bb)>3*therange)*(np.abs(aa-bb)-3*(therange)**2)
                    try:
                        params, _ = curve_fit(gauss1d,x,a,p0=guess)
                        params[3]=np.abs(params[3])+0.5
                    except RuntimeError:
                        params = [1,1,1,1]
                    return np.nan_to_num(np.real(np.asarray(params.flatten())))
                    """
                    return guess
                    """
                    
                global paramsyk
                class paramsyk: pass
                try:
                    paramsyk=gfit1d(k)
                except RuntimeError:
                    #print(curse())
                    paramsyk=np.array([1,1,1,1])
                return np.asarray(paramsyk.flatten())
           # multiprocessing.set_start_method('spawn')
           # q=multiprocess.Queue(columnsy)
            pool=multiprocess.Pool(2)
            paramsy = pool.map(lambda x: np.asarray(thefit(columnsy[x])).flatten(), range(sizey[1]))
            pool.close
            pool.join
        
        return np.transpose(np.stack(paramsy,axis=1))

My goal is to parallelize a bunch of 1d gaussian fits on a bunch of columns in an image. I've used the module multiprocess instead of multiprocessing because I couldn't figure out a way to pass arrays as the arguments of a function being pickled without it. This now works fine.

However, as soon as I use curve_fit inside of the function being pickled (or, in this case, dilled...), I have problems. If I comment out the curve_fit section and simply return the initial guess parameters, this all works fine. But I'm unable to handle curve_fit RuntimeError as I might when processing data in a for loop, as such:

                    try:
                        params, _ = curve_fit(gauss1d,x,a,p0=guess)
                        params[3]=np.abs(params[3])+0.5
                    except RuntimeError:
                        params = [1,1,1,1]

Instead, if the fit doesn't converge, pool throws this error:

File "blablabla\lib\site-packages\multiprocess\pool.py", line 771, in get
raise self._value

error: Result from function call is not a proper array of floats.

ValueError: object too deep for desired array

Which is due to this little fella:
File "blablabla\lib\site-packages\scipy\optimize\minpack.py", line 423, in leastsq
retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
minpack.error: Result from function call is not a proper array of floats.

I believe I'm hitting the max function evals and getting ValueError, but my except is no longer handling this inside the pool for whatever reason. It's also possible I'm chasing a red herring.

Full code below. Adding additional exceptions (such as ValueError) didn't change anything. All input appreciated.

 def processframe(framey):
        import numpy as np
        import scipy.ndimage as nd
        from scipy.optimize import curve_fit
        sizey = framey.shape
        paramsy = np.zeros([sizey[1],4])
        columnsy = np.split(framey,sizey[1],axis=1)
        
        if __name__ == '__main__':
            import numpy as np
            def thefit(k):
                def gfit1d(a): #simple 1d gaussian fit.  Fast.  Effective.  Uses 1d gauss stats to set initial parameters
                    def gguess1d(a): #This assumes a single gaussian 1d distribution on approximately the scale of a bright image (0 to 255), and returns stats for a 1d distribution, using physics rules (width is 1/e^2 radius)
                        x=np.linspace(1,np.amax(a.shape),np.amax(a.shape),endpoint=True)
                        fata = nd.gaussian_filter1d(a,2)
                        GuessY0 = np.amin(a)
                        GuessA = np.amax(a)-GuessY0
                        X0A = np.amax(fata)
                        X0Y0 = np.amin(fata)
                        X0form = np.multiply(fata+X0Y0,(fata+X0Y0>(X0A+X0Y0)/3))
                        GuessX0 = np.sum(np.multiply(X0form,x))/(1+np.sum(X0form))
                        GuessSigma = np.sqrt(np.sum(np.square(np.multiply(x-GuessX0,np.multiply(a-GuessY0,a-GuessY0>2))))/(1+np.sum(np.multiply(a-GuessY0,a-GuessY0>2))))/4
                        solution=np.asarray([GuessY0, GuessA, GuessX0, GuessSigma])
                        return np.nan_to_num(np.real(solution.flatten()))

                    therange = np.amax(a)-np.amin(a)
                    size = np.amax(a.shape)
                    x=np.linspace(1,size,size,endpoint=True)
                    guess =np.nan_to_num(np.real( gguess1d(a))).flatten()
                   
                    def gauss1d(x,aa,bb,cc,dd):
                        return aa+bb*np.exp(-2*(x-cc)**2/(np.abs(dd)+0.5)**2)+2000*(cc<1)*(1-cc)**2+2000*(cc>size)*(cc-size)**2+2000*(np.abs(aa-bb)>3*therange)*(np.abs(aa-bb)-3*(therange)**2)
                    try:
                        params, _ = curve_fit(gauss1d,x,a,p0=guess)
                        params[3]=np.abs(params[3])+0.5
                    except RuntimeError:
                        params = [1,1,1,1]
                    return np.nan_to_num(np.real(np.asarray(params.flatten())))
                    """
                    return guess
                    """
                    
                global paramsyk
                class paramsyk: pass
                try:
                    paramsyk=gfit1d(k)
                except RuntimeError:
                    #print(curse())
                    paramsyk=np.array([1,1,1,1])
                return np.asarray(paramsyk.flatten())
           # multiprocessing.set_start_method('spawn')
           # q=multiprocess.Queue(columnsy)
            pool=multiprocess.Pool(2)
            paramsy = pool.map(lambda x: np.asarray(thefit(columnsy[x])).flatten(), range(sizey[1]))
            pool.close
            pool.join
        
        return np.transpose(np.stack(paramsy,axis=1))

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

云之铃。 2025-02-16 05:25:31
  1. 每当编译器能够编译
paramsy = pool.map(lambda x: np.asarray(thefit(columnsy[x])).flatten(), range(sizey[1]))

成功编译中的任何内容本身时,本身就是一个错误。 lambda中的所有事物都需要在lambda中定义,

  1. 将复杂的对象传递给诸如thefit()之类的函数,可以通过经理通过列表上的池运行池来实现。
def processframefitmulti(framey):
        t=time.time()
        import numpy as np
        import scipy.ndimage as nd
        from scipy.optimize import curve_fit
        sizey = framey.shape
            if __name__ == '__main__':
                def thefit(k):
                    import warnings
                    warnings.filterwarnings("ignore")
                    import numpy as np
                    import scipy.ndimage as nd
                    kk=np.array(k).flatten()    #This step is critical to this implementation, as it turns the object from a list back into an np array
                    #do some stuff

            columnsy = np.split(framey,sizey[1],axis=1)
            pool=multiprocess.Pool(3)
            mange=multiprocess.Manager()
            dataset=mange.list(columnsy)
            paramsy=pool.map(thefit,dataset)
            pool.close
            pool.join
     return np.stack(paramsy)
  1. 需要在多处理池中处理异常,因为
try:
    params, _ = curve_fit(gauss1d,x,a,p0=guess)
    params[3]=np.abs(params[3])+0.5
except Exception as e:
    params = [1,1,1,1]

E:E:在多线程应用程序中至关重要,以防止整个事物磨碎到停顿。

  1. Any time the compiler was able to compile anything in
paramsy = pool.map(lambda x: np.asarray(thefit(columnsy[x])).flatten(), range(sizey[1]))

The successful compile was itself a bug. All things in lambda need to be defined within lambda,

  1. Passing complicated objects to a function such as thefit() can be accomplished with a manager, by running the pool over a list.
def processframefitmulti(framey):
        t=time.time()
        import numpy as np
        import scipy.ndimage as nd
        from scipy.optimize import curve_fit
        sizey = framey.shape
            if __name__ == '__main__':
                def thefit(k):
                    import warnings
                    warnings.filterwarnings("ignore")
                    import numpy as np
                    import scipy.ndimage as nd
                    kk=np.array(k).flatten()    #This step is critical to this implementation, as it turns the object from a list back into an np array
                    #do some stuff

            columnsy = np.split(framey,sizey[1],axis=1)
            pool=multiprocess.Pool(3)
            mange=multiprocess.Manager()
            dataset=mange.list(columnsy)
            paramsy=pool.map(thefit,dataset)
            pool.close
            pool.join
     return np.stack(paramsy)
  1. Exceptions need to be handled within multiprocessing pools as
try:
    params, _ = curve_fit(gauss1d,x,a,p0=guess)
    params[3]=np.abs(params[3])+0.5
except Exception as e:
    params = [1,1,1,1]

except Exception as e: is critical within multithreaded applications to prevent the whole thing from grinding to a halt.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文