Numba 函数参数类型没有匹配的定义 ListType[array(float64, 2d, C)] 错误

发布于 2025-01-10 08:55:34 字数 1295 浏览 0 评论 0原文

我在 Numba 中有一个使用 List(float64[:,::1]) 类型的函数,这是一个尝试该类型的虚拟函数,我会这样做for循环下有很多操作。它有一种奇怪的行为,而 to arr 列表具有相同的 numba.typeof() 签名,一个有效,另一个无效,并告诉我它不匹配类型。

这绝对是一种对象错误,但我可以弄清楚。

这是要加载的文件 test.npy:

https://drive.google.com/file/d/1guAe1C2sKZyy2U2_qXAhMA1v46PfeKnN/view?usp=sharing

错误

    raise TypeError(msg)
TypeError: No matching definition for argument type(s) ListType[array(float64, 2d, C)]

代码

import numpy as np
import numba
from numba.typed import List

branches = np.load('test.npy', allow_pickle=True).item()


@numba.njit('List(int64)(List(float64[:, ::1]))')
def not_test(branches):
    a_list = []
    for branch in branches:
        for i in range(len(branch)):
            a_list.append(i)
    return a_list

# this does not work
arr = []
for branch in branches.features[:2]:
    arr.append(np.asarray(branch.geometry.coordinates).copy())
arr = List(arr.copy())

print(numba.typeof(arr))
no_test(arr)

# this works
arr = List([np.array([np.array([7.0,7.3]), np.array([7.4,8.6])])])
print(numba.typeof(arr))
no_test(arr)

I have a function in Numba that uses a List(float64[:,::1]) type, this is a dummy function to try out the type, I will do a lot of operations under the for loop. It has an strange behavior, while the to arr lists have the same excatly numba.typeof() signature, one works and the other does not and tells me it is not matching type.

This is definitely a type of object error, but I can figure it out.

this is the file test.npy to load:

https://drive.google.com/file/d/1guAe1C2sKZyy2U2_qXAhMA1v46PfeKnN/view?usp=sharing

Error

    raise TypeError(msg)
TypeError: No matching definition for argument type(s) ListType[array(float64, 2d, C)]

Code

import numpy as np
import numba
from numba.typed import List

branches = np.load('test.npy', allow_pickle=True).item()


@numba.njit('List(int64)(List(float64[:, ::1]))')
def not_test(branches):
    a_list = []
    for branch in branches:
        for i in range(len(branch)):
            a_list.append(i)
    return a_list

# this does not work
arr = []
for branch in branches.features[:2]:
    arr.append(np.asarray(branch.geometry.coordinates).copy())
arr = List(arr.copy())

print(numba.typeof(arr))
no_test(arr)

# this works
arr = List([np.array([np.array([7.0,7.3]), np.array([7.4,8.6])])])
print(numba.typeof(arr))
no_test(arr)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

旧梦荧光笔 2025-01-17 08:55:34

对于陷入这样的琐事的任何人来说,事实证明正确的签名类型是:

@numba.njit('List(int64)(ListType(float64[:, ::1]))')

我不明白 ListListType 之间的区别,并且我无法在 List 上找到它。代码>Numba官方网站。我可以为装饰器中的类型提供一份备忘单,因为仅从函数参数中找到的可用数据类型来推断它们应该如何编写并不容易。另外,拥有一个基于 numba.typeof() 返回的解析器函数确实会有很大帮助,并且能够创建装饰器的字符串。

另外,到 List() 的转换非常慢,我在 Numba GitHub 上找到了一篇讨论这个问题的帖子。这是原始帖子 以 Python 列表作为参数提高 numba.typed.List 构造函数的性能

def convert2(x, dtype=np.float64):
    try:
        # Try and convert x to a Numpy array. If this succeeds
        # then we have reached the end of the nesting-depth.
        y = np.asarray(x, dtype=dtype)
    except:
        # If the conversion to a Numpy array fails, then it can
        # be because not all elements of x can be converted to
        # the given dtype. There is currently no way to distinguish
        # if this is because x is a nested list, or just a list
        # of simple elements with incompatible types.

        # Recursively call this function on all elements of x.
        y = [convert2(x_, dtype=dtype) for x_ in x]

        # Convert Python list to Numba list.
        y = numba.typed.List(y)

    return y

编辑

我发现了一个超级有用的行来获取 numba 函数的类型签名

print(not_test.inspect_types())
#not_test (ListType[array(float64, 2d, C)], array(float64, 1d, A))

For anybody stuck in something trivial like this, turns out the correct signature type is:

@numba.njit('List(int64)(ListType(float64[:, ::1]))')

I do not understand the differences between List and ListType and I could not find it on the Numba official website. I could be really helpfull to have a cheatsheet for types in decorators as is not easy to infer how they should be written just from the available data type found in the arguments of the function. Also, it could really be of great help to have a parser function based on numba.typeof() return, and been able to create the string of the decorator just of off this.

Plus, the conversion to List() is pretty slow, I found a post on Numba GitHub that talks about this this problem. This is the original post improve performance of numba.typed.List constructor with Python list as arg

def convert2(x, dtype=np.float64):
    try:
        # Try and convert x to a Numpy array. If this succeeds
        # then we have reached the end of the nesting-depth.
        y = np.asarray(x, dtype=dtype)
    except:
        # If the conversion to a Numpy array fails, then it can
        # be because not all elements of x can be converted to
        # the given dtype. There is currently no way to distinguish
        # if this is because x is a nested list, or just a list
        # of simple elements with incompatible types.

        # Recursively call this function on all elements of x.
        y = [convert2(x_, dtype=dtype) for x_ in x]

        # Convert Python list to Numba list.
        y = numba.typed.List(y)

    return y

EDIT

I found a super helpfull line to get the type signatures off of a numba function

print(not_test.inspect_types())
#not_test (ListType[array(float64, 2d, C)], array(float64, 1d, A))
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文