有没有办法将 pythonappend 与 SWIG 的新内置功能一起使用?

发布于 2025-01-05 09:23:07 字数 1754 浏览 0 评论 0原文

我有一个小项目,可以与 SWIG 完美配合。特别是,我的一些函数返回 std::vector,它们在 Python 中被转换为元组。现在,我做了很多数字运算,所以我只是让 SWIG 在从 C++ 代码返回它们后将它们转换为 numpy 数组。为此,我在 SWIG 中使用类似以下内容的内容。

%feature("pythonappend") My::Cool::Namespace::Data() const %{ if isinstance(val, tuple) : val = numpy.array(val) %}

(实际上,有几个名为 Data 的函数,其中一些返回浮点数,这就是为什么我检查 val 实际上是一个元组。)这工作得非常漂亮。

但是,我还想使用现在可用的 -builtin 标志。对这些数据函数的调用很少,而且大多是交互式的,因此它们的缓慢不是问题,但还有其他缓慢的循环可以通过内置选项显着加速。

问题是,当我使用该标志时, pythonappend 功能会被默默地忽略。现在,Data 只是再次返回一个元组。有什么方法我仍然可以返回 numpy 数组吗?我尝试使用类型映射,但结果变得一团糟。

编辑:

Borealid 很好地回答了这个问题。为了完整起见,我包含了一些我需要的相关但略有不同的类型映射,因为我通过 const 引用返回,并且使用向量的向量(不要开始!)。这些差异足够大,我不希望其他人跌跌撞撞地试图找出细微的差异。

%typemap(out) std::vector<int>& {
  npy_intp result_size = $1->size();
  npy_intp dims[1] = { result_size };
  PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
  int* dat = (int*) PyArray_DATA(npy_arr);
  for (size_t i = 0; i < result_size; ++i) { dat[i] = (*$1)[i]; }
  $result = PyArray_Return(npy_arr);
}
%typemap(out) std::vector<std::vector<int> >& {
  npy_intp result_size = $1->size();
  npy_intp result_size2 = (result_size>0 ? (*$1)[0].size() : 0);
  npy_intp dims[2] = { result_size, result_size2 };
  PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_INT);
  int* dat = (int*) PyArray_DATA(npy_arr);
  for (size_t i = 0; i < result_size; ++i) { for (size_t j = 0; j < result_size2; ++j) { dat[i*result_size2+j] = (*$1)[i][j]; } }
  $result = PyArray_Return(npy_arr);
}

编辑2:

虽然不完全是我想要的,但使用@MONK的方法也可以解决类似的问题(此处解释) 。

I have a little project that works beautifully with SWIG. In particular, some of my functions return std::vectors, which get translated to tuples in Python. Now, I do a lot of numerics, so I just have SWIG convert these to numpy arrays after they're returned from the c++ code. To do this, I use something like the following in SWIG.

%feature("pythonappend") My::Cool::Namespace::Data() const %{ if isinstance(val, tuple) : val = numpy.array(val) %}

(Actually, there are several functions named Data, some of which return floats, which is why I check that val is actually a tuple.) This works just beautifully.

But, I'd also like to use the -builtin flag that's now available. Calls to these Data functions are rare and mostly interactive, so their slowness is not a problem, but there are other slow loops that speed up significantly with the builtin option.

The problem is that when I use that flag, the pythonappend feature is silently ignored. Now, Data just returns a tuple again. Is there any way I could still return numpy arrays? I tried using typemaps, but it turned into a giant mess.

Edit:

Borealid has answered the question very nicely. Just for completeness, I include a couple related but subtly different typemaps that I need because I return by const reference and I use vectors of vectors (don't start!). These are different enough that I wouldn't want anyone else stumbling around trying to figure out the minor differences.

%typemap(out) std::vector<int>& {
  npy_intp result_size = $1->size();
  npy_intp dims[1] = { result_size };
  PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
  int* dat = (int*) PyArray_DATA(npy_arr);
  for (size_t i = 0; i < result_size; ++i) { dat[i] = (*$1)[i]; }
  $result = PyArray_Return(npy_arr);
}
%typemap(out) std::vector<std::vector<int> >& {
  npy_intp result_size = $1->size();
  npy_intp result_size2 = (result_size>0 ? (*$1)[0].size() : 0);
  npy_intp dims[2] = { result_size, result_size2 };
  PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_INT);
  int* dat = (int*) PyArray_DATA(npy_arr);
  for (size_t i = 0; i < result_size; ++i) { for (size_t j = 0; j < result_size2; ++j) { dat[i*result_size2+j] = (*$1)[i][j]; } }
  $result = PyArray_Return(npy_arr);
}

Edit 2:

Though not quite what I was looking for, similar problems may also be solved using @MONK's approach (explained here).

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

花心好男孩 2025-01-12 09:23:07

我同意你的观点,使用 typemap 会有点混乱,但这是完成此任务的正确方法。您也是对的,SWIG 文档并没有直接说 %pythonappend-builtin 不兼容,但强烈暗示:%pythonappend 添加到Python代理类,并且Python代理类根本不与-builtin标志一起存在。

之前,您所做的是将 C++ std::vector 对象转换为 Python 元组,然后将这些元组传递回 numpy - 在那里它们再次被转换。

您真正想做的是在 C 级别将它们转换一次。

下面是一些将所有 std::vector 对象转换为 NumPy 整数数组的代码:

%{
#include "numpy/arrayobject.h"
%}

%init %{
    import_array();
%}

%typemap(out) std::vector<int> {
    npy_intp result_size = $1.size();

    npy_intp dims[1] = { result_size };

    PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
    int* dat = (int*) PyArray_DATA(npy_arr);

    for (size_t i = 0; i < result_size; ++i) {
        dat[i] = $1[i];
    }

    $result = PyArray_Return(npy_arr);
}

这使用 C 级 numpy 函数来构造并返回一个数组。按照顺序,它:

  • 确保 NumPy 的 arrayobject.h 文件包含在 C++ 输出文件中
  • 导致在加载 Python 模块时调用 import_array (否则,所有 NumPy 方法都将被调用) segfault)
  • 使用 typemapstd::vector 的任何返回映射到 NumPy 数组中。

应放置此代码%import包含返回std::vector的函数的标头之前。除了该限制之外,它是完全独立的,因此它不应该给您的代码库添加太多主观“混乱”。

如果您需要其他向量类型,则只需更改 NPY_INT 以及所有 int*int 位即可,否则会重复上述函数。

I agree with you that using typemap gets a little messy, but it is the right way to accomplish this task. You are also right that the SWIG documentation does not directly say that %pythonappend is incompatible with -builtin, but it is strongly implied: %pythonappend adds to the Python proxy class, and the Python proxy class does not exist at all in conjunction with the -builtin flag.

Before, what you were doing was having SWIG convert the C++ std::vector objects into Python tuples, and then passing those tuples back down to numpy - where they were converted again.

What you really want to do is convert them once, at the C level.

Here's some code which will turn all std::vector<int> objects into NumPy integer arrays:

%{
#include "numpy/arrayobject.h"
%}

%init %{
    import_array();
%}

%typemap(out) std::vector<int> {
    npy_intp result_size = $1.size();

    npy_intp dims[1] = { result_size };

    PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
    int* dat = (int*) PyArray_DATA(npy_arr);

    for (size_t i = 0; i < result_size; ++i) {
        dat[i] = $1[i];
    }

    $result = PyArray_Return(npy_arr);
}

This uses the C-level numpy functions to construct and return an array. In order, it:

  • Ensures NumPy's arrayobject.h file is included in the C++ output file
  • Causes import_array to be called when the Python module is loaded (otherwise, all NumPy methods will segfault)
  • Maps any returns of std::vector<int> into NumPy arrays with a typemap

This code should be placed before you %import the headers which contain the functions returning std::vector<int>. Other than that restriction, it's entirely self-contained, so it shouldn't add too much subjective "mess" to your codebase.

If you need other vector types, you can just change the NPY_INT and all the int* and int bits, otherwise duplicating the function above.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文