PyTuple_SetItem 的限制
我有一个 Python 扩展模块,它创建一个元组作为另一个对象的属性,并在元组中设置项目。每当我在 Python 中执行此模块时,我都会收到错误 SystemError: bad argument to inside function
在阅读了 PyTuple
的文档并调试了我的程序几个小时后,我还是没明白到底是怎么回事。通过调试器运行我的程序表明问题发生在 Python 解释器内的库调用中。于是,最后我查看了Python源码,终于意识到了问题所在。 PyTuple_SetItem 函数有一个我不知道的有趣限制,也找不到明确的记录。
这是 Python 源代码中的重要函数(为了清晰起见进行了编辑):
int PyTuple_SetItem(register PyObject *op, register Py_ssize_t i, PyObject *newitem)
{
.....
if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
Py_XDECREF(newitem);
PyErr_BadInternalCall();
return -1;
}
.....
}
这里重要的一行是条件 op->ob_refcnt != 1。所以问题是:你甚至不能调用 PyTuple_SetItem
除非 Tuple 的引用计数为 1。看起来这里的想法是你永远不应该使用 PyTuple_SetItem< /code> 除非在使用
PyTuple_New()
创建元组之后。我想这是有道理的,因为毕竟元组应该是不可变的,所以这个限制有助于让你的 C 代码更符合 Python 类型系统的抽象。
但是,我在任何地方都找不到此限制的记录。相关文档似乎在此处和此处,两者都没有指定此限制。文档基本上说,当您调用 PyTuple_New(X)
时,元组中的所有项目都将初始化为 NULL
。由于 NULL 不是有效的 Python 值,因此扩展模块程序员需要确保在将元组返回到解释器之前,元组中的所有槽都已填充正确的 Python 值。但它并没有在任何地方说,当 Tuple 对象的引用计数为 1 时,必须执行此操作。
所以现在,问题是我基本上已经将自己编码到了一个角落,因为我没有意识到这一点(未记录? ) 对 PyTuple_SetItem
的限制。我的代码的结构方式是,在元组本身成为另一个对象的属性之前,将项目插入到元组中非常不方便。因此,当需要填充元组中的项目时,元组已经具有更高的引用计数。
我可能必须重组我的代码,但我正在认真考虑暂时将 Tuple 上的引用计数设置为 1,插入项目,然后恢复原始引用计数。当然,我知道这是一个可怕的黑客行为,而不是任何永久的解决方案。无论如何,我想知道有关元组引用计数的要求是否在任何地方都有记录。它只是 CPython 的实现细节,还是 API 用户可以依赖的预期行为?
I have a Python extension module which creates a tuple as an attribute of another object, and sets items in the tuple. Whenever I execute this module in Python, I keep getting the error SystemError: bad argument to internal function
After reading over the docs for PyTuple
, and debugging my program for a few hours, I still couldn't figure out what the hell was going on. Running my program through a debugger indicated the problem was occurring within a library call inside the Python interpreter. So, finally, I looked at the Python source code, and at long last I realized the problem. The PyTuple_SetItem
function has an interesting restriction which I didn't know about, and can't find explicitly documented.
Here is the important function in the Python source (edited for clarity):
int PyTuple_SetItem(register PyObject *op, register Py_ssize_t i, PyObject *newitem)
{
.....
if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
Py_XDECREF(newitem);
PyErr_BadInternalCall();
return -1;
}
.....
}
The important line here is the condition op->ob_refcnt != 1. So here's the problem: you can't even call PyTuple_SetItem
unless the Tuple has a ref-count of 1. It looks like the idea here is that you're never supposed to use PyTuple_SetItem
except right after you create a tuple using PyTuple_New()
. I guess this makes sense, since Tuples are supposed to be immutable, after all, so this restriction helps keep your C code more in line with the abstractions of the Python type system.
However, I can't find this restriction documented anywhere. Relevant docs seem to be here and here, neither of which specify this restriction. The docs basically say that when you call PyTuple_New(X)
, all items in the tuple are initialized to NULL
. Since NULL
is not a valid Python value, it's up to the extension-module programmer to make sure that all slots in the Tuple are filled in with proper Python values before returning the Tuple to the interpreter. But it doesn't say anywhere that this must be done while the Tuple object has a ref count of 1.
So now, the problem is that I've basically coded myself into a corner because I wasn't aware of this (undocumented?) restriction on PyTuple_SetItem
. My code is structured in such a way that it's very inconvenient to insert items into the tuple until after the tuple itself has become an attribute of another object. So, when it comes time to fill in the items in the tuple, the tuple already has a higher ref count.
I'll probably have to restructure my code, but I was seriously considering just temporarily setting the ref count on the Tuple to 1, inserting the items, and then restoring the original ref count. Of course, that's a horrible hack, I know, and not any kind of permanent solution. Regardless, I'd like to know if the requirement regarding the ref count on the Tuple is documented anywhere. Is it just an implementation detail of CPython, or is it something that API users can rely on as expected behavior?
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
我确信您可以使用
PyTuple_SET_ITEM
而不是PyTuple_SetItem
来绕过这些限制。PyTuple_SET_ITEM
是在tupleobject.h
中定义的宏,如下所示:因此,如果您绝对、绝对、完全确定:
op
是一个元组 对象i
v
的引用,并且您想让元组窃取它,并且PyTuple_SET_ITEM
之前执行任何操作,我想您可以安全地使用
PyTuple_SET_ITEM
。I'm quite sure that you can get around the restrictions by using
PyTuple_SET_ITEM
instead ofPyTuple_SetItem
.PyTuple_SET_ITEM
is a macro defined intupleobject.h
as follows:So, if you are absolutely, definitely and utterly sure that:
op
is a tuple objecti
in the tuple so farv
and you want to let the tuple steal it andPyTuple_SET_ITEM
then I guess you are safe to use
PyTuple_SET_ITEM
.Python C API 非常的文档记录不足,如果任何地方没有提到此限制,我不会感到惊讶。
当然,一旦元组被某些东西控制,无论如何你都不应该修改它们;要么传入需要放入元组中的元素,要么使用列表。
The Python C API is very underdocumented and I would not be surprised if this restriction wasn't mentioned anywhere.
Of course, you should never be modifying tuples once something has gotten a hold of them regardless; either pass in the elements you need to put in the tuple, or use a list instead.