使用 Python C API 将 Python 中的函数移植到 C 时遇到问题
我在 Python 中有一个校验和函数：
def checksum(data):
a = b = 0
l = len(data)
for i in range(l):
a += ord(data[i])
b += (l - i)*ord(data[i])
return (b << 16) | a, a, b
我正在尝试将其移植到 C 模块以提高速度。这是 C 函数：
static PyObject *
checksum(PyObject *self, PyObject *args)
{
int i, length;
unsigned long long a = 0, b = 0;
unsigned long long checksum = 0;
char *data;
if (!PyArg_ParseTuple(args, "s#", &data, &length)) {
return NULL;
}
for (i = 0; i < length; i++) {
a += (int)data[i];
b += (length - i) * (int)data[i];
}
checksum = (b << 16) | a;
return Py_BuildValue("(Kii)", checksum, (int)a, (int)b);
}
我通过打开一个文件并向其提供 4096 数据块来使用它。对于小字符串，它们都返回相同的值，但是当我直接从文件中向其提供二进制数据时，C 版本返回截然不同的值。任何帮助将不胜感激。
I have a checksum function in Python:
def checksum(data):
a = b = 0
l = len(data)
for i in range(l):
a += ord(data[i])
b += (l - i)*ord(data[i])
return (b << 16) | a, a, b
that I am trying to port to a C module for speed. Here's the C function:
static PyObject *
checksum(PyObject *self, PyObject *args)
{
int i, length;
unsigned long long a = 0, b = 0;
unsigned long long checksum = 0;
char *data;
if (!PyArg_ParseTuple(args, "s#", &data, &length)) {
return NULL;
}
for (i = 0; i < length; i++) {
a += (int)data[i];
b += (length - i) * (int)data[i];
}
checksum = (b << 16) | a;
return Py_BuildValue("(Kii)", checksum, (int)a, (int)b);
}
I use it by opening a file and feeding it a 4096 block of data. They both return the same values for small strings, but when I feed it binary data straight from a file, the C version returns wildly different values. Any help would be appreciated.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
我猜你的局部变量有某种溢出。 b 可能会变大。只需转储这些值以进行调试，您就应该看看这是否是问题所在。正如您提到的，出于性能原因您正在移植该方法。你检查过心理吗？可能足够快而且容易得多。还有更多其他工具可以将 Python 代码的一部分即时编译为 C，但我脑子里没有名字。
I would guess that you have some kind of overflow in your local variables. Probably b gets to large. Just dump the values for debugging purposes and you should see if it's the problem. As you mention, that you are porting the method for performance reasons. Have you checked psyco? Might be fast enough and much easier. There are more other tools which compile parts of python code on the fly to C, but I don't have the names in my head.
我建议原来的校验和函数是“不正确的”。校验和返回的值的大小不受限制（对于以 MB 为单位的任何给定大小，您可以构造一个输入，其校验和至少为该大小）。如果我的计算正确，对于小于 260 MB 的输入，该值可以容纳在 64 位中，并且
b可以容纳小于 4096 字节的整数。现在，我可能对这个数字不满意，但这意味着对于更大的输入，这两个函数保证以不同的方式工作。要将第一个函数转换为 C 语言，您需要将
b和c保留为 Python 整数，并以 Python 表达式的形式执行最后一个计算。不过，这可以改进：n，则a的最大值为n * 255，b的最大值为len（数据）* n * 255。将它们存储在 Clong long变量中时，尝试将它们保留在2**63-1下。long long而不是unsigned long long，并在每次在调试模式下变为负值时引发RuntimeError。另一种解决方案是使用
a & 将 Python 等效项限制为 64 位。 0xffffffffffffffff和b & 0xffffffffffffffff。最好的解决方案是使用另一种校验和，例如 binascii.crc32。
I'd suggest that the original checksum function is "incorrect". The value returned for checksum is of unlimited size (for any given size in MB, you could construct an input for which the checksum will be at least of this size). If my calculations are correct, the value can fit in 64 bits for inputs of less than 260 MB, and
bcan fit in an integer for anything less than 4096 bytes. Now, I might be off with the number, but it means that for larger inputs the two functions are guaranteed to work differently.To translate the first function to C, you'd need to keep
bandcin Python integers, and to perform the last calculation as a Python expression. This can be improved, though:long longvariables to store an intermediate sum and add it to the Python integers after a certain number of iterations. If the number of iterations isn, the maximum value foraisn * 255, and forbislen(data) * n * 255. Try to keep those under2**63-1when storing them in Clong longvariables.long longinstead ofunsigned long long, and raise aRuntimeErrorevery time it gets negative in debug mode.Another solution would be to limit the Python equivalent to 64 bits by using
a & 0xffffffffffffffffandb & 0xffffffffffffffff.The best solution would be to use another kind of checksum, like
binascii.crc32.