Python：SWIG 与 ctypes
在 python 中，在什么情况下 SWIG 是调用共享库中的入口点比 ctypes 更好的选择？ 假设您还没有 SWIG 接口文件。
两者的性能指标是什么？
In python, under what circumstances is SWIG a better choice than ctypes for calling entry points in shared libraries? Let's assume you don't already have the SWIG interface file(s).
What are the performance metrics of the two?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（10）
我有丰富的使用 swig 的经验。 SWIG 声称它是包装物品的快速解决方案。 但在现实生活中...
缺点：
SWIG 的开发是通用的，适用于所有人和 20 多种语言。 一般来说，它会带来缺点：
- 需要配置（SWIG .i 模板），有时很棘手，
- 缺乏对一些特殊情况的处理（进一步参见 python 属性），
- 某些语言缺乏性能。
Python 缺点：
1) 代码风格不一致。 C++ 和 python 的代码风格非常不同（这当然是显而易见的），让目标代码变得更加 Pythonish 的可能性非常有限。 举个例子，从 getter 和 setter 创建属性是很糟糕的。 请参阅此问答
2) 缺乏广泛的社区。 SWIG 有一些很好的文档。 但是，如果发现文档中没有的内容，则根本没有任何信息。 博客和谷歌搜索都没有帮助。 因此，在这种情况下，人们必须深入挖掘 SWIG 生成的代码...我可以说，这太糟糕了...
优点：
在简单的情况下，它确实快速、简单且直接
如果您生成了 swig 接口文件一次，您就可以将此 C++ 代码包装为任何其他 20 多种语言（!!!）。
SWIG 的一大担忧是性能。 自版本 2.04 起，SWIG 包含“-builtin”标志，这使得 SWIG 比其他自动包装方式更快。 至少一些基准显示了这一点。
何时使用 SWIG？
因此，我自己总结了 swig 适合使用的两种情况：
2）如果需要为多种语言包装 C++ 代码。 或者，如果可能有一天，人们需要分发多种语言的代码。 在这种情况下，使用 SWIG 是可靠的。
1) 如果需要快速包装某个 C++ 库中仅几个函数以供最终使用。
现场体验
更新：
我们使用 SWIG 对库进行转换已经过去了一年半的时间。
首先，我们做了一个python版本。 有好几次我们在使用 SWIG 时遇到了麻烦——这是事实。 但现在我们将库扩展到了 Java 和 .NET。 因此，我们有 3 种语言和 1 个 SWIG。 我可以说 SWIG 非常出色，可以节省大量时间。
更新 2：
我们为这个库使用 SWIG 已经有两年了。 SWIG 已集成到我们的构建系统中。 最近我们对 C++ 库的 API 进行了重大更改。 SWIG 工作完美。 我们唯一需要做的就是向 .i 文件添加几个 %rename，这样我们的
CppCamelStyleFunctions()现在就变成了 Python 中的looks_more_pythonish。 首先我担心可能会出现一些问题，但没有出现任何问题。 这是惊人的。 只需进行几次编辑，所有内容就以 3 种语言发布。 现在我确信在我们的案例中使用 SWIG 是一个很好的解决方案。更新3：
我们的图书馆使用 SWIG 已有 3 年多了。 主要变化：Python部分完全用纯Python重写。 原因是我们库的大多数应用程序现在都使用 Python。 即使纯 python 版本比 C++ 包装运行得慢，但用户使用纯 python 更方便，而不必与本机库苦苦挣扎。
SWIG 仍用于 .NET 和 Java 版本。
这里的主要问题是“如果我们从头开始这个项目，我们会使用 SWIG for python 吗？”。 我们会！ SWIG 使我们能够将我们的产品快速分发到多种语言。 它运行了一段时间，使我们有机会更好地了解用户的需求。
I have a rich experience of using swig. SWIG claims that it is a rapid solution for wrapping things. But in real life...
Cons:
SWIG is developed to be general, for everyone and for 20+ languages. Generally, it leads to drawbacks:
- needs configuration (SWIG .i templates), sometimes it is tricky,
- lack of treatment of some special cases (see python properties further),
- lack of performance for some languages.
Python cons:
1) Code style inconsistency. C++ and python have very different code styles (that is obvious, certainly), the possibilities of a swig of making target code more Pythonish is very limited. As an example, it is butt-heart to create properties from getters and setters. See this q&a
2) Lack of broad community. SWIG has some good documentation. But if one caught something that is not in the documentation, there is no information at all. No blogs nor googling helps. So one has to heavily dig SWIG generated code in such cases... That is terrible, I could say...
Pros:
In simple cases, it is really rapid, easy and straight forward
If you produced swig interface files once, you can wrap this C++ code to ANY of other 20+ languages (!!!).
One big concern about SWIG is a performance. Since version 2.04 SWIG includes '-builtin' flag which makes SWIG even faster than other automated ways of wrapping. At least some benchmarks shows this.
When to USE SWIG?
So I concluded for myself two cases when the swig is good to use:
2) If one needs to wrap C++ code for several languages. Or if potentially there could be a time when one needs to distribute the code for several languages. Using SWIG is reliable in this case.
1) If one needs to rapidly wrap just several functions from some C++ library for end use.
Live experience
Update :
It is a year and a half passed as we did a conversion of our library by using SWIG.
First, we made a python version. There were several moments when we experienced troubles with SWIG - it is true. But right now we expanded our library to Java and .NET. So we have 3 languages with 1 SWIG. And I could say that SWIG rocks in terms of saving a LOT of time.
Update 2:
It is two years as we use SWIG for this library. SWIG is integrated into our build system. Recently we had major API change of C++ library. SWIG worked perfectly. The only thing we needed to do is to add several %rename to .i files so our
CppCamelStyleFunctions()nowlooks_more_pythonishin python. First I was concerned about some problems that could arise, but nothing went wrong. It was amazing. Just several edits and everything distributed in 3 languages. Now I am confident that it was a good solution to use SWIG in our case.Update 3:
It is 3+ years we use SWIG for our library. Major change: python part was totally rewritten in pure python. The reason is that Python is used for the majority of applications of our library now. Even if the pure python version works slower than C++ wrapping, it is more convenient for users to work with pure python, not struggling with native libraries.
SWIG is still used for .NET and Java versions.
The Main question here "Would we use SWIG for python if we started the project from the beginning?". We would! SWIG allowed us to rapidly distribute our product to many languages. It worked for a period of time which gave us the opportunity for better understanding our users requirements.
SWIG 生成（相当丑陋的）C 或 C++ 代码。 对于简单的函数（可以直接翻译的东西）来说它很简单，对于更复杂的函数（例如带有输出参数的函数需要额外的翻译步骤才能在 Python 中表示）也相当容易使用。需要编写 C 代码作为接口文件的一部分。 除了简单的使用之外，您还需要了解 CPython 以及它如何表示对象——这并不难，但需要记住一些事情。
ctypes 允许您直接访问 C 函数、结构和其他数据，并加载任意共享库。 您不需要为此编写任何 C，但您确实需要了解 C 的工作原理。 您可能会说，这是 SWIG 的另一面：它不生成代码，并且在运行时不需要编译器，但除了简单使用之外，它确实要求您了解 C 数据类型、转换等内容是如何进行的。内存管理和对齐工作。 您还需要手动或自动将 C 结构、联合和数组转换为等效的 ctypes 数据结构，包括正确的内存布局。
在纯粹的执行中，SWIG 可能比 ctypes 更快——因为实际工作的管理是在编译时用 C 完成的，而不是在运行时用 Python 完成的。 然而，除非您连接许多不同的 C 函数，但每个函数只连接几次，否则开销不太可能真正引人注目。
在开发时，ctypes 的启动成本要低得多：您不必了解接口文件，不必生成 .c 文件并编译它们，不必检查和消除警告。 您只需投入最少的精力即可开始使用单个 C 函数，然后将其扩展为更多函数。 您可以直接在 Python 解释器中进行测试和尝试。 包装大量代码有点乏味，尽管有人尝试使其变得更简单（例如 ctypes-configure）。
另一方面，SWIG 可用于生成多种语言的包装器（除非需要填写特定于语言的详细信息，就像我上面提到的自定义 C 代码一样。）当包装 SWIG 可以在很少帮助的情况下处理大量代码时，代码生成的设置也比 ctypes 等效项简单得多。
SWIG generates (rather ugly) C or C++ code. It is straightforward to use for simple functions (things that can be translated directly) and reasonably easy to use for more complex functions (such as functions with output parameters that need an extra translation step to represent in Python.) For more powerful interfacing you often need to write bits of C as part of the interface file. For anything but simple use you will need to know about CPython and how it represents objects -- not hard, but something to keep in mind.
ctypes allows you to directly access C functions, structures and other data, and load arbitrary shared libraries. You do not need to write any C for this, but you do need to understand how C works. It is, you could argue, the flip side of SWIG: it doesn't generate code and it doesn't require a compiler at runtime, but for anything but simple use it does require that you understand how things like C datatypes, casting, memory management and alignment work. You also need to manually or automatically translate C structs, unions and arrays into the equivalent ctypes datastructure, including the right memory layout.
It is likely that in pure execution, SWIG is faster than ctypes -- because the management around the actual work is done in C at compiletime rather than in Python at runtime. However, unless you interface a lot of different C functions but each only a few times, it's unlikely the overhead will be really noticeable.
In development time, ctypes has a much lower startup cost: you don't have to learn about interface files, you don't have to generate .c files and compile them, you don't have to check out and silence warnings. You can just jump in and start using a single C function with minimal effort, then expand it to more. And you get to test and try things out directly in the Python interpreter. Wrapping lots of code is somewhat tedious, although there are attempts to make that simpler (like ctypes-configure.)
SWIG, on the other hand, can be used to generate wrappers for multiple languages (barring language-specific details that need filling in, like the custom C code I mentioned above.) When wrapping lots and lots of code that SWIG can handle with little help, the code generation can also be a lot simpler to set up than the ctypes equivalents.
CTypes 非常酷，并且比 SWIG 简单得多，但它有一个缺点，即编写不当或恶意的 Python 代码实际上可能会导致 Python 进程崩溃。 您还应该考虑 boost python。 恕我直言，它实际上比 swig 更容易，同时让你可以更好地控制最终的 python 界面。 如果您无论如何都在使用 C++，那么您也无需添加任何其他语言。
CTypes is very cool and much easier than SWIG, but it has the drawback that poorly or malevolently-written python code can actually crash the python process. You should also consider boost python. IMHO it's actually easier than swig while giving you more control over the final python interface. If you are using C++ anyway, you also don't add any other languages to your mix.
您还可以使用 Pyrex，它可以充当胶水高级Python代码和低级C代码之间。 例如，lxml 是用 Pyrex 编写的。
You can also use Pyrex, which can act as glue between high-level Python code and low-level C code. lxml is written in Pyrex, for instance.
根据我的经验，ctypes 确实有一个很大的缺点：当出现问题时（对于任何复杂的接口来说总是如此），调试起来非常困难。
问题是你的堆栈的很大一部分被 ctypes/ffi 魔法遮盖了，并且没有简单的方法来确定你如何到达特定点以及为什么参数值是这样的。
In my experience, ctypes does have a big disadvantage: when something goes wrong (and it invariably will for any complex interfaces), it's a hell to debug.
The problem is that a big part of your stack is obscured by ctypes/ffi magic and there is no easy way to determine how did you get to a particular point and why parameter values are what they are..
我将持相反观点，建议如果可以的话，您应该使用 标准 Python 编写扩展库API。 从 C 和 Python 的角度来看，它确实集成得很好...如果您有 Perl API 的经验，您会发现它非常令人惊喜。
Ctypes 也很好，但正如其他人所说，它不支持 C++。
您要包装的图书馆有多大？ 代码库变化的速度有多快？ 还有其他维护问题吗？ 这些都可能会影响编写 Python 绑定的最佳方式的选择。
I'm going to be contrarian and suggest that, if you can, you should write your extension library using the standard Python API. It's really well-integrated from both a C and Python perspective... if you have any experience with the Perl API, you will find it a very pleasant surprise.
Ctypes is nice too, but as others have said, it doesn't do C++.
How big is the library you're trying to wrap? How quickly does the codebase change? Any other maintenance issues? These will all probably affect the choice of the best way to write the Python bindings.
只是想添加一些我尚未看到提及的注意事项。
[编辑：哎呀，没有看到 Mike Steder 的答案]
如果您想尝试使用非 Cpython 实现（如 PyPy、IronPython 或 Jython），那么 ctypes 是唯一的选择。 PyPy 不允许编写 C 扩展，因此排除了pyrex/cython 和Boost.python。 出于同样的原因，ctypes 是唯一适用于 IronPython 和（最终，一旦它们全部工作）jython 的机制。
正如其他人提到的，不需要编译。 这意味着，如果出现新版本的 .dll 或 .so，您只需将其放入并加载该新版本即可。 只要接口没有改变，就可以减少更换。
Just wanted to add a few more considerations that I didn't see mentioned yet.
[EDIT: Ooops, didn't see Mike Steder's answer]
If you want to try using a non Cpython implementation (like PyPy, IronPython or Jython), then ctypes is about the only way to go. PyPy doesn't allow writing C-extensions, so that rules out pyrex/cython and Boost.python. For the same reason, ctypes is the only mechanism that will work for IronPython and (eventually, once they get it all working) jython.
As someone else mentioned, no compilation is required. This means that if a new version of the .dll or .so comes out, you can just drop it in, and load that new version. As long as the none of the interfaces changed, it's a drop in replacement.
ctypes 很棒，但不处理 C++ 类。 我还发现 ctypes 比直接 C 绑定慢大约 10%，但这在很大程度上取决于您所调用的内容。
如果您打算使用 ctypes，一定要查看 Pyglet 和 Pyopengl 项目，它们有大量 ctype 绑定的示例。
ctypes is great, but does not handle C++ classes. I've also found ctypes is about 10% slower than a direct C binding, but that will highly depend on what you are calling.
If you are going to go with ctypes, definitely check out the Pyglet and Pyopengl projects, that have massive examples of ctype bindings.
需要记住的是，SWIG 仅针对 CPython 实现。 由于 PyPy 和 IronPython 实现也支持 ctypes，因此可能值得使用 ctypes 编写模块以与更广泛的 Python 生态系统兼容。
Something to keep in mind is that SWIG targets only the CPython implementation. Since ctypes is also supported by the PyPy and IronPython implementations it may be worth writing your modules with ctypes for compatibility with the wider Python ecosystem.
我发现 SWIG 的方法有点臃肿（一般来说，不仅仅是 Python），并且很难实现，除非必须克服以明确的心态编写 Python 代码以对 SWIG 友好的痛点，而不是干净地编写- 编写Python代码。 恕我直言，将 C 绑定写入 C++（如果使用 C++）然后使用 ctypes 与任何 C 层进行接口是一个更简单的过程。
如果您要连接的库将 C 接口作为库的一部分，则 ctypes 的另一个优点是您不必编译单独的 python 绑定库来访问第三方库。 这在制定纯 python 解决方案时特别好，可以避免跨平台编译问题（对于在不同平台上提供的第三方库）。 必须以跨平台友好的方式将编译后的代码嵌入到您希望部署在 PyPi 等平台上的包中是一件很痛苦的事情； 关于使用 SWIG 或底层显式 C 代码的 Python 包，我最恼火的一点是它们普遍不可用跨平台。 因此，如果您正在使用跨平台可用的第三方库并围绕它们开发 python 解决方案，请考虑这一点。
作为一个现实世界的示例，请考虑 PyGTK。 这（我相信）使用 SWIG 生成 C 代码来连接 GTK C 调用。 我使用它的时间最短，却发现它的设置和使用非常痛苦，如果您在设置时没有按照正确的顺序执行操作，那么通常会出现奇怪的错误。 这是一次非常令人沮丧的经历，当我查看 GTK 在网络上提供的接口定义时，我意识到编写这些接口到 python ctypes 接口的翻译器是多么简单的练习。 一个名为 PyGGI 的项目诞生了，有一天我能够将 PyGTK 重写为一个功能更强大、更有用的产品，与 GTK C 面向对象的接口完全匹配。 而且它不需要编译 C 代码，使其具有跨平台友好性。 （我实际上是在连接到 webkitgtk 之后，它不是那么跨平台）。 我还可以轻松地将 PyGGI 部署到任何支持 GTK 的平台。
I have found SWIG to be be a little bloated in its approach (in general, not just Python) and difficult to implement without having to cross the sore point of writing Python code with an explicit mindset to be SWIG friendly, rather than writing clean well-written Python code. It is, IMHO, a much more straightforward process to write C bindings to C++ (if using C++) and then use ctypes to interface to any C layer.
If the library you are interfacing to has a C interface as part of the library, another advantage of ctypes is that you don't have to compile a separate python-binding library to access third-party libraries. This is particularly nice in formulating a pure-python solution that avoids cross-platform compilation issues (for those third-party libs offered on disparate platforms). Having to embed compiled code into a package you wish to deploy on something like PyPi in a cross-platform friendly way is a pain; one of my most irritating points about Python packages using SWIG or underlying explicit C code is their general inavailability cross-platform. So consider this if you are working with cross-platform available third party libraries and developing a python solution around them.
As a real-world example, consider PyGTK. This (I believe) uses SWIG to generate C code to interface to the GTK C calls. I used this for the briefest time only to find it a real pain to set up and use, with quirky odd errors if you didn't do things in the correct order on setup and just in general. It was such a frustrating experience, and when I looked at the interace definitions provided by GTK on the web I realized what a simple excercise it would be to write a translator of those interface to python ctypes interface. A project called PyGGI was born, and in ONE day I was able to rewrite PyGTK to be a much more functiona and useful product that matches cleanly to the GTK C-object-oriented interfaces. And it required no compilation of C-code making it cross-platform friendly. (I was actually after interfacing to webkitgtk, which isn't so cross-platform). I can also easily deploy PyGGI to any platform supporting GTK.