带有后备实施的包装Python绑定的最佳实践(PEP 518)

发布于 2025-02-09 21:11:09 字数 2247 浏览 0 评论 0原文

让我们考虑以下示例。 有一个静态的C ++库libfoo,分布成CMake子项目。 现在,有一个称为pyfoo的python实现,该实现应为libfoo提供python绑定(使用pybind11), 但是,如果无法构建libfoo 在目标平台上。

这样一个项目的存储库看起来就是这样:

 +--+ libFoo
 |  +--- CMakeLists.txt (export a static library target foo::foo)
 |  +--- src/..., inc/... (c++ sources for libFoo)
 |
 +--+ pyFoo
    +--- pyproject.toml
    +--- ... (other metadata/package files, tests, etc)
    +--+ foo (main package dir)
       +--+ _native
       |  +--- CMakeLists.txt (imports libFoo/CMakeLists.txt as a subproject, configures, adds pybind11 module)
       |  +--- foo_bindings.cpp (literally pybind11 bindings using imported foo::foo)
       |
       +--- _fallback/... (pure Python implementation of libFoo for compatibility)
       +--- __init__.py (essentially "try: from ._native.foo_bindings import *; except ImportError: from ._fallback import *")

现在,这种结构对我来说是一个开发环境(pyfoo/foo/_native cmake cmake cmake cmake commake complate complate complate foo_bindings to >忍者安装), 为了创建分布和车轮,这似乎不是处理构建本机(C ++)部分的正确方法。

我对其他软件包如何处理此类情况进行了一些研究,但是我只能找到以下方案:

  • 软件包只是一个lib +绑定(没有纯Python实施),因此设置需要本机构建;
  • 该软件包将使用CTYPES/CFFI导入外部库(例如Libssl.So),如果导入失败,则将其倒入Python实现,因此从构建系统的角度来看,它是一个纯粹的Python软件包。

我在这里要实现的目标是拥有一个可以在尽可能多的方案中使用的软件包,包括用户可能无法编译C ++源的情况(例如没有MSVC的Windows)车轮无法用于其平台。

我将如何指定使用尽可能多的现代工具/解决方案(考虑使用PEP 518而不是将所有内容放入设置中), 在保持界面简单清洁的同时?

pip安装pyfoo的所需决策树是:

  • 如果当前平台有轮子,请使用它,
  • 如果没有轮子,请尝试构建foo._native.bindings.bindings从源(使用CMAKE),包括libfoo的直接依赖性。
    • 如果构建成功,一切都还好吧,请在最终设置中使用foo._native.bindings
    • 如果构建失败,请不要断开设置,而是安装除foo._native package以外的所有内容。

奖励问题:如果foo._fallback具有附加的软件包依赖项(超出foo/foo.___native requience quients ,在pyproject.toml中列出它们是一个好习惯吗?如果默认实现是foo._native中的默认实现?

一般指针会没事的。我已经介绍了有关此事的几篇文章,但它们似乎是不完整或基本的。 例如[https://packaging.python.org/en/latest/guides/packaging-binary-extensions/]甚至给出以下示例:

示例:导入DateTime时,python返回DateTime.py模块,如果C实现(_DateTeTimeModule.c)可用。

但是随后无法解决体内的这种情况。

Let's consider the following example.
There's a static C++ library libFoo, distributed as a CMake sub-project.
Now, there's a Python implementation of Foo, called pyFoo, which should provide Python bindings for libFoo (using pybind11),
but does also provide a pure Python implementation of Foo's capabilities, in case libFoo cannot be build
on the target platform.

The repository for such a project would look like this:

 +--+ libFoo
 |  +--- CMakeLists.txt (export a static library target foo::foo)
 |  +--- src/..., inc/... (c++ sources for libFoo)
 |
 +--+ pyFoo
    +--- pyproject.toml
    +--- ... (other metadata/package files, tests, etc)
    +--+ foo (main package dir)
       +--+ _native
       |  +--- CMakeLists.txt (imports libFoo/CMakeLists.txt as a subproject, configures, adds pybind11 module)
       |  +--- foo_bindings.cpp (literally pybind11 bindings using imported foo::foo)
       |
       +--- _fallback/... (pure Python implementation of libFoo for compatibility)
       +--- __init__.py (essentially "try: from ._native.foo_bindings import *; except ImportError: from ._fallback import *")

Now, while this kind of structure worked for me as a development environment (pyFoo/foo/_native CMake moves compiled foo_bindings into an appropriate place upon ninja install),
this doesn't seem like a proper way to handle building the native (C++) part for the sake of creating distributions and wheels.

I did some research on how other packages handle these kinds of situations, but I could only find the following scenarios:

  • The package is just a lib + binding, (no pure Python implementation), so native build is required for setup;
  • The package imports an external library (e.g. libssl.so) using ctypes/cffi and falls back to Python implementation if the import fails, so from the point of view of the build system, it's a pure Python package.

What I'm trying to achieve here, is to have a package that will work in as many scenarios as possible, including cases where users may not be able to compile C++ sources (e.g. Windows without MSVC) and wheels are not available for their platforms.

How would I go about specifying that using as many modern tools/solutions (think using PEP 518 instead of putting everything into setup.py) as possible,
while keeping the interface simple and clean?

A desired decision tree for a pip install pyfoo would be:

  • If there is a wheel for the current platform, use it
  • If there is no wheel, try building foo._native.bindings from source (using CMake), including direct dependency of libfoo.
    • If the build succeeds, everything's all right, use foo._native.bindings in the final setup
    • If the build fails, don't break setup, instead install everything except for the foo._native package.

Bonus question: If foo._fallback has an additional package dependencies (beyond what foo/foo._native require), is it a good practice to list them in pyproject.toml if the default implementation is that in foo._native?

General pointers would be OK. I've looked through several articles on the matter, but they seem to be incomplete, or rather basic.
For instance [https://packaging.python.org/en/latest/guides/packaging-binary-extensions/] even gives the following example:

Example: When importing datetime, Python falls back to the datetime.py module if the C implementation ( _datetimemodule.c) is not available.

but then fails to address this scenario in the body.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文