当前位置：文江博客话题详情

使用 SCons 作为 distutils 的构建引擎

发布于 2024-08-28 19:46:38 字数 280 浏览 11 评论 0原文

我有一个 python 包，其中包含构建扩展所需的一些 C 代码（以及一些重要的构建需求）。我使用 SCons 作为我的构建系统，因为它非常好且灵活。

我正在寻找一种方法来编译我的 python 扩展，并准备好与 distutils 一起分发。我希望用户只需键入 setup.py install 并获得使用 SCons 而不是默认的 distutils 构建引擎编译的扩展。

我想到的一个想法是在 distutils 中重新定义 build_ext 命令，但我找不到它的详细文档。

有什么建议吗？

原文

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

玩物 2024-09-04 19:46:38

enscons 包似乎旨在执行问题所要求的操作。此处是使用它构建具有 C 扩展的包的示例。

您可以有一个基本的包结构，如下

pkgroot/
    pyproject.toml
    setup.py
    SConstruct
    README.md
    pkgname/
        __init__.py
        pkgname.py
        cfile.c

所示：在这个 pyproject.toml 文件中，

[build-system]
requires = ["enscons"]

[tool.enscons]
name = "pkgname"
description = "My nice packahe"
version = "0.0.1"
author = "Me"
author_email = "[email protected]"
keywords = ["spam"]
url = "https://github.com/me/pkgname"
src_root = ""
packages = ["pkgname"]

[tool.enscons] 部分包含许多与 setuptools 熟悉的内容/distutils setup 函数。从此处复制，setup.py函数可能包含类似的内容：

#!/usr/bin/env python

# Call enscons to emulate setup.py, installing if necessary.

import sys, subprocess, os.path

sys.path[0:0] = ['setup-requires']

try:
    import enscons.setup
except ImportError:
    requires = ["enscons"] 
    subprocess.check_call([sys.executable, "-m", "pip", "install", 
        "-t", "setup-requires"] + requires)
    del sys.path_importer_cache['setup-requires'] # needed if setup-requires was absent
    import enscons.setup

enscons.setup.setup()

最后，SConstruct 文件可能类似于：

# Build pkgname

import sys, os
import pytoml as toml
import enscons, enscons.cpyext

metadata = dict(toml.load(open('pyproject.toml')))['tool']['enscons']

# most specific binary, non-manylinux1 tag should be at the top of this list
import wheel.pep425tags
full_tag = next(tag for tag in wheel.pep425tags.get_supported() if not 'manylinux' in tag)

env = Environment(tools=['default', 'packaging', enscons.generate, enscons.cpyext.generate],
                  PACKAGE_METADATA=metadata,
                  WHEEL_TAG=full_tag)

ext_filename = os.path.join('pkgname', 'libcfile')

extension = env.SharedLibrary(target=ext_filename,
                              source=['pkgname/cfile.c'])

py_source = Glob('pkgname/*.py')

platlib = env.Whl('platlib', py_source + extension, root='')
whl = env.WhlFile(source=platlib)

# Add automatic source files, plus any other needed files.
sdist_source=list(set(FindSourceFiles() + 
    ['PKG-INFO', 'setup.py'] + 
    Glob('pkgname/*', exclude=['pkgname/*.os'])))

sdist = env.SDist(source=sdist_source)
env.Alias('sdist', sdist)

install = env.Command("#DUMMY", whl, 
                      ' '.join([sys.executable, '-m', 'pip', 'install', '--no-deps', '$SOURCE']))
env.Alias('install', install)
env.AlwaysBuild(install)

env.Default(whl, sdist)

之后，您应该能够运行

sudo python setup.py install

编译 C 扩展并构建轮子，并安装 python 包，或者

python setup.py sdist

构建源代码发行版。

我认为您基本上可以使用 SConstruct 文件中的 SCons 做任何事情。

The enscons package seems to be designed to do what the question asked. An example of using it to build a package with C extensions is here.

You could have a basic package structure like:

pkgroot/
    pyproject.toml
    setup.py
    SConstruct
    README.md
    pkgname/
        __init__.py
        pkgname.py
        cfile.c

In this the pyproject.toml file could look something like:

[build-system]
requires = ["enscons"]

[tool.enscons]
name = "pkgname"
description = "My nice packahe"
version = "0.0.1"
author = "Me"
author_email = "[email protected]"
keywords = ["spam"]
url = "https://github.com/me/pkgname"
src_root = ""
packages = ["pkgname"]

where the [tool.enscons] section contains many things familiar to the setuptools/distutils setup functions. Copying from here, the setup.py function could contain something like:

#!/usr/bin/env python

# Call enscons to emulate setup.py, installing if necessary.

import sys, subprocess, os.path

sys.path[0:0] = ['setup-requires']

try:
    import enscons.setup
except ImportError:
    requires = ["enscons"] 
    subprocess.check_call([sys.executable, "-m", "pip", "install", 
        "-t", "setup-requires"] + requires)
    del sys.path_importer_cache['setup-requires'] # needed if setup-requires was absent
    import enscons.setup

enscons.setup.setup()

Finally, the SConstruct file could look something like:

# Build pkgname

import sys, os
import pytoml as toml
import enscons, enscons.cpyext

metadata = dict(toml.load(open('pyproject.toml')))['tool']['enscons']

# most specific binary, non-manylinux1 tag should be at the top of this list
import wheel.pep425tags
full_tag = next(tag for tag in wheel.pep425tags.get_supported() if not 'manylinux' in tag)

env = Environment(tools=['default', 'packaging', enscons.generate, enscons.cpyext.generate],
                  PACKAGE_METADATA=metadata,
                  WHEEL_TAG=full_tag)

ext_filename = os.path.join('pkgname', 'libcfile')

extension = env.SharedLibrary(target=ext_filename,
                              source=['pkgname/cfile.c'])

py_source = Glob('pkgname/*.py')

platlib = env.Whl('platlib', py_source + extension, root='')
whl = env.WhlFile(source=platlib)

# Add automatic source files, plus any other needed files.
sdist_source=list(set(FindSourceFiles() + 
    ['PKG-INFO', 'setup.py'] + 
    Glob('pkgname/*', exclude=['pkgname/*.os'])))

sdist = env.SDist(source=sdist_source)
env.Alias('sdist', sdist)

install = env.Command("#DUMMY", whl, 
                      ' '.join([sys.executable, '-m', 'pip', 'install', '--no-deps', '$SOURCE']))
env.Alias('install', install)
env.AlwaysBuild(install)

env.Default(whl, sdist)

After this you should be able to run

sudo python setup.py install

to compile the C extension and build a wheel, and install the python package, or

python setup.py sdist

to build a source distribution.

I think you can basically do anything you could with SCons in the SConstruct file though.

回复收藏 0 原文

初心 2024-09-04 19:46:38

请参阅页面： http://www.scons.org/wiki/PythonExtensions

我是使用稍微修改过的版本来构建 python 的pyrex-c 扩展。

回复收藏 0 原文

分開簡單 2024-09-04 19:46:38

我使用 scons 生成 setup.py 文件。因此我创建了一个名为 setup.py.in 的模板，并使用 scons 扩展该模板以生成 setup.py。

以下是我的项目的一些链接：

setup.py.in 模板

SConstruct

这会计算一个键、值对的字典，以替换到 setup.py.in 模板中以生成 setup.py。

因此，最终用户做了两件事：

scons setup.py
python setup.py install

警告：我的 sconstruct 内容有点混乱，因为我之前写过它，但它应该演示这个概念。

如果您学习如何编写正确的 scons 工具，那么可以将其转换为单个目标，例如：

scons --pymod

这是一个 scons 工具，它使用 SWIG 生成 Python 包装器：

import SCons.Action
from SCons.Script import EnsureSConsVersion

SCons.Script.EnsureSConsVersion(0,96,92)

SwigGenAction = SCons.Action.Action('$SWIGGENCOM', '$SWIGGENCOMSTR')

def emitter(target, source, env):
    """
    Add dependency from target to source
    """

    env.Depends(target, source)

    return target, source


def generate(env):
    """
    Add builders and construction variables for the SwigGen builder.
    """

    if 'SWIGCOM' not in env:
        raise SystemError("SCons build environment could not detect tool: swig")

    bld = env.Builder(
        action = SwigGenAction,
        emitter = emitter,
        target_factory = env.fs.File)

    env['BUILDERS']['SwigGen'] = bld

    env['SWIGGENCOM'] = env['SWIGCOM']


def exists(env):
    return env.Detect('swig')

现在，您可以从 SConstruct 文件生成包装器代码：

foobar_cc = env.SwigGen("foobar_wrap.cc", "foobar.i")

如果你修改 foobar.i 它将重新生成 foobar_wrap.cc。一旦有了这个，您就可以编写其他工具来实际执行 python setup.py install ，因此当提供 --pymod 时，它将构建 python 模块。

I use scons to generate a setup.py file. So I created a template called setup.py.in and use scons to expand the template to generate setup.py.

Here are some links to my project that does this:

setup.py.in template

The SConstruct

This computes a dictionary of key, value pairs to substitute into the setup.py.in template to generate setup.py.

So the end user does two things:

scons setup.py
python setup.py install

Warning: my sconstruct stuff is a bit messy as I wrote it awhile ago, but it should demonstrate the concept.

If you learn how to write proper scons tools, then this can be transformed into a single target, for example:

scons --pymod

Here is a scons tool the generates a Python wrapper with SWIG:

import SCons.Action
from SCons.Script import EnsureSConsVersion

SCons.Script.EnsureSConsVersion(0,96,92)

SwigGenAction = SCons.Action.Action('$SWIGGENCOM', '$SWIGGENCOMSTR')

def emitter(target, source, env):
    """
    Add dependency from target to source
    """

    env.Depends(target, source)

    return target, source


def generate(env):
    """
    Add builders and construction variables for the SwigGen builder.
    """

    if 'SWIGCOM' not in env:
        raise SystemError("SCons build environment could not detect tool: swig")

    bld = env.Builder(
        action = SwigGenAction,
        emitter = emitter,
        target_factory = env.fs.File)

    env['BUILDERS']['SwigGen'] = bld

    env['SWIGGENCOM'] = env['SWIGCOM']


def exists(env):
    return env.Detect('swig')

Now from your SConstruct file, you can generate the wrapper code:

foobar_cc = env.SwigGen("foobar_wrap.cc", "foobar.i")

If you modify foobar.i it will regenerate foobar_wrap.cc. Once you have this, you can write other tools for actually executing python setup.py install for you, so when --pymod is provided it will build the python module.

回复收藏 0 原文

爱*していゐ 2024-09-04 19:46:38

解决方案是提供从 distutils.cmd.Commmand 派生的自定义 cmdclass，它可以按照您想要的方式构建模块：

import distutils.cmd

class build_py_cmd(distutils.cmd.Command):
    def initialize_options(self):
        pass

    def finalize_options(self):
        pass
    def run(self):
        print("Calling SCons to build the module")
        pbs.scons()


setup(name = 'riak3k',
      packages = ['riak3k'],
      package_dir = {'': 'build/release'},
      cmdclass = {'build_py': build_py_cmd},

The solution is to provide custom cmdclass derived from distutils.cmd.Commmand which builds the module how you want it:

import distutils.cmd

class build_py_cmd(distutils.cmd.Command):
    def initialize_options(self):
        pass

    def finalize_options(self):
        pass
    def run(self):
        print("Calling SCons to build the module")
        pbs.scons()


setup(name = 'riak3k',
      packages = ['riak3k'],
      package_dir = {'': 'build/release'},
      cmdclass = {'build_py': build_py_cmd},

回复收藏 0 原文

~没有更多了~