Python 依赖分析器库

发布于 2024-11-17 06:22:07 字数 988 浏览 3 评论 0原文

我需要一种方法来在运行时找到每个 Python 包子模块的依赖关系,以便我可以按正确的顺序初始化它们(请参阅我当前的[编辑: 以前 ] 解决方案 这里,效果不太好),所以一开始我使用了标准的Python模块modulefinder,但这太慢了(~1-2每个模块的秒数)。

我的下一个选择是分析每个模块的所有全局变量,并从这些全局变量中找到每个子模块依赖于哪个子模块。 (这是我当前的解决方案编辑:我现在有一个更好的解决方案 - 请参阅我的答案)。此算法比 modulefinder得多(每个模块需要 <200 毫秒),但它仅适用于相对导入,而不适用于完全限定的导入样式,即不可接受的。

所以,我需要的是:

  • 更快的替代 modulefinder
  • 替代算法

注意: 我在每个模块的开头调用我的依赖分析器,如下所示:(

# File my_package/module3.py

import my_package.module1 # Some misc. module
import my_package.module2 # Some other misc. module
import my_package.dependency_analyzer

my_package.dependency_analyzer.gendeps()

就在如果对您有帮助。)

谢谢!

编辑:我现在有一个解决方案 - 请参阅我的答案。

I need a way to find the dependencies for each of my Python package's sub-modules at runtime so I can initialize them in a proper order (see my current [EDIT: former] solution here, which doesn't work to well), so at first I used the standard Python module modulefinder, but that was way too slow (~1-2 seconds per module).

My next choice was to analyze all the globals of each module, and find from those globals which sub-module each sub-module depends upon. (This is my current solution EDIT: I have a better solution now - see my answer). This algorithm is much faster than modulefinder (it takes <200ms per module), but it only works for relative imports, instead of the fully-qualified import style, which is unacceptable.

So, what I need is either:

  • A quicker alternative to modulefinder
  • An alternative algorithm

NOTE: I call my dependency analyzer at the start of each module, like so:

# File my_package/module3.py

import my_package.module1 # Some misc. module
import my_package.module2 # Some other misc. module
import my_package.dependency_analyzer

my_package.dependency_analyzer.gendeps()

(Just in case it helps you any.)

Thank you!

EDIT: I have a solution now - see my answer.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

天煞孤星 2024-11-24 06:22:07

我想我对自己的问题有一个解决方案:)

这是上面讨论的 dependency_analyzer 模块的内容:

import sys
from sys import _getframe as getframe
import atexit

examined_modules = []

def gendeps():
    """Adds the calling module to the initialization queue."""
    # Get the calling module's name, and add it to the intialization queue
    calling_module_name = getframe(1).f_globals['__name__']
    examined_modules.append(calling_module_name)

def init():
    """Initializes all examined modules in the correct order."""

    for module in examined_modules:
        module = sys.modules[module]
        if hasattr(module, 'init'):
            module.init()
        if hasattr(module, 'deinit'):
            # So modules get de-initialized in the correct order,
            # as well
            atexit.register(module.deinit)

现在,在每个模块的开始处(所有import 语句 - 这很重要),调用 gendeps 。该算法之所以有效,是因为每次导入模块时,都会执行对 gendeps 的调用。但是,由于所有 import 语句都放置在您自己的模块中对 gendeps 的调用之前,因此最不依赖的模块首先放置在初始化队列中,然后是最依赖的模块。依赖模块最后放置在初始化队列中。

I think I have a solution to my own question :)

Here's what would go into the dependency_analyzer module talked about above:

import sys
from sys import _getframe as getframe
import atexit

examined_modules = []

def gendeps():
    """Adds the calling module to the initialization queue."""
    # Get the calling module's name, and add it to the intialization queue
    calling_module_name = getframe(1).f_globals['__name__']
    examined_modules.append(calling_module_name)

def init():
    """Initializes all examined modules in the correct order."""

    for module in examined_modules:
        module = sys.modules[module]
        if hasattr(module, 'init'):
            module.init()
        if hasattr(module, 'deinit'):
            # So modules get de-initialized in the correct order,
            # as well
            atexit.register(module.deinit)

Now, at the start of each module (after all the import statements - this is crucial), a call to gendeps is placed. This algorithm works because each time a module is imported, that call to gendeps is executed. However, since all of the import statements are placed before the call to gendeps in your own module, the least-dependent modules are placed in the initialization queue first, and the most-dependent modules are placed in the initialization queue last.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文