Python 依赖分析器库
我需要一种方法来在运行时找到每个 Python 包子模块的依赖关系,以便我可以按正确的顺序初始化它们(请参阅我当前的[编辑: 以前 ] 解决方案 这里,效果不太好),所以一开始我使用了标准的Python模块modulefinder,但这太慢了(~1-2每个模块的秒数)。
我的下一个选择是分析每个模块的所有全局变量,并从这些全局变量中找到每个子模块依赖于哪个子模块。 (这是我当前的解决方案编辑:我现在有一个更好的解决方案 - 请参阅我的答案)。此算法比 modulefinder 快得多(每个模块需要 <200 毫秒),但它仅适用于相对导入,而不适用于完全限定的导入样式,即不可接受的。
所以,我需要的是:
- 更快的替代 modulefinder
- 替代算法
注意: 我在每个模块的开头调用我的依赖分析器,如下所示:(
# File my_package/module3.py
import my_package.module1 # Some misc. module
import my_package.module2 # Some other misc. module
import my_package.dependency_analyzer
my_package.dependency_analyzer.gendeps()
就在如果对您有帮助。)
谢谢!
编辑:我现在有一个解决方案 - 请参阅我的答案。
I need a way to find the dependencies for each of my Python package's sub-modules at runtime so I can initialize them in a proper order (see my current [EDIT: former] solution here, which doesn't work to well), so at first I used the standard Python module modulefinder, but that was way too slow (~1-2 seconds per module).
My next choice was to analyze all the globals of each module, and find from those globals which sub-module each sub-module depends upon. (This is my current solution EDIT: I have a better solution now - see my answer). This algorithm is much faster than modulefinder (it takes <200ms per module), but it only works for relative imports, instead of the fully-qualified import style, which is unacceptable.
So, what I need is either:
- A quicker alternative to modulefinder
- An alternative algorithm
NOTE: I call my dependency analyzer at the start of each module, like so:
# File my_package/module3.py
import my_package.module1 # Some misc. module
import my_package.module2 # Some other misc. module
import my_package.dependency_analyzer
my_package.dependency_analyzer.gendeps()
(Just in case it helps you any.)
Thank you!
EDIT: I have a solution now - see my answer.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
我想我对自己的问题有一个解决方案:)
这是上面讨论的 dependency_analyzer 模块的内容:
现在,在每个模块的开始处(在所有import 语句 - 这很重要),调用 gendeps 。该算法之所以有效,是因为每次导入模块时,都会执行对 gendeps 的调用。但是,由于所有 import 语句都放置在您自己的模块中对 gendeps 的调用之前,因此最不依赖的模块首先放置在初始化队列中,然后是最依赖的模块。依赖模块最后放置在初始化队列中。
I think I have a solution to my own question :)
Here's what would go into the dependency_analyzer module talked about above:
Now, at the start of each module (after all the import statements - this is crucial), a call to gendeps is placed. This algorithm works because each time a module is imported, that call to gendeps is executed. However, since all of the import statements are placed before the call to gendeps in your own module, the least-dependent modules are placed in the initialization queue first, and the most-dependent modules are placed in the initialization queue last.