允许在 Python 命令行中覆盖配置选项的最佳方法是什么?
我有一个 Python 应用程序,需要相当多(~30)个配置参数。到目前为止,我使用 OptionParser 类在应用程序本身中定义默认值,并且可以在调用应用程序时在命令行中更改各个参数。
现在我想使用“正确的”配置文件,例如来自 ConfigParser 类的配置文件。同时,用户仍然应该能够在命令行更改各个参数。
我想知道是否有任何方法可以结合这两个步骤,例如使用 optparse (或较新的 argparse )来处理命令行选项,但从 ConfigParse 语法中的配置文件中读取默认值。
有什么想法如何以简单的方式做到这一点?我真的不喜欢手动调用 ConfigParse,然后手动将所有选项的所有默认值设置为适当的值...
I have a Python application which needs quite a few (~30) configuration parameters. Up to now, I used the OptionParser class to define default values in the app itself, with the possibility to change individual parameters at the command line when invoking the application.
Now I would like to use 'proper' configuration files, for example from the ConfigParser class. At the same time, users should still be able to change individual parameters at the command line.
I was wondering if there is any way to combine the two steps, e.g. use optparse (or the newer argparse) to handle command line options, but reading the default values from a config file in ConfigParse syntax.
Any ideas how to do this in an easy way? I don't really fancy manually invoking ConfigParse, and then manually setting all defaults of all the options to the appropriate values...
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(11)
我刚刚发现你可以使用 argparse.ArgumentParser.parse_known_args() 来做到这一点。首先使用
parse_known_args()
从命令行解析配置文件,然后使用 ConfigParser 读取它并设置默认值,然后使用parse_args()
解析其余选项>。这将允许您拥有默认值,使用配置文件覆盖该默认值,然后使用命令行选项覆盖该默认值。例如:没有用户输入的默认值:
来自配置文件的默认值:
来自配置文件的默认值,由命令行覆盖:
argprase-partial.py如下。正确处理
-h
for help 稍微复杂一些。I just discovered you can do this with
argparse.ArgumentParser.parse_known_args()
. Start by usingparse_known_args()
to parse a configuration file from the commandline, then read it with ConfigParser and set the defaults, and then parse the rest of the options withparse_args()
. This will allow you to have a default value, override that with a configuration file and then override that with a commandline option. E.g.:Default with no user input:
Default from configuration file:
Default from configuration file, overridden by commandline:
argprase-partial.py follows. It is slightly complicated to handle
-h
for help properly.查看 ConfigArgParse - 它是一个新的 PyPI 包(开源),作为 argparse 的替代品,增加了对配置文件和环境变量的支持。
Check out ConfigArgParse - its a new PyPI package (open source) that serves as a drop in replacement for argparse with added support for config files and environment variables.
我使用 ConfigParser 和 argparse 以及子命令来处理此类任务。下面代码中的重要一行是:
这会将子命令(来自 argparse)的默认值设置为配置文件部分中的值。
更完整的示例如下:
I'm using ConfigParser and argparse with subcommands to handle such tasks. The important line in the code below is:
This will set the defaults of the subcommand (from argparse) to the values in the section of the config file.
A more complete example is below:
我不能说这是最好的方法,但我有一个我制作的 OptionParser 类,它的作用就像 optparse.OptionParser 一样,默认值来自配置文件部分。您可以拥有它...
随意浏览来源。测试位于同级目录中。
I can't say it's the best way, but I have an OptionParser class that I made that does just that - acts like optparse.OptionParser with defaults coming from a config file section. You can have it...
Feel free to browse the source. Tests are in a sibling directory.
更新:这个答案仍然有问题;例如,它无法处理必需的参数,并且需要笨拙的配置语法。相反, ConfigArgParse 似乎正是这个问题所要求的,并且是一个透明的、直接的替代品。
当前的一个问题是,如果配置文件中的参数无效,则不会出错。这是一个具有不同缺点的版本:您需要在键中包含
--
或-
前缀。这是 python 代码(带有 MIT 许可证的 Gist 链接):
这是配置文件的示例:
现在,运行
但是,如果我们的配置文件有错误:
运行脚本将根据需要产生错误:
主要缺点是它使用了
parser.parse_args
来进行错误检查来自 ArgumentParser,但我不知道有任何替代方案。Update: This answer still has issues; for example, it cannot handle
required
arguments, and requires an awkward config syntax. Instead, ConfigArgParse seems to be exactly what this question asks for, and is a transparent, drop-in replacement.One issue with the current is that it will not error if the arguments in the config file are invalid. Here's a version with a different downside: you'll need to include the
--
or-
prefix in the keys.Here's the python code (Gist link with MIT license):
Here's an example of a config file:
Now, running
However, if our config file has an error:
Running the script will produce an error, as desired:
The main downside is that this uses
parser.parse_args
somewhat hackily in order to obtain the error checking from ArgumentParser, but I am not aware of any alternatives to this.您可以使用 ChainMap
您可以组合来自命令行的值,环境变量、配置文件,如果该值不存在,则定义一个默认值。
You can use ChainMap
You can combine values from command line, environment variables, configuration file, and in case if the value is not there define a default value.
fromfile_prefix_chars
也许不是最干净的 API,但值得了解。
main.py
然后:
文档: https://docs.python .org/3.6/library/argparse.html#fromfile-prefix-chars
这个
@opts.txt
约定有一些先例,例如在GCC工具链中:命令行中的“@”是什么意思?如何使用正确的 CLI 选项来指示选项文件而不是丑陋的
@
东西:如何让 argparse 从带有选项而不是前缀的文件中读取参数在 Python 3.6.5、Ubuntu 18.04 上测试。
fromfile_prefix_chars
Maybe not the cleanest of APIs, but worth knowing about.
main.py
Then:
Documentation: https://docs.python.org/3.6/library/argparse.html#fromfile-prefix-chars
This
@opts.txt
convention has some precedents e.g. in the GCC toolchain: What does "@" at the command line mean?How to use a proper CLI option to indicate the options file rather than the ugly
@
thing: how to get argparse to read arguments from a file with an option rather than prefixTested on Python 3.6.5, Ubuntu 18.04.
这里值得一提的是 jsonargparse (注意我是该库的作者),具有 MIT 许可证并可在 PyPI 上使用。它也是 argparse 的扩展,支持从配置文件和环境变量加载。它与 ConfigArgParse 类似,但它更新,具有更多有用的功能并且维护得很好。
一个示例
main.py
如下:拥有一个配置文件
config.yaml
,其内容为:然后从命令行运行一个示例:
Worth mentioning here is jsonargparse (Note I am an author of the library), with MIT license and available on PyPI. It is also an extension of argparse that supports loading from config files and environment variables. It is similar to ConfigArgParse, but it is newer, with many more useful features and well maintained.
An example
main.py
would be:Having a config file
config.yaml
with content:Then an example run from the command line:
尝试以这种方式
使用它:
并创建示例配置:
Try to this way
Use it:
And create example config:
parse_args()
可以获取现有的命名空间并将现有的命名空间与当前正在解析的参数/选项合并; “当前解析”中的选项 args/options 优先覆盖现有命名空间中的任何内容:我已经将其模拟为真正的配置文件选项。我解析两次,一次作为“预解析”以查看用户是否传递了配置文件,然后再次进行集成可选配置文件命名空间的“最终解析”。
我有一个非常简单的 JSON 配置文件 config.ini:
当我
使用各种 cmd 行设置运行此命令时,我得到:
parse_args()
can take an existing Namespace and merge the existing Namespace with args/options it's currently parsing; the options args/options in the "current parsing" take precedence an override anything in the existing Namespace:I've mocked it up for a real config file option. I'm parsing twice, once, as a "pre-parse" to see if the user passed a config-file, and then again for the "final parse" that integrates the optional config-file Namespace.
I have this very simple JSON config file, config.ini:
and when I run this:
with various cmd-line settings, I get:
我发现现有的答案不足,尤其是在处理子命令和所需参数时。
这是我最终得到的解决方案:
也许有点时髦,但它非常适合我的用例。
I've found the existing answers to come up short, especially when dealing with subcommands and required arguments.
Here's the solution I ended up with:
Maybe a little funky, but it works well for my use case.