如何让wandb按位置传递参数？

发布于 2025-01-09 13:04:23 字数 551 浏览 2 评论 0原文

我正在尝试探索 python 脚本“train.py”上不同参数设置的结果。为此，我使用了 wandb 扫描。每个 wandb 代理都会执行文件“train.py”并向其传递一些参数。根据 wandb 文档 (https://docs.wandb.ai/guides/sweeps/配置#command），例如两个参数“param1”和“param2”的情况下，每个代理使用命令启动文件。

/usr/bin/env python train.py --param1=value1 --param2=value2

但是，“train.py”期望

/usr/bin/env python train.py value1 value2

并按位置解析参数值。我没有编写 train.py，并且希望尽可能不更改它。如何让 wandb 传递前面没有“--param1=”的值？

原文

I am trying to explore the results of different parameter settings on my python script "train.py". For that, I use a wandb sweep. Each wandb agent executes the file "train.py" and passes some parameters to it. As per the wandb documentation (https://docs.wandb.ai/guides/sweeps/configuration#command), in case of e.g. two parameters "param1" and "param2" each agents starts the file with the command

/usr/bin/env python train.py --param1=value1 --param2=value2

However, "train.py" expects

/usr/bin/env python train.py value1 value2

and parses the parameter values by position. I did not write train.py and would like to not change it if possible. How can I get wandb to pass the values without "--param1=" in front?

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

骑趴 2025-01-16 13:04:23

不要认为您可以从 W&B Sweeps 中获得位置参数。不过，您可以尝试一些不需要接触 train.py 文件的解决方法。

您可以创建一个调用程序文件，我们将其命名为 invoke.py。现在，您可以使用它删除关键字参数名称。像这样的东西可能会起作用：

import sys
import subprocess

if len(sys.argv[0]) <= 1:
  print(f"{sys.argv[0]} program_name param0=<param0> param1=<param1> ...")
  sys.exit(0)

program = sys.argv[1]
params = sys.argv[2:]

posparam = []
for param in params:
  _, val = param.split("=")
  posparam.append(val)

command = [sys.executable, program, *posparam]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
sys.stdout.write(out.decode())
sys.stdout.flush()
sys.stderr.write(err.decode())
sys.stderr.flush()
sys.exit(process.returncode)

这允许您调用 train.py 文件，如下所示：

$ python3 invoke.py /path/to/train.py param0=0.001 param1=20 ...

现在要执行 W&B 扫描，您可以创建一个 command: 部分 (参考）在您的 sweeps.yaml 文件中扫描参数param0 和 param1。例如：

program: invoke.py
...
parameters:
  param0:
    distribution: uniform
    min: 0
    max: 1
  param1:
    distribution: categorical
    values: [10, 20, 30]
command:
 - ${env}
 - ${program}
 - /path/to/train.py
 - ${args_no_hyphens}

Don't think you can get positional arguments from W&B Sweeps. However, there's a little work around you can try that won't require you touching the train.py file.

You can create an invoker file, let's call it invoke.py. Now, you can use it get rid of the keyword argument names. Something like this might work:

import sys
import subprocess

if len(sys.argv[0]) <= 1:
  print(f"{sys.argv[0]} program_name param0=<param0> param1=<param1> ...")
  sys.exit(0)

program = sys.argv[1]
params = sys.argv[2:]

posparam = []
for param in params:
  _, val = param.split("=")
  posparam.append(val)

command = [sys.executable, program, *posparam]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
sys.stdout.write(out.decode())
sys.stdout.flush()
sys.stderr.write(err.decode())
sys.stderr.flush()
sys.exit(process.returncode)

This allows you to invoke your train.py file as follows:

$ python3 invoke.py /path/to/train.py param0=0.001 param1=20 ...

Now to perform W&B sweeps you can create a command: section (reference) in your sweeps.yaml file while sweeping over the parameters param0 and param1. For example:

program: invoke.py
...
parameters:
  param0:
    distribution: uniform
    min: 0
    max: 1
  param1:
    distribution: categorical
    values: [10, 20, 30]
command:
 - ${env}
 - ${program}
 - /path/to/train.py
 - ${args_no_hyphens}

回复收藏 0 原文

~没有更多了~