是否有一种干净的方法可以在单个节点上运行NextFlow(基于DSL2)的子工作流?
我在子工作流中定义了一个管道:
workflow pipeline{
main:
task1()
task2(task1.out)
...
}
由主工作流来调用不同输入:
workflow{
params.in = "$baseDir/input"
input_ch = Channel.fromPath(params.in + "/*.*")
main:
pipeline(input_ch)
}
它易于添加标签以处理以定义其资源。我想做的就是将这样的标签添加到我的(整个)子Workflow中。他们的标签应该迫使子处理程序执行为一个单个过程(这意味着它不会调用所有进程的新slurm调用,而仅调用每个进程的呼叫),例如:
- pipeline for input 1 on node 1 as one slurmjob
- pipeline for input 2 on node 2 as one slurmjob
- main: - collect all the outputs
...
- pipeline for input n on node n as one slurmjob
我是通过在nextflow内部调用nextflow来做到的:
process nextflowofnextflows {
label 'use_whatever_is_needed'
input:
file input1
output:
path("$out/*.*", emit: output)
script:
"""
nextflow run pipline.nf --in=${input1}
"""
}
...-但这是超级不洁的,我想以一种干净的方式对其进行处理。
实现所描述行为的最佳实践是什么?
最好,t。
i have a pipeline defined in a sub-workflow:
workflow pipeline{
main:
task1()
task2(task1.out)
...
}
that gets called by the main-workflow for different inputs:
workflow{
params.in = "$baseDir/input"
input_ch = Channel.fromPath(params.in + "/*.*")
main:
pipeline(input_ch)
}
its easy to add a label to process to define its resources. what i want to do is add such a label to my (whole) subworkflow. they label is supposed to force the subworkflow to execute as a if it was a single process (meaning that it does not invoke a new slurm call for all processes but only per subworkflow) like:
- pipeline for input 1 on node 1 as one slurmjob
- pipeline for input 2 on node 2 as one slurmjob
- main: - collect all the outputs
...
- pipeline for input n on node n as one slurmjob
i did as above by calling nextflow inside of nextflow:
process nextflowofnextflows {
label 'use_whatever_is_needed'
input:
file input1
output:
path("$out/*.*", emit: output)
script:
"""
nextflow run pipline.nf --in=${input1}
"""
}
... - but this is super unclean and i´d like to to it in a clean way.
whats the best practice to achieve the described behavior?
best, t.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论