在目标上运行后,避免运行所有makefiles

发布于 2025-02-08 08:05:07 字数 2676 浏览 1 评论 0原文

我是新手,并且正在尝试使用它来自动化ML管道。我已经在两个文件中定义了两个规则:第一个makefile具有文件位置和一个清洁数据集的目标。第二个makefile.label.surv1具有使用Python脚本提取标签的目标。以下是两者的代码:

makefile

#--------------------------------------------------------------------------
# Survival Analysis - Churn prediction top makefile
#--------------------------------------------------------------------------

# date of the snapshot to consider
SNAP_TRN := 2019-06-18
SNAP_TST := 2019-07-24

# directories
DIR_DATA := data
DIR_BUILD := build
DIR_FEATURE := $(DIR_BUILD)/feature
DIR_METRIC := $(DIR_BUILD)/metric
DIR_MODEL := $(DIR_BUILD)/model
DIR_CONFIG := configs

# data files for training and predict
DATA_TRN := $(DIR_DATA)/processed/
DATA_TST := $(DIR_DATA)/processed/

# NOS feature selection
FEATS := $(DIR_CONFIG)/featimp_churnvol.csv
# Config files
CONFIG_PANEL := $(DIR_CONFIG)/config_panel.yaml
CONFIG_INPUT := $(DIR_CONFIG)/config_inpute.yaml

# Generates a clean dataset (inputted and one hot encoded) and labels for train and test
buildDataset: $(DATA_TRN) $(DATA_TST) $(CONFIG_PANEL) $(CONFIG_INPUT) $(FEATS)
    python src/buildDataset.py --train-file $< \
                               --test-file $(word 1, $^) \
                               --config-panel $(word 2, $^) \
                               --config-input $(word 3, $^) \
                               --feats $(lastword $^)   

makefile.label.surv1

#--------------------------------------------------------------------------
# surv1: survival labels
#--------------------------------------------------------------------------
include Makefile


FEATURE_NAME := surv1

GRAN := weekly
STUDY_DUR := 1
       
Y_SURV_TRN := $(DIR_DATA)/survival/$(FEATURE_NAME)_train_$(SNAP_TRN)_$(GRAN)_$(STUDY_DUR).pkl
Y_SURV_TST := $(DIR_DATA)/survival/$(FEATURE_NAME)_test_$(SNAP_TST)_$(GRAN)_$(STUDY_DUR).pkl

$(Y_SURV_TRN) $(Y_SURV_TST): $(DATA_TRN) $(DATA_TST) $(CONFIG_PANEL) $(STUDY_DUR) $(GRAN)
    python ./src/generate_surv_labels.py --train-file $< \
                                         --test-file $(word 1, $^) \
                                         --train-label-file $(Y_SURV_TRN) \
                                         --test-label-file $(Y_SURV_TST)\
                                         --config-panel $(word 2, $^) \
                                         --study-dur $(word 3, $^) \
                                         --granularity $(lastword $^)

因此,当我运行make -f make -file.label.surv1时,它还重新运行了目标builddataset,哪个在这种情况下,我不想。在这种情况下,我没有对buildDataSet进行任何更改,所以我不明白为什么要重新运行这个目标...无论如何是否有防止目标重新运行其他目标?

I'm new to Make, and am trying to use it to automate a ml pipeline. I have defined two rules in two files: the first Makefile has file locations and a target to clean a dataset. The second one Makefile.label.surv1 has a target to extract labels using a python script. Below is the code for both:

Makefile

#--------------------------------------------------------------------------
# Survival Analysis - Churn prediction top makefile
#--------------------------------------------------------------------------

# date of the snapshot to consider
SNAP_TRN := 2019-06-18
SNAP_TST := 2019-07-24

# directories
DIR_DATA := data
DIR_BUILD := build
DIR_FEATURE := $(DIR_BUILD)/feature
DIR_METRIC := $(DIR_BUILD)/metric
DIR_MODEL := $(DIR_BUILD)/model
DIR_CONFIG := configs

# data files for training and predict
DATA_TRN := $(DIR_DATA)/processed/
DATA_TST := $(DIR_DATA)/processed/

# NOS feature selection
FEATS := $(DIR_CONFIG)/featimp_churnvol.csv
# Config files
CONFIG_PANEL := $(DIR_CONFIG)/config_panel.yaml
CONFIG_INPUT := $(DIR_CONFIG)/config_inpute.yaml

# Generates a clean dataset (inputted and one hot encoded) and labels for train and test
buildDataset: $(DATA_TRN) $(DATA_TST) $(CONFIG_PANEL) $(CONFIG_INPUT) $(FEATS)
    python src/buildDataset.py --train-file 
lt; \
                               --test-file $(word 1, $^) \
                               --config-panel $(word 2, $^) \
                               --config-input $(word 3, $^) \
                               --feats $(lastword $^)   

Makefile.label.surv1

#--------------------------------------------------------------------------
# surv1: survival labels
#--------------------------------------------------------------------------
include Makefile


FEATURE_NAME := surv1

GRAN := weekly
STUDY_DUR := 1
       
Y_SURV_TRN := $(DIR_DATA)/survival/$(FEATURE_NAME)_train_$(SNAP_TRN)_$(GRAN)_$(STUDY_DUR).pkl
Y_SURV_TST := $(DIR_DATA)/survival/$(FEATURE_NAME)_test_$(SNAP_TST)_$(GRAN)_$(STUDY_DUR).pkl

$(Y_SURV_TRN) $(Y_SURV_TST): $(DATA_TRN) $(DATA_TST) $(CONFIG_PANEL) $(STUDY_DUR) $(GRAN)
    python ./src/generate_surv_labels.py --train-file 
lt; \
                                         --test-file $(word 1, $^) \
                                         --train-label-file $(Y_SURV_TRN) \
                                         --test-label-file $(Y_SURV_TST)\
                                         --config-panel $(word 2, $^) \
                                         --study-dur $(word 3, $^) \
                                         --granularity $(lastword $^)

So when I run make -f Makefile.label.surv1, it also re-runs the target buildDataset, which I don't want in this case. In this case I haven't made any changes to buildDataset so I don't understand why make re-runs this target... Is there anyway to prevent a target from re-running others?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

温柔少女心 2025-02-15 08:05:07

如果您不提供在命令行上构建的目标,则Make将构建默认目标。默认目标是第一个明确的目标,在您的makefile中定义了。

在您的makefile.label.survival定义任何其他目标之前要做的第一件事是include makefile。这意味着,如果在makefile中定义任何目标,则将被视为第一个明确的目标。

而且,实际上,makefile定义buildDataSet,因此这是默认目标,如果您运行make而无需任何特定目标被建造。

另外,该规则很可能不正确:

$(Y_SURV_TRN) $(Y_SURV_TST): ...

我不确定您希望这会做什么,但是如果您期望这是一个人,则可以解释这意味着配方的一个调用将构建这两个文件,这不是什么该语法的意思是。

If you don't provide a target to build on the make command line, make will build the default target. The default target is the FIRST explicit target defined in your makefile(s).

In your Makefile.label.survival the first thing you do before you define any other target, is include Makefile. That means that if any target is defined in Makefile, it will be considered the first explicit target.

And, indeed, Makefile defines buildDataset and so that is the default target and if you run make without any specific target, that's the target that will be built.

Also, this rule is very likely not right:

$(Y_SURV_TRN) $(Y_SURV_TST): ...

I'm not sure what you're hoping this will do, but if you expect that make will interpret this to mean that one invocation of the recipe will build both these files, that's not what this syntax means.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文