气流 - 在循环任务之前添加一个任务,该任务取决于先前的任务

发布于 2025-02-04 18:58:45 字数 1293 浏览 0 评论 0原文

我试图使我的任务依赖关系正确。到目前为止,我有8个任务取决于以前的任务,这很棒,但是我想先在这些任务之前运行一个任务,但是我似乎无法正确正确。在屏幕截图中,您会看到8个任务Initized_costs_to_s3_0 ... 1 ... 2 ........ 7。但是在此之前,我想拥有最新的cost_cost,就像我对交通界的项目一样(请参阅屏幕截图)。这是我针对这两个任务的代码:

”在此处输入图像描述” 我希望它看起来如何:

l = []
for endpoint in ENDPOINTS:
    latest_only = LatestOnlyOperator(
        task_id=f'{endpoint.name}_latest_only',
    )

    if endpoint.name is 'itemized_costs':
        a = []
        # Load each end points data to S3
        for i in range(0, 8):
            s3 = PToS3Operator(
                task_id=f'{endpoint.name}_to_S3_{i}',
                task_no=f'{int(i)}',
                pool_slots=5,
                endpoint=endpoint
            )
            a.append(s3)
            if i not in [0]:
                a[i-1] >> a[i]

    else:
        s3 = PToS3Operator(
            task_id=f'{endpoint.name}_to_S3',
            pool_slots=5,
            endpoint=endpoint
        )
    latest_only >> s3 

I'm trying to get my task dependencies correct. So far I have 8 tasks that are dependent on the previous task which is great, but then I want to have a task run before these tasks first, but I can't seem to get it correct. In the screenshot you'll see 8 tasks itemized_costs_to_s3_0... 1...2........7. but before that I want to have itemized_cost latest only like I have for the traffic line items (see screenshot). This is my code for those 2 tasks:

enter image description here
how I would like it to look:
enter image description here

l = []
for endpoint in ENDPOINTS:
    latest_only = LatestOnlyOperator(
        task_id=f'{endpoint.name}_latest_only',
    )

    if endpoint.name is 'itemized_costs':
        a = []
        # Load each end points data to S3
        for i in range(0, 8):
            s3 = PToS3Operator(
                task_id=f'{endpoint.name}_to_S3_{i}',
                task_no=f'{int(i)}',
                pool_slots=5,
                endpoint=endpoint
            )
            a.append(s3)
            if i not in [0]:
                a[i-1] >> a[i]

    else:
        s3 = PToS3Operator(
            task_id=f'{endpoint.name}_to_S3',
            pool_slots=5,
            endpoint=endpoint
        )
    latest_only >> s3 

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文