气流 - 在循环任务之前添加一个任务,该任务取决于先前的任务
我试图使我的任务依赖关系正确。到目前为止,我有8个任务取决于以前的任务,这很棒,但是我想先在这些任务之前运行一个任务,但是我似乎无法正确正确。在屏幕截图中,您会看到8个任务Initized_costs_to_s3_0 ... 1 ... 2 ........ 7。但是在此之前,我想拥有最新的cost_cost,就像我对交通界的项目一样(请参阅屏幕截图)。这是我针对这两个任务的代码:
l = []
for endpoint in ENDPOINTS:
latest_only = LatestOnlyOperator(
task_id=f'{endpoint.name}_latest_only',
)
if endpoint.name is 'itemized_costs':
a = []
# Load each end points data to S3
for i in range(0, 8):
s3 = PToS3Operator(
task_id=f'{endpoint.name}_to_S3_{i}',
task_no=f'{int(i)}',
pool_slots=5,
endpoint=endpoint
)
a.append(s3)
if i not in [0]:
a[i-1] >> a[i]
else:
s3 = PToS3Operator(
task_id=f'{endpoint.name}_to_S3',
pool_slots=5,
endpoint=endpoint
)
latest_only >> s3
I'm trying to get my task dependencies correct. So far I have 8 tasks that are dependent on the previous task which is great, but then I want to have a task run before these tasks first, but I can't seem to get it correct. In the screenshot you'll see 8 tasks itemized_costs_to_s3_0... 1...2........7. but before that I want to have itemized_cost latest only like I have for the traffic line items (see screenshot). This is my code for those 2 tasks:
l = []
for endpoint in ENDPOINTS:
latest_only = LatestOnlyOperator(
task_id=f'{endpoint.name}_latest_only',
)
if endpoint.name is 'itemized_costs':
a = []
# Load each end points data to S3
for i in range(0, 8):
s3 = PToS3Operator(
task_id=f'{endpoint.name}_to_S3_{i}',
task_no=f'{int(i)}',
pool_slots=5,
endpoint=endpoint
)
a.append(s3)
if i not in [0]:
a[i-1] >> a[i]
else:
s3 = PToS3Operator(
task_id=f'{endpoint.name}_to_S3',
pool_slots=5,
endpoint=endpoint
)
latest_only >> s3
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
![扫码二维码加入Web技术交流群](/public/img/jiaqun_03.jpg)
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论