I would like to do simple for loop with prefect,
with Flow('task') as flow:
dt_list = get_date()
dt = task1.map(dt_list)
task2.map(dt_list)
but get this dependence:
Task 'task1[0]
Task 'task1[1]
Task 'task1[2]
....
Task 'task2[0]'
Task 'task2[1]'
Task 'task2[2]'
try to use set_upstream or trigger any_suceesful but did't work too
with Flow('task') as flow:
dt_list = get_target_date()
dt = task1.map(dt_list)
task2.map(dt_list)
task2.set_upstream(task1)
How Can I bind the task dependence between task1 and task2 like this?
Thanks
Task 'task1[0]'
Task 'task2[0]'
....
Task 'task1[1]'
Task 'task2[1]'
....
Task 'task1[2]'
Task 'task2[2]'
....
...
@ponggung I'm not sure I fully understand what you're trying to do here but Prefect does have a concept of looping with dynamic DAGs and this may help achieve your goal
@ponggung I believe what you are looking for is what we call "Depth First Execution": #2041
We plan to prioritize this work for one of the next releases!
@joshmeek @cicdw
Yes, Depth First Execution is exactly what I am looking for !
Thank you for your kindly reply and information,
and I expect the next releases to come.
Most helpful comment
@ponggung I believe what you are looking for is what we call "Depth First Execution": #2041
We plan to prioritize this work for one of the next releases!