Prefect: Improve the experience of mapped child tasks

[emcake]

Current behavior

Mapping is an excellent tools, especially when needing to perform dynamic-like operations on large sets of data - however, the current user experience can leave a little to be desired.

Currently problems (as I see it) with mapped tasks are:

• There's no way in the UI to see how many mapped tasks there _will be_. Mapped tasks appear as they discovered, even though (in principle) at the point that the parent state transitions to Mapped, it's possible to know the total number of mapped children that will appear.
• It is hard to identify a given mapped child and what it represents - This can be a headache when you are trying to work out what differentiates why Mapped Child 38272 worked but Mapped Child 17368 failed.
• The UI representation of mapped children is quite small - that is, either mapped children are available on the same Gantt chart view as all of the other tasks, or as a small window on the schematic view. The Gantt chart becomes unusable very quickly when working with O(10-100k) mapped children. While the schematic representation shows a list of mapped runs, the small drop-down again becomes hard to use with large numbers of mapped children. There also doesn't seem to be a way to view sub-schematics created by apply_map - they are clearly known as they appear in the visualisation, but I think viewing it as its own specific sub-schematic on a mapped child basis would be very useful.

Proposed behavior

I think it would be useful to:

• Provide some kind of 'Mapped Children' view on the UI. I think having a specific table view of each mapped child and its current status would be valuable, especially for higher numbers of mapped tasks.
• Provide a sub-schematic view. Having a schematic view for a given apply_mapped child DAG would be very useful.
• Allow attaching a name to mapped children. In general this could be something more general (something like attaching run context to a given task run but being able to override the name with some kind of dynamic_name that would be used in preference on the UI would be a good idea IMO. This could be provided based on a Callable dynamic_name parameter on Task.map and apply_map that gets given the inputs and outputs a str.

[cicdw]

Hey @emcake - thanks for the feedback, these are really great suggestions; coincidentally we have line-of-sight to releasing your first point with some new Core / UI releases next week.

For point 2, we made a first pass attempt at that here but ran into some complexities as described in that PR. We are considering exposing a mutation that allows you to update the name of your task run dynamically, after it has been created, which we could then expose as a programmatic call in a mapped pipeline and display that in the UI as you describe.

For the UI specific feedback, we might want to migrate to a separate issue on the UI repo but in the meantime pinging @znicholasbrown for visibility.

[cicdw]

Another related PR that furthers this: https://github.com/PrefectHQ/prefect/pull/3420

[cicdw]

Hi @emcake - I'm going to close this issue as I believe most of your feedback has now been incorporated:

• task runs can now set their own run names dynamically through the API and through Prefect Core natively
• there is a new Mapped Children tab on the task run page of a mapped task that allows you to sort / filter for children
• the schematic loads much quicker for large mapped pipelines and displays both the expected number of children as well as their state statistics
• there is _some_ progress on sub-schematic type views on the mapped children run page, but it doesn't capture a full apply_map call. I will still close this issue in favor of a more targeted issue for that request in the future.

Thanks for the feedback and for your patience as we rolled these changes out!