Kibana: [APM] Show parent/span relationship in the span graph

[elasticmachine]

Original comment by @makwarth:

Problem

It's currently supported in (some) agents and the Server that spans can have a child, but we don't show that relationship in the APM UI

Solution

Indicate self/child time on the parent span by lightening child time. Typically there will only be two levels of spans (parent and childs), but theoretically there can be N number of child spans. All child spans are indicated identical on the parent span.

Mockup

Ack

@elastic/apm-agent-devs please give thumbs up on this or leave a comment. We won't proceed till all have given their OK. (This was brought up by @beniwohli)

[elasticmachine]

Original comment by @felixbarny:

+1 on this. Not sure if the color coding is the best solution to indicate this. For reference, this is how zipkin does it: LINK REDACTED Seems like it is possible to expand or collapse child spans.

[elasticmachine]

Original comment by @Qard:

I still think AppNeta TraceView had the best UI I've seen for this feature.

!LINK REDACTED

A span occurring _within_ another, would be a directly below its parent. Two different things at the same level would appear linearly. The view of the graph was also zoomable to a reduced slice that could be dragged around with the bar below the stacked graph. When a span was selected, it'd create a vertical highlight across the entire stacked graph, as they could get somewhat tall in complex systems. Also, the striped span represented a distributed tracing service call, which could be selected to zoom the graph to that service request.

Async things like Node.js callback executions looked like this, though I wasn't a huge fan of it:

!LINK REDACTED

Selecting a span in the graph would also replace the "Trace notes" section with a span details section:

!LINK REDACTED

Rather than our way of storing everything in one big transaction bucket, TraceView treated everything as a span and just used the lack of a parent to identify the outer bounds of the DAG.

[axw]

It would be good to see this with some real data (is that real data?)

I'm wondering if we'll end up with the parent typically being all light because it does nothing but spawn child spans. If the parent gets all washed out, it might end up being non-obvious. I think I'd prefer to see collapsible containers like in the Zipkin UI, with an offset to indicate hierarchy.

[beniwohli]

@axw the screenshot in the issue is a real example of a Django app

[axw]

@axw the screenshot in the issue is a real example of a Django app

OK, cool. In that case, +1. We could easily expand on this as needed.

[alvarolobato]

@formgeist how does this match with the Distributed tracing design?

[formgeist]

@alvarolobato The parent/child relationship is going to be visualized by adding the vertical lines and indention shown here https://github.com/elastic/kibana/issues/20660. We don't show a difference in the parent span visual to indicate its own duration vs. its children.

[formgeist]

@axw @beniwohli @felixbarny Just pinging you, since you had a conversation about showing the relationship. As mentioned above, we're planning to implement an interactive parent/child relationship navigation in the Timeline (https://github.com/elastic/kibana/issues/20660), but I'm not sure if that covers the visual of how we used to display the relationship in Opbeat. The way I see it, there's a benefit in both ways (navigation tree on the left, and showing the self-time vs. children). What do you think? Also is this applicable for all agents?

[mikker]

The new timeline side thing looks good!

I still like this multi-color span duration bar. It gives an instant impression of how much time is spent on children/self. Nothing to understand or parse, it's immediately understood.

[beniwohli]

I think a major issue we need to address here is that the waterfall also kinda looks like a parent/child relationship if you don't look too closely. Having an actual parent/child relationship inside of the waterfall could potentially confuse people into thinking that the waterfall symbolizes a causal (span A caused span B) instead of a temporal relationship (span A happened before span B).

The navigation tree on the left might help with that, but it's hard to tell in your examples. How would it look if most spans don't have children?

[axw]

The way I see it, there's a benefit in both ways (navigation tree on the left, and showing the self-time vs. children). What do you think?

Yes, I think there's value to both. The striped bar gives a nice visual summary.

Also is this applicable for all agents?

Nothing agent-specific as far as I can see. Some _applications_ might not make much use of the stripes due to hypothetical scenarios like I described in https://github.com/elastic/kibana/issues/18205#issuecomment-385346598. I don't think that's a problem, though.

[roncohen]

The light coloration helps determine if something is taking up time itself, or something is taking up time because a child span is taking up time. In a sense it's showing self-time vs. child-time.

To do this properly, it seems to me that we would need to differentiate between async child spans and sync child spans. For example, in a Django app, child spans are practically always synchronous and there it makes sense to color the parent in the light color when there's a child span. For Node.js, you could imagine a span that causes a child span, but because its async, we shouldn't color the parent in the light color. Typically, the parent span will end soon after it's started the child(ren).

[formgeist]

@roncohen I assume that the differentiation process will happen in the UI by checking whether the child span is aligned with the parent or not, and then lighten the parent span in the area where the synchronous child span is, right? Just wanted to understand where the process lies in your proposal.

[felixbarny]

There is a proposal to add an async flag to spans. If that is set to false (as opposed to null, which means no info whether the span was sync or async), the UI could lighten the parent of the parent span.

[alvarolobato]

This issue is quite old and hasn't come up again. Will revisit if we see the need again.