Vulkan-docs: Should there be a synchronization pipeline?

[krOoze]

In the Implicit Synchronization Guarantees chapter it is said:

_Action_ and _synchronization commands_ recorded to a command buffer execute the VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT pipeline stage in submission order - forming an implicit execution dependency between this stage in each command.

Based on the above in #552 it was suggested synchronization commands execute VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT.

But:

1) The spec only recognizes graphics, compute, transfer, host, and NVX command processing pipelines. None of them has only those two stages.

2) I get the feeling VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT are supposed to be "fake" stages (i.e. have no operations inside them; useful only to the API because of how srcStage and dstStage is defined). I.e. it is not obvious in which stage the synchronization command would take effect (or how it is divided between those two).

3) If the effect of synchronization command takes full effect only after VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT then that is a problem, because previous synchronization command blocks that stage.

4) The commands before the synchronization command (which it is supposed to synchronize) could already be completed before the VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT is even executed.

This should be cleaned.
I personally would prefer description more akin to state-setting commands. I.e. no stages; only a description based on submission order.

[ghost]

So I'd be ok with being more explicit with how the synchronization commands interact with stages in the pipeline description section, though I'm reluctant to call out a specific pipeline type, as they basically act in-situ with the other pipelines. More detail below:

1. The spec only recognizes graphics, compute, transfer, host, and NVX command processing pipelines. None of them has only those two stages.

Those two stages are explicitly described as applying to all commands. Adding wording saying "synchronization commands execute top and bottom and 'something else'" is probably reasonable.

1. I get the feeling VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT and VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT are supposed to be "fake" stages (i.e. have no operations inside them; useful only to the API because of how srcStage and dstStage is defined). I.e. it is not obvious in which stage the synchronization command would take effect (or how it is divided between those two).

I think you're getting yourself confused somewhat here...
TOP of pipe is "fake" (i.e. doesn't execute anything itself) and runs in submission order, sure. But that is meaningless unless you want to ensure something earlier completes before top of pipe - in which case that would be fine.
BOTTOM_OF_PIPE doesn't "execute" anything either, but also has no implicit order between commands. All this means is that you can wait at a later point in command submission to ensure that all prior commands (including synchronization) have completed.
The work a sync command does is "somewhere between" the TOP_OF_PIPE and BOTTOM_OF_PIPE stages for that command only - it has no bearing on neighbouring commands.

1. If the effect of synchronization command takes full effect only after VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT then that is a problem, because previous synchronization command blocks that stage.

This is never a problem - and in fact is a deliberate choice. If a barrier waits on bottom of pipe, it implicitly waits on all prior pipeline stages anyway, which means that all the previous work is stalled until that stage, ignorant of whether any other synchronization commands are there (which would be fulfilled by the same guarantee). The advantage of this is to be able to do things like wait on prior cache flushes caused by memory barriers in another synchronization command, for instance, to avoid a hazard.

1. The commands before the synchronization command (which it is supposed to synchronize) could already be completed before the VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT is even executed.

I'm not sure I follow what this point is saying - nothing you've said is wrong in this point, I just don't understand the relevance. Do you think this is a problem somehow? The GPU always has to check that the dependency is fulfilled - there's always something that happens.

I personally would prefer description more akin to state-setting commands. I.e. no stages; only a description based on submission order.

I don't agree with this at all. State setting commands don't do anything; they just define state which other commands which do things should abide by and are "consumed" by relevant action commands. The same is not true for synchronization commands - you explicitly set and reset an event, then check its status. Ditto with fences and semaphores. No reason to separate pipeline barriers from this either since they are logically an in-place set and wait. They also perform cache flushes/invalidates which could alone be considered on par with action commands.

[krOoze]

Those two stages are explicitly described as applying to all commands. Adding wording saying "synchronization commands execute top and bottom and 'something else'" is probably reasonable.

That's the less preferrable fix. It digs a hole which is hard to specify yourself out of. It makes a exacerbates the problem in 4. I should elaborate there, because I get the feeling I was not understood on this point.

The work a sync command does is "somewhere between" the TOP_OF_PIPE and BOTTOM_OF_PIPE stages for that command only - it has no bearing on neighbouring commands.

Yeah, that is what I meant by "fake". They must not do anything. It is only needed to be able to express "before first stage" in srcStage and "after last stage" in dstStage because of the specific way those parameters are defined. E.g. if there were modifier bits BEFORE and AFTER (note: just a thought experiment, not a proposition), those two stages would not need to exist; TOP_OF_PIPE would mean something like BEFORE | VK_PIPELINE_STAGE_DRAW_INDIRECT_BIT.

But I digress from the point: So let's consider there is another stage there in between that does all the work of synchronization command.

This is never a problem - and in fact is a deliberate choice. If a barrier waits on bottom of pipe, it implicitly waits on all prior pipeline stages anyway, which means that all the previous work is stalled until that stage, ignorant of whether any other synchronization commands are there (which would be fulfilled by the same guarantee). The advantage of this is to be able to do things like wait on prior cache flushes caused by memory barriers in another synchronization command, for instance, to avoid a hazard.

That's not what I meant. For this issue nr. 3 I assumed all or some part of the synchronization itself is done in the VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT. If there are two synchronization commands, that would mean the second one is executed only after anything and everything else (at which point it is too late for it to have any effect without time traveling).

Introducing hidden VK_PIPELINE_STAGE_SYNCHONIZATION does help here a bit, so let's just skip it.

1. The commands before the synchronization command (which it is supposed to synchronize) could already be completed before the VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT is even executed.

I'm not sure I follow what this point is saying - nothing you've said is wrong in this point, I just don't understand the relevance. Do you think this is a problem somehow? The GPU always has to check that the dependency is fulfilled - there's always something that happens.

This is probably the meat of the issue. It would require million exceptions to work in the pipelined framework.
And introducing VK_PIPELINE_STAGE_SYNCHONIZATION does not help.
It introduces even worse problem:
The commands after the synchronization command could already be completed before the synchronization command even starts.

Per the spec the synchronization commands only "introduce a dependency".
The action commands itself perform the two halves of the dependency in the stated stage or later stage (resp. earlier stage for dstStage). (That's what makes it more like a state setting command)

Consider:

vkCmdDraw1();
vkCmdBarrier(srcStage=VERTEX_SHADER, dstStage=ALL_COMMANDS)
vkCmdDraw2();

• On first step I can only go vkCmdDraw1::TOP_OF_PIPE.
• On second step I can vkCmdBarrier::TOP_OF_PIPE or vkCmdDraw1::WHATEVER_STAGE. Lets assume I have gone all the way to BOTTOM_OF_PIPE in vkCmdDraw1.
• On third step I have to do vkCmdBarrier::TOP_OF_PIPE
• On fourth step I can choose vkCmdBarrier::VK_PIPELINE_STAGE_SYNCHONIZATION or vkCmdDraw2::TOP_OF_PIPE. Let's say I chosen the synchronization on only now the dependency is known.
• Ups I am out of stages for vkCmdDraw1. I have no stage to put synchronization in now

Or let's say instead I have gone

vkCmdDraw1::TOP_OF_PIPE -> vkCmdBarrier::TOP_OF_PIPE -> vkCmdDraw2::TOP_OF_PIPE all the way to vkCmdDraw2::BOTTOM_OF_PIPE -> vkCmdBarrier::VK_PIPELINE_STAGE_SYNCHONIZATION.
-> Ups, vkCmdDraw2 already completed. I have to time travel in order to enforce that dependency I learned of only now.

I don't agree with this at all. State setting commands don't do anything;

Neither really do synchronization commands.
They only "introduce dependency" or otherwisely tell action commands what to do.

The problem is they require knowledge of previous commands in the queue, so they would need a priority in the queue or some other kind of ugly exceptions. Doesn't really seem worth it to me and alternative, more static description based on submission order seems simpler (and mostly already there in the spec).

[NicolBolas]

The commands after the synchronization command could already be completed before the synchronization command even starts.

No, they cannot. If that were to happen, then the synchronization command would be unable to introduce a dependency. And since the specification requires that synchronization commands introduce a dependency, then such an implementation would not be valid.

I think you're really overthinking synchronization here. It's not something that happens in the normal pipeline processing. Like state setting, it is not part of the pipeline.

[ghost]

This is probably the meat of the issue. It would require million exceptions to work in the pipelined framework.
And introducing VK_PIPELINE_STAGE_SYNCHONIZATION does not help.
It introduces even worse problem:
The commands after the synchronization command could already be completed before the synchronization command even starts.

I'm still really not sure why you think there's a problem, though this is an inherently hard thing to talk about I guess. Maybe best if I examine your example and we go from there:

vkCmdDraw1();
vkCmdBarrier(srcStage=VERTEX_SHADER, dstStage=ALL_COMMANDS)
vkCmdDraw2();

So your list of steps is... confusing to say the least. Mostly I think because you're trying to express with a list something that cannot be expressed with a list. So here's a dependency graph showing how that pans out:

Note that the "TOP OF PIPE" dependencies are expressed in this graph and they basically affect... well nothing really.

If you want to have another functional pipeline barrier after the fact, you can, since currently nothing is depending on anything after the VERTEX_SHADER stage for for the first draw. So for instance if I wanted to add a dependency on the fragment stage from both draws, I could do this:

vkCmdDraw1();
vkCmdBarrier(srcStage=VERTEX_SHADER, dstStage=ALL_COMMANDS)
vkCmdDraw2();
vkCmdBarrier(srcStage=FRAGMENT_SHADER, ...)

And here's that in graph form:

Again, TOP_OF_PIPE dependencies don't really do anything here. There's no "running out of stages" to synchronize (though I'm still not clear what you mean by that).

I hope that clarifies some things, but I'm still not convinced I understand your objection...

[ghost]

No, they cannot. If that were to happen, then the synchronization command would be unable to introduce a dependency. And since the specification requires that synchronization commands introduce a dependency, then such an implementation would not be valid.

Ahhhh whoops missed this because I started typing yesterday and didn't hit refresh before finishing it.

I think you're really overthinking synchronization here. It's not something that happens in the normal pipeline processing. Like state setting, it is not part of the pipeline.

I think partly that's his point of why they shouldn't do TOP and BOTTOM. I think the thing that's confusing is trying to think in terms of there being "one" pipeline. Each command executes a set of steps, which are chunked into pipeline stages - except state setting commands. It's a list of tasks which need doing, dependencies happen between those tasks - that's it.

[krOoze]

I think you're really overthinking synchronization here. It's not something that happens in the normal pipeline processing. Like state setting, it is not part of the pipeline.

I think partly that's his point of why they shouldn't do TOP and BOTTOM. I think the thing that's confusing is trying to think in terms of there being "one" pipeline. Each command executes a set of steps, which are chunked into pipeline stages - except state setting commands. It's a list of tasks which need doing, dependencies happen between those tasks - that's it.

Yes, that's my case.
The "overthinking" part is where I tried to paranoidly extrapolate what would happen if synchronization command did happen as part of the pipeline+queue (with all perks and issues), for which I am sorry now for wasting your time on such trivial matter — the thing is simply that I found that sentence in the spec and it seemed out of place (i.e. TMK that the rest of the spec does not seem to treat or need the synchronization commands as a pipeline processed stuff).

[ghost]

for which I am sorry now for wasting your time on such trivial matter

Literally no worries - I'd rather you asked and expressed your confusion than suffering silently! This stuff is really very hard, and whilst the spec is "correct" it can only express one model sanely, and that doesn't always map cleanly to people's pre-conceptions. We just had to find a model which made sense.

the thing is simply that I found that sentence in the spec and it seemed out of place (i.e. TMK that the rest of the spec does not seem to treat or need the synchronization commands as a pipeline processed stuff).

Right so I guess that's the thing, perhaps we do need something like a "synchronization pipeline" expressed for this?

And actually after this discussion I think I'd actually like to make it clearer that the stages are not necessarily stages of hardware, rather that they are distinct steps in a task (which may or may execute on a distinct piece of underlying hardware). I think that's actually what we've tried to express, but I do think that can be clearer. The question is how much cleanup that would be, I'm not up for a major rewrite again :P

[ghost]

NB: I'm still unsure about pipeline barriers - in theory they just define a dependency and don't really need to execute anything anywhere. I need to think about this more. Ditto for WaitEvents...

[krOoze]

@TobiasHector Right. If you wondered, the difference between our two conceptual models is that I considered the "instruction/command fetch" part of the "Barrier" pipeline stage. Before the "Barrier" pipeline stage is executed, the synchronization parameters (dependencies) would not be known, so you would not be able to draw arrows to and from it as you did. And without writing some exceptions you cannot force the stage to be executed. (I think that's a valid interpretation anyway and that's the problem.) I hope this makes it bit clearer what I meant.

You seemed to treat it as half state and half action command. I.e. the dependencies would be settled beforehand when it is recorded or enqueued and the pipeline would only execute some barrier assembly instruction in appropriate place.

[ghost]

@krOoze Yea I think that's not an unreasonable interpretation - though I suspect the model as a whole prevents the situation you were worried about (I'll re-read to double check though).

I think realistically "TOP_OF_PIPE" is kind of meant to be "instruction fetch", but we don't really explicitly say it. We do sort of say it means "the command is queued for execution on the GPU", but not clearly. If you think about it that way, maybe it makes more sense? I haven't entirely thought through the ramifications of being explicit about that yet though - there's no real observable effect to deferring instruction fetch until a barrier has completed, since it doesn't consume or produce anything directly visible to the app.

[ghost]

Ok I've got an MR in flight internally that clarifies this by changing the definition of pipeline stages to mean "discrete steps that a command performs" rather than being "a set of hardware bits that each task flows through". Was a fairly lightweight change thankfully.

I've also specified in the list of pipeline types that synchronization commands execute TOP and BOTTOM, with a note that other operations they perform execute between those two stages, making it obvious that you can synchronize with a synchronization command itself.

[oddhack]

This should be fixed (in master branch) for the 1.1.70 spec release.