Vulkan-docs: Relaxing queue family index requirements on buffer/image barriers.

[warp-10]

I swear I've seen this request here before, but I'm not able to find it.

From the documentation: srcQueueFamilyIndex and dstQueueFamilyIndex must either both be VK_QUEUE_FAMILY_IGNORED, or both be a valid queue family, which can be found under both VkImageMemoryBarrier and VkBufferMemoryBarrier.

What is wrong with: If either srcQueueFamilyIndex or dstQueueFamilyIndex are VK_QUEUE_FAMILY_IGNORED, then they are both treated as VK_QUEUE_FAMILY_IGNORED by the implementation.?

When collecting these into arrays for automatic scheduling, it is convenient to use the source and destination indices directly instead of having to produce these structures on demand, and only during a submission.

[krOoze]

What is wrong with: "If either srcQueueFamilyIndex or dstQueueFamilyIndex are VK_QUEUE_FAMILY_IGNORED, then they are both treated as VK_QUEUE_FAMILY_IGNORED by the implementation."?

Looks confusing, and requires more complicated test by the driver than simply if(srcQueueFamilyIndex == dstQueueFamilyIndex).

When collecting these into arrays for automatic scheduling, it is convenient to use the source and destination indices directly [...]

What/how exactly is preventing you to do so?

[warp-10]

How is it confusing? I can't reduce the English there any further.

Having to set both to VK_QUEUE_FAMILY_IGNORED means the structures can't be used directly by a scheduler for sorting, then used to determine which command buffers they should be written into. It means duplicating queue indices, and possibly all the same state in the barrier structure, in yet another structure. This is a problem, because vulkan only works with arrays of VkXXX structures, so we'd have to use ANOTHER allocation filled with just these little tracking structures - overhead and all. With that in place, the application gets to shuffle more data around, which means more cache misses, more energy usage, and overall more time spent doing something peripheral to rendering.

Whatever time was saved in the driver for having to omit two subtractions and conditional jumps was taken from the application several fold.

The test on the driver side reduces to: if(VK_QUEUE_FAMILY_IGNORED == srcQueueFamilyIndex || VK_QUEUE_FAMILY_IGNORED == dstQueueFamilyIndex || srcQueueFamilyIndex == dstQueueFamilyIndex). Irrelevant overhead if barriers are scheduled correctly by the application.

[krOoze]

I swear I've seen this request here before, but I'm not able to find it.

It might be your previous Issue #508, where you proposed passing stride along with an array, which would allow passing array of structures extended with user's metadata into the Vulkan API.

Irrelevant overhead

It's not only about the overhead, but also about the driver surface area for errors. Indeed it is not problem making few exceptions, but still it is the Vulkan paradigm. In the lack of other considerations, even this small consideration wins.
(Also as I mentioned, your proposition is slightly more confusing semantically — it is not immediately obvious what transfer from IGNORED to N means without opening the spec).

How is it confusing? I can't reduce the English there any further.

It's not about the english. I simply have trouble imagining what are you trying to do in code, and why it becomes a problem. Stuff like "automatic scheduling" leaves too much to the imagination.

After your clarifications, if I understand correctly, you basically want to make one of the fields ignored, so you can abuse it to store some metadata (e.g. "which command buffers they should be written into") for you, right?

[warp-10]

While designing this API, did anyone consider what it would take to "automate" it? Like a minimal OpenGL-like implementation on top of it?

By "automatic scheduling", I mean something that picks queues automatically - a task is submitted to a scheduler, and requests a queue from a selection of families based on the kind of transaction that needs to occur. The queue requests are satisfied and then the transactor proceeds to fill out a list of barriers for any resources affected. These are fed into a sort of "crossbar" that performs two passes: Sort by srcQueueFamilyIndex and dispatch release operations (if applicable), which conveniently stops at VK_QUEUE_FAMILY_IGNORED. Then, the same for dstQueueFamilyIndex. This can be made particularly efficient if the target command buffers are indexed by their queue family.

From a high level, I don't want to see queues, layouts, or descriptor sets. Queues are a device-specific concept, and don't exist for software rasterizers. Layouts can be inferred automatically, and lazy transitions are easier to predict and manage than sprinkling transition code all over the place. Descriptor sets are defined by shaders. I have a policy of "one-language" definitions, and this very much applies to descriptor sets. Mirroring that in C++ is difficult, requires the yet-to-exist-as-of-this-post reflection facilities to be robust from C++ to shader code, or auto-generating code from shaders (I'm tired, and in a previous edit, I confused descriptor sets for uniform buffers mentally).

Based on the way the rendering is done, descriptor sets don't offer any advantage - basically, it works somewhat like a "parametric display list" where all the "hot loops" over thousands of instances with similar descriptors, etc... are replaced with loop objects (really handles to loop objects) that can be updated outside the display list. This way, only those parts of the command stream that changed as a result of some external modification are updated, leading to a significant improvement in rendering performance, as well as a dramatic reduction of client code complexity.

So, that is a short-ish explanation of what I'm trying to accomplish here. I feel the API could have followed more accessible patterns in certain areas. For example: The select-style loop over fences collected from a set of vkQueueSubmits instead of an IOCP/epoll like interface that would eliminate the need for VkFence altogether (one more handle to allocate and put somewhere). There are really too many places to list here where the concerns of one entry point cut right across what another one is expecting the client application to do.

EDIT: Descriptor sets offer an advantage for the display list compiler, in that well-organized sets tend to loop better. However, the client code has no use for them, and they would just end up adding more handle clutter. The focus of the application is predominantly functional, not object oriented.

BLOG: Most of the problems I have are due to the design of Vk structures. This isn't just creeping into the code, its blowing it all to pieces. There is a very strong emphasis on using arrays, yet they also hold raw pointers to other structures which themselves must reside in arrays. This means having to record an onerous amount of metadata before pre-allocating all the required parameter structures for something as basic as a renderpass, among other gripes, or making very slow code full of pointer-chasing cludgeyness.

[NicolBolas]

How is it confusing? I can't reduce the English there any further.

Just because something is terse does not mean it is easy to understand. And personally, the current specification is much easier to understand.

It also makes a lot more sense from the perspective of someone reading the code. If someone sees that the source is ignored but the destination is not... what are they to think that this means? What is the code trying to do? Did they intend to do a queue family transfer but screwed it up, or did they intend to not perform a queue family transfer?

By making such a thing explicitly incorrect usage, there's no confusion as to what the code is trying to do.

While designing this API, did anyone consider what it would take to "automate" it? Like a minimal OpenGL-like implementation on top of it?

Probably not. Vulkan is a high-performance, low-level API. The basic idea of Vulkan is that you know what your rendering data is going to look like, and given access to some general idea of what the hardware provides, you can build an abstraction that efficiently renders with that particular workload.

Any form of "automation" would by necessity involve discarding potential optimizations in favor of a one-size-fits-all approach. Vulkan is not a good basis for an OpenGL-like API.

Nor is it meant to be.

From a high level, I don't want to see queues, layouts, or descriptor sets.

Nor should your high level code see such things. But your general approach is trying to jump too high level too quickly. You have a low-level API, but your next level up is trying to be too abstracted from the particulars of that API. You need a few levels between the high level of "draw stuff" and the low levels of "queues, layouts, or descriptor sets".

Everything in your design seems very "square-peg-in-round-hole" to me. That the problem is not the API, so much as your approach to the API. You're trying to tell the API what to be, rather than using what the API is to make your rendering fast.

Low-level programmers don't get to decide what those low-levels look like. They build abstractions based on what the low-levels actually are, up to the point of interfacing with code outside of the renderer (the part that says "draw stuff"). You don't start with, "I want the rendering to work like this, so let's look at how to force the low-level system to deal with that." It's "The low-level system looks like this, so I need to design a renderer that optimally executes given those low-levels".

For example:

a task is submitted to a scheduler, and requests a queue from a selection of families based on the kind of transaction that needs to occur.

Why is this task requesting a queue to work on? Presumably, this task performs a well-defined operation. Since its operation is well defined, what queue family it will use for its commands ought to be obvious based on what that operation is. If the task is doing a graphics operation, you pick the graphics queue. If it's a compute operation, you may pick the compute queue (if available) or the graphics queue. And so forth.

These kinds of selections can be made at the time the Vulkan device is created. So for each particular kind of task, you should be able to know up front what queue family it's going to execute on. This is not the sort of thing that should be selected at task scheduling time.

[warp-10]

Vulkan is a high-performance, low-level API

That isn't an excuse for leaving out considerations for more complex and demanding use cases. Benchmarks are not useful software. They sell hardware to enthusiasts, but deferring to a marketing slogan when reality demands something different will not magically alleviate that difficulty. We need more precise control over resources (mainly where they are allocated), and again control over how many disparate tasks may be completed in parallel. This cannot be done with OpenGL. It would have been nice to have an "in-between" that doesn't expect developers to do the vendor's jobs for them.

...and given access to some general idea of what the hardware provides, you can build an abstraction that efficiently renders with that particular workload.

Which contradicts your previous statement, emphasis mine. This mess I'm talking about is the "general idea".

Why is this task requesting a queue to work on? Presumably, this task performs a well-defined operation.

I just explained in the paragraph that follows. I'm providing a valid use-case due to the requirements of our application, not some fiction invented to provoke that question.

These tasks, or transactions, are well-defined. They are actually collections of well-defined operations for which the composition thereof is not known until run-time. Resource ownership and layout transitions can be optimized if the manner in which they are used is known before a transaction is "compiled", which is the case, but since these tasks require more than one queue family on average, and need to be able to complete in parallel on different threads, it isn't reasonable to serialize them all into a small selection of command buffers. There is always a small amount of work to be done before submitting a command stream, and it is not reasonable to stall the client thread while this happens.

This need for a "pool of queues" is due to external synchronization required on all queues. If that weren't required, only one queue from each family would be allocated. It can still cope with exactly one queue, should that ever be the case, as is required of scalable software.

These kinds of selections can be made at the time the Vulkan device is created. So for each particular kind of task, you should be able to know up front what queue family it's going to execute on. This is not the sort of thing that should be selected at task scheduling time.

See the above.

MORE BLOG: This brings me back to why using structures in a raw C API is a very bad idea. While the binary layouts of structs have settled into whatever the relevant hardware requires, it imposes a very rigid and fragile ABI on both the client and implementation. We're already passing in allocator callbacks, so why are we filling out arrays of glorified parameter lists? The implementation could be granted more flexibility by using separate entry points to create and define related parameters of internal structures before submitting an "initialization object" to a constructor. I predict every driver engineer tasked with developing a backend for Vulkan will invariably lament the inflexibility and various inconveniences brought about by the use of structures and pointers thereto within a userland-facing API - the implementation now has even more pointers to prod with access tests, lest a simple, easily overlooked application mistake could crash the system. There is no room for an access violation in kernel mode. If the implementation could work with its own internal structures directly, needing to validate only once per user-supplied allocation (through that otherwise seldom-used set of application provided allocator callbacks), I imagine it would be easier to ensure correct and efficient behavior.

[Novum]

The Vulkan part of the driver does not usually live in the kernel. At least it does not on desktop Linux or Windows implementations.

[warp-10]

Great! Nice to know we get to waste memory keeping thousands of VkGypsyWagon around.

It was exciting to think we could bypass unpredictably high-latency driver cruft. OpenGL is so far detached from the hardware that it was is impossible to get the typical ICD to perform consistently. So now, the only other option we have is designed to be used by hand, over and over again from the ground up, and only for disposable applications or those with very limited functionality. Sure, everyone just writes benchmarks or game engines. Who cares about simulation or visialization? Who cares about flexibility? Content creation? Nobody?

There was VERY strong emphasis this would ultimately evolve into something appropriate for general-purpose use.

[Novum]

I think the Vulkan API is very well designed & works great in practice. I still have this opinion after shipping two AAA titles with it.

[warp-10]

So, my arguments have been:

• OpenGL is an ageing and unpredictable framework that doesn't meet our requirements when handling very large datasets. It has thread affinity, and the typical ICD ends up serializing multiple threads accross every entry point (or just resource mutations), which is a huge performance issue. We need a more robust option. Even single-threaded API calls take too long for the "deferred command buffer" idea to be efficient.

• We can have greater flexibility by using modern (circa 2004?) concurrent programming techniques. Completion-port-esque task engines are not hard to implement or use.

• We need guarantees about the POD-ness of device-local objects for defragmentation. No such guarantee exists, there is no way to ask the driver, so defragmentation is forced to employ a very inefficient, failure-prone strategy in the general case. Fine, we'll just defer to an ever-growing database of devices and driver versions to know on which ones "memory is just memory".

• We need a way to know about the performance of certain operations within queue families up-front. Too bad device databases are not maintainable, and that our software needs to survive longer than one generation of hardware and run reliably on large clusters of diverse equipment for weeks at a time while also producing visualizable snapshots at regular intervals.

• Games are nice, but don't adequately reflect the general case. Game/benchmark == Delightful Rube Goldberg machine of profitable and entertaining novelty. I would know this; I've been taking a professional hiatus from that industry for the past 2 years.

And I always get:

• Its great for games.

• We're too far invested in this now, so nothing will ever need to change.

• That's just the way the hardware works. Stick it where the sun doesn't shine.

• Just employ best practises (whatever that means when literally next to nothing can be known about the hardware from the API).

• Just use OpenGL because tl;dr and its just like, your opinion, man.

• -- miffed or confused cricket noises --

Yep. I'm just doing this to antagonize and bewilder; not trying to provide input, or anything.

[Novum]

The Vulkan API queues map pretty closely to how the hardware works, because it's intended to be low level. All you said can be done by putting abstractions on top of it.

It's not the scope of Vulkan to provide a convenient programming interface. It's purpose is to get to the lowest level interface to the common denominator of available hardware.

I also don't see how games differ from other applications in this regard.

As for your request in this thread: The requirement to specify the queue family for the barrier is for performance reasons. If you use resources with concurrent access, this will disable compression on at least AMD hardware. In this context it makes no sense to give only one index for non-concurrent resources. The driver does not know how to de-compress or re-compress a resource with just one of them (e.g. the GFX queue can use compressed resources, but the DMA queue cannot).

You can use concurrent access on all your resources and then just always pass VK_QUEUE_FAMILY_IGNORED. Either that or you have to track & manage the queues, because the driver is not going to do that for you.

Note that this is also not an arbitrary restriction put into the API. There are real world reasons for it.

[NicolBolas]

And I always get:

• Its great for games.
• We're too far invested in this now, so nothing will ever need to change.
• That's just the way the hardware works. Stick it where the sun doesn't shine.
• Just employ best practises (whatever that means when literally next to nothing can be known about the hardware from the API).
• Just use OpenGL because tl;dr and its just like, your opinion, man.
• -- miffed or confused cricket noises --

This list is highly reductive and does not accurately represent the actual position of anyone who has responded to you.

In particular, I'd like to confront the following strawmen:

We're too far invested in this now, so nothing will ever need to change.

I assume you file under this any arguments of the form "that's how Vulkan is _supposed_ to be." Well, that's not what those arguments mean at all. They mean exactly what I said: that's how Vulkan is supposed to be.

It's not a matter of being "too invested" in anything; it's a matter of having the right tool for the right job.

Vulkan is a low-level, explicit API. That is its job. And however much you may not like it, appreciate it, or realize it, a lot of people _need_ a low-level, explicit API. Vulkan is a good tool for its purpose, and the things you want to change about it deviate _sharply_ from its purpose.

Just consider something like automating layout transitions. To do this, command buffer submission would be much more complicated and take longer to work. For each CB in a batch, the driver would have to figure out which images are being used within that batch, then insert layout transition commands into the existing commands in the CB to make them those work. It has to do this because it doesn't know until submission time what the _previous_ layout is, and transitions often require that previous state.

By contrast, if the user provides layout transition info, none of that has to happen. A CB submission can just be a memcpy directly into the queue's command stream, possibly with some pointer-fixup here and there. For one-time, non-simultaneous CBs, the memcpy may not even be needed; just point the queue's command stream to the CB's storage and tell it to execute that data.

That is the power of a low-level, explicit API: a command buffer is not some construct that requires further processing of an indeterminate nature. It simply converts commands into implementation-defined tokens, ready to be used directly in the command queue.

Many people have applications where managing layout transitions on their own is not hard. Others can redesign their programs to make such management manageable without too much pain/thread synchronization.

And this is the simplest case. If you want to add automatic queue management for memory allocations, that's even bigger issue, since it involves synchronization between things happening on other threads. It's certainly doable, but it's an expensive process that requires keeping around a bunch of information.

I'm not attacking your particular use case, but you have to see that what you want is not free. This is not a matter of what we're "too far invested" in; it's a matter of us not wanting to pay a cost that's easy enough for us to bear, just because it's not easy for _you_ to bear.

Just employ best practises (whatever that means when literally next to nothing can be known about the hardware from the API).

What "best practises[sic]" means has nothing to do with exactly how much hardware information you can divine from Vulkan. It has to do with how you interact with Vulkan itself, that you don't design your application without looking at how the API works first.

Consider what you said about tasks:

Resource ownership and layout transitions can be optimized if the manner in which they are used is known before a transaction is "compiled", which is the case, but since these tasks require more than one queue family on average, and need to be able to complete in parallel on different threads, it isn't reasonable to serialize them all into a small selection of command buffers.

Let me see if I can translate this into something more specific:

Your rendering system has recognized the need to do a DMA into a number of images, so you're going to create a task for that. But those images are also going to be rendered with in a separate task. These are two separate tasks that will execute on two separate threads, and they may well be submitted on two separate queues (hardware willing).

Does this adequately describe the circumstance you're talking about?

If so, then the answer involves a degree of complexity on your end. The task that builds the DMA CB also needs to build a CB that takes those images and transfers ownership back to the main graphics queue you intend to use them with (since the CB that does the DMA will transfer ownership to itself). That CB will be executed on the same queue as the one you're rendering to, and it should have a wait semaphore on the queue doing the DMA.

Of course, you only need to do this if the DMA queue isn't the same as the eventual rendering queue.

I bring this up because you later say:

This need for a "pool of queues" is due to external synchronization required on all queues. If that weren't required, only one queue from each family would be allocated. It can still cope with exactly one queue, should that ever be the case, as is required of scalable software.

Well, that sounds very much like each task is submitting the CBs it generates. That's a _hugely_ bad idea.

There should be tasks that generate CBs bound for a particular queue, and a task for each active queue, which will take the CBs bound for that queue and submit them all at once. The submission task should only be invoked at the competition of all of the CB tasks that went into generating it.

External synchronization requirements should not be what makes you use a "pool of queues". Multiple queues from the same familiy is primarily about being able to execute more stuff; you shouldn't use multiple queues to avoid accessing the same queue from multiple threads.

You should instead design your application so that it never _wants to_ access the same queue from multiple threads. That's what "best practices" mean.

---

As for your arguments:

Great! Nice to know we get to waste memory keeping thousands of VkGypsyWagon around.

... huh? The basic Vulkan state data structures (the ones you provide to create objects or set up state) don't have to continue to exist past the execution of the command that provides them. There are very few places where this is not the case.

So it's not clear what "thousands of VkGypsyWagon"s you're talking about having to waste memory on.

We can have greater flexibility by using modern (circa 2004?) concurrent programming techniques. Completion-port-esque task engines are not hard to implement or use.

Vulkan is very favorable to modern concurrent programming techniques. However, it is hostile to improperly designed concurrent programming techniques. That is, if you do things the wrong way, you can easily trap yourself into a problematic design.

So long as you pay attention to Vulkan's needs before designing your concurrent renderer, you should be fine.

We need guarantees about the POD-ness of device-local objects for defragmentation. No such guarantee exists, there is no way to ask the driver, so defragmentation is forced to employ a very inefficient, failure-prone strategy in the general case. Fine, we'll just defer to an ever-growing database of devices and driver versions to know on which ones "memory is just memory".

If you're talking about unilaterally memcpying implementation allocations to other locations in memory, being POD is _insufficient_ as a requirement. Any struct which points to another struct can be POD in terms of the C++ definition, but that doesn't mean you can just move the object it points to around without changing the pointers. And you can't tell where those pointers are without the driver effectively describing its data structures to you.

You can't just walk memory and try to deduce if a particular 4-8-byte sequence is a pointer, after all. It might be a value that just so happens to have the same numerical value as a piece of CPU memory.

And if you're not talking about that, I don't know how you intend to defragment "device-local objects".

We need a way to know about the performance of certain operations within queue families up-front.

How can performance characteristics be effectively communicated in a way that is meaningful across a variety of platforms?

---

Yep. I'm just doing this to antagonize and bewilder; not trying to provide input, or anything.

Nobody thinks you're trying to "antagonize and bewilder". But you seem to be in a place where either you want Vulkan to be something it isn't, you've trying to run uphill (aka: your code design doesn't work the way Vulkan wants to), or you're trying to fit a square peg (Vulkan) into a round hole (what you're application needs to do).

Vulkan is not supposed to be all things for all people. It serves a particular need and it serves that need very well. You said "It would have been nice to have an 'in-between' that doesn't expect developers to do the vendor's jobs for them." That's fine, but expecting Vulkan to be that "in-between" makes no sense. It was never intended to be that kind of thing.

It is low-level and explicit. These are fundamental design goals of the API. Your input seems to be "I don't want the responsibilities that come with low-level, explicit power." That's fine but... that doesn't mean that other people don't want that power.

Also, there really can't be an "in-between" that automatically handles this stuff without having to build something like OpenGL. Oh sure, there's some room between OpenGL and Vulkan, but you'd be surprised how quickly "automatic" things build up to create something that retains 80% of OpenGL's flaws.

For example, consider layout transitions, queue memory management, and performance consistency work. If all of these are manual, then how long it takes to perform a logical submission of a frame's work is entirely in the hands of the user. How many semaphores are needed, how many layout transitions, etc, it's all in your hands.

If they're automatic, that means that vkQueueSubmit now becomes a lot less determinate in how long it takes. One CB may use very few images, while another has a lot that have to be transitioned. And so forth. Some frames may need more semaphores than others. And there's no way to know when a tight frame happens.

And since you don't know when it might happen, it's a lot harder to _mitigate_ such circumstances. For example, one application might try to avoid stalls due to DMAs by simply not rendering any object until its images have been properly uploaded. Or maybe it'll switch to lower resolution versions of those images. In both cases, the application prevents the stall, not by waiting on a semaphore, but by changing how it renders.

[krOoze]

~~ miffed or confused cricket noises ~~

We should probably discuss in the forums. The OP proposition was quite simple in scope and I don't think it warrants this wall of text.

On the original topic, I am still not quite clear why setting both srcQueueFamilyIndex and dstQueueFamilyIndex to VK_QUEUE_FAMILY_IGNORED or both to the current queue family does not suffice for the purpose. Even if you do want to store the resource's current queue family index for your own purposes, the second option does allow that.

And so I can sort (or do whatever) with the index, and I can detect transition by srcQueueFamilyIndex != dstQueueFamilyIndex. And I don't need to introduce any new hackery to the driver.

[warp-10]

I'm creating a thread in the forum. I'll post the link shortly.

[warp-10]

Forum link: https://forums.khronos.org/showthread.php/13653-Contiuation-of-Github-650?p=42982#post42982

[oddhack]

@warp-10 thanks for the suggestion, and the forum post. As several people also said in the issue comments, Vulkan is consciously targeted at an explicit and very low level. Khronos believes that layers or helper APIs are an appropriate way to achieve the convenience you desire. We are closing this issue, since we do not expect to include functionality like this in Vulkan in the forseeable future.