Vulkan-docs: Vulkan doesn't support the internal usage of 16-bit floats. At all.

[NicolBolas]

It's odd to call this a bug, but hear me out.

The SPIR-V Float16 capability is not enabled by any Vulkan feature or extension. As such, Vulkan implementations cannot consume it.

VK_AMD_gpu_shader_half_float adds a SPIR-V extension that allows 16-bit float usage for certain functions. But because it _doesn't_ activate general Float16 in SPIR-V as a feature, you can't really use it.

So at the very least, VK_AMD_gpu_shader_half_float ought to also enable the Float16 feature in SPIR-V.

[krOoze]

Well, to be fair SPV_AMD_gpu_shader_half_float seems to be obsolete. The changes made by it seem to already be part of GLSL.std.450 extended instruction set version 1.00.3. So enabling that extension should be noop.

The names of the extensions are bit misleading too... shader_half_float and shader_int16 seem to have been only "bugfix" extensions for extended functions. They seem orthogonal to Float16\Int16 support.

But it is true that Float16 has not been added to Vulkan, so enabling VK_AMD_gpu_shader_half_float is theoretically pointless.

[amdrexu]

With Vulkan 1.1 VK_KHR_16bit_storage and GL_KHX_shader_explicit_arithmetic_types, float16 could be used optionally now. It is true that prior to Vulkan 1.1 and SPIR-V 1.3, float16 type is disabled. So AMD_gpu_shader_half_float is expected to fill this gap and provide developers with such capability like how we have handled with float and double. When the extension SPV_AMD_gpu_shader_half_float string is specified by OpExtension in SPIR-V binary, it is acceptable that float16 could be used as this extension describes and the SPIR-V capability Float16 takes effect accordingly.

My understanding of your questions:
(1) Float16 should not be used even if AMD extension enables it.
Answer: Extension string could change the existing behavior if this is documented in the extension spec. Float16 is valid after Vulkan 1.1.

(2) The extension string is meaningless because it just extended the accepted data type of GLSL.std.450 instructions and now this standard spec adds all of them.
Answer: Prior to the draft of SPV_AMD_gpu_shader_half_float, part of GLSL.std.450 instructions are able to accept float and double types without float16 type. Even if the standard spec is revised now, the extension is still meaningful, written against old revisions of GLSL.std.450.

[krOoze]

With Vulkan 1.1 VK_KHR_16bit_storage and GL_KHX_shader_explicit_arithmetic_types, float16 could be used optionally now.

VK_KHR_16bit_storage only enables addressing 16 bit floats from a storage buffer. Otherwisely it (resp. SPV_KHR_16bit_storage) says:

Other uses of 16-bit types are not enabled by this capability.

The other extension is not published, and besides it is a GLSL extension, so it does not enter the discussion here.

It is true that prior to Vulkan 1.1 and SPIR-V 1.3, float16 type is disabled.

TMK, after that versions that is true as well. More precisely it is enableable cap in SPIR-V. And it is not formally supported by Vulkan (extended or otherwise).

So AMD_gpu_shader_half_float is expected to fill this gap and provide developers with such capability like how we have handled with float and double.

OK, but that is not documented in the spec. It allows passing 16-bit floats to GLSL.std.450 Extended Instruction Set functions. But it does not allow creating such floats in the first place (or it is not written so in the SPV_AMD_gpu_shader_half_float spec).

Answer: Extension string could change the existing behavior if this is documented in the extension spec. Float16 is valid after Vulkan 1.1.

So where is it documented?
In Appendix A: Vulkan Environment for SPIR-V:

The application must not pass a SPIR-V module containing any of the following to vkCreateShaderModule:

• any OpCapability not listed above,

And Float16 is indeed not listed above.

Even if the standard spec is revised now, the extension is still meaningful, written against old revisions of GLSL.std.450.

Theoretically, it does break full compatibility. But to me this is a meaningless distinctinction. Vulkan spec does not even state which Revision is supposed to be supported. Might as well assume it was fixed spec bug and only the newest Revision always applies.

[oscarbg]

Sadly because of this we can’t access 2x peak fp16 flops on Skylake Intel HD GPUs on Vulkan driver.. even driver supports fp16 on D3D 11 & 12 and OpenCL..
Don’t know situation on Nvidia GPUs with fast fp16 like TitanV..
Strange is not standardized with mobile GPUs from Qualcomm and Arm having fast fp16 as shown by his opencl drivers having cL khr fp16..
Standard float16 on Vulkan could be useful for DNN on Vulkan..

[amdrexu]

Thanks for your feedback. So I think there is 2 issues in the spec statements. Please correct me if my understanding is inaccurate.

(1) SPV_AMD_gpu_shader_half_float
Should state that float16 type could be created with OpTypeFloat. And operands with such type can act as source and destination of certain extended instructions defined by GLSL.std.450.

(2) VK_AMD_gpu_shader_half_float
Should state that Float16 capability is allowed to appear in SPIR-V module to enable the handling of float16 operands and corresponding instructions.

[krOoze]

I think I would let (1) be. AIS, it is currently 0 diff extension. Letting it use half for the extended ops, but not the core ops would just complicate matters. It would be better if the Float16 capability is used systematically, and not overriden like this.

Ad (2), right.
Though it would be a bit confusing as similar VK_AMD_gpu_shader_int16 extension does not enable Int16 support. Maybe a new EXT, KHR, or even KHR_maintenance extension including new VkPhysicalDeviceFeatures entry for float16?

[krOoze]

PS: ad (1): Would you be able to dig up why the 16-bit support was added to the original document in Revision 3? Vanilla GLSL does not support it, so it would make more sense being enabled through the extension...

[amdrexu]

For (1), we make an internal discussion today.

• Currently, the GLSL.std.450 spec (revision 5) does support 16-bit floating-point in those extended instructions we expect to change. So the SPV AMD extension spec could remove most modification statements now.
• OpTypeFloat will not depend on Float16 according to SPIR-V spec. It could define any floating-point types without restrictions. So we are free to use this instruction and Float16 capability will not have any interference.

We still have some confusion about your feedback on item (1). Maybe you could explain it in detail and we will evaluate your suggestion once again. Thank you very much.

[krOoze]

So the SPV AMD extension spec could remove most modification statements now.

Right, the InterpolateAt* ops were not changed to support half in the unextended original, so the extension is still needed.
Though that again begs the question why some seemingly randomly chosen ops were changed in the unextended SPIR-V document in Rev 3 (while breaking full compatibility) to support float16, while unextended GLSL lang does not even have half.

OpTypeFloat will not depend on Float16 according to SPIR-V spec.

It's its whole reason for being:

Float16
Uses OpTypeFloat to declare the 16-bit floating-point type.

Spec should use better language, but by common sense it follows that if the capability is not declared, the shader program does not "use OpTypeFloat to declare the 16-bit floating-point type".

PS: Sorry, what you meant by "will"? You plan to change something about this restriction?

[krOoze]

We still have some confusion about your feedback on item (1). Maybe you could explain it in detail and we will evaluate your suggestion once again. Thank you very much.

Let's see...

My understanding of your original point (1) is that you would change SPV_AMD_gpu_shader_half_float to also allow creation of 16-bit OpTypeFloat irregardless of Float16 capability.

It is not clear whether you would allow use of such created 16-bit floats only to the GLSL extended instructions, or to all instructions. But the former solution feels potentially confusing and useless, the later feels unsystematic while there is perfectly fine unextended SPIR-V Cap (Float16) that does the same.

Also begs the question how similar extension VK_AMD_gpu_shader_int16 should work (#715).

Seems like a better solution to keep SPV_AMD_gpu_shader_half_float as is (i.e. it allows the arguments of the three ops to be 16-bit floats) and let Float16 Cap do its job (i.e. allows a 16-bit float variable to be defined).
So, only thing needed would be to allow Float16 Cap containing shader modules in Vulkan (i.e. your point (2)).

[amdrexu]

Thank you for your detailed explanations. We are clear. The spec of VK_AMD_gpu_shader_half_float and VK_AMD_gpu_shader_int16 will be revised to add Float16 and Int16 capability enablement. Also, we will revisit SPV_AMD_gpu_shader_half_float and SPV_AMD_gpu_shader_int16 to get rid of those statements that allow 16-bit data type if latest GLSL.std.450 spec has already relaxed the restrictions.

[krOoze]

The spec of VK_AMD_gpu_shader_half_float and VK_AMD_gpu_shader_int16 will be revised to add Float16 and Int16 capability enablement.

Unlike float16, Vulkan already has VkPhysicalDeviceFeatures::shaderInt16.

[amdrexu]

You are right. We are revising the extension spec , making the statements more appropriate and accurate.

[Tobski]

Thanks all for this input!

As @oscarbg pointed out, fp16 is supported on a lot of Vulkan-capable hardware and can be a substantial perf win for shaders that can use it. The Vulkan WG is working actively on an extension to expose this, though at the moment I can’t give you an ETA. If you’ve got non-obvious requirements or feature requests, we’d be interested to hear them.

NB: Apologies for the late response to this, we had intended to provide an official response to this when it first came up, but for various reasons it got delayed.

[amdrexu]

Sorry for the delay. The spec writer is busy with other urgent tasks during this period of time. He will be available soon and will help to change the contents.

[Cazadorro]

@Tobski What types will be supported once the extension is exposed? I would like all builtin functions to be have fp16 equivalents (sin cos etc), and all current floating point datatype to be able to be converted to fp16 equivalents (ie vec[2-4] and mat[2-4]x[2x4]). Additionally it may be the case that some hardware requires special loads and stores for moving data into the right place to actually use 16bit arithmetic on actual 16bit ALUs, such as with Nvidias Tensor Core units. Can this be rationalized in both GLSL and SPIR-V? Or is this something that should be automatically be handled with the use of the f16mat4x4 type and SPIR-V jit compilation?

And if you are talking feature requests, would it be feasible to get int10 from fp16 and in the same vein int24 from fp32 into glsl? On some hardware (ie not AMD gpus) its faster to use 24 bit integer types via accessing floating point arithmetic on the mantissa, likewise it can also be faster to use int10 on hardware implemented fp16. It may just be more useful overall to support unsigned and signed bounded integers from int32->int2 and uint32->uint2, though 52 bit integers would have the same benefit with double precision hardware. Having such bounded size integers could boost performance, as it appears the glsl spec requires integer and uint types to be of size 32bits.

[amdrexu]

We have proposed two patches of this extension today. Wait for approval and final merging. Thank you.

[Tobski]

@Cazadorro I'll make sure we discuss this in the group, and we'll try to get back to you with a useful answer.

[Tobski]

And if you are talking feature requests, would it be feasible to get int10 from fp16 and in the same vein int24 from fp32 into glsl? On some hardware (ie not AMD gpus) its faster to use 24 bit integer types via accessing floating point arithmetic on the mantissa, likewise it can also be faster to use int10 on hardware implemented fp16. It may just be more useful overall to support unsigned and signed bounded integers from int32->int2 and uint32->uint2, though 52 bit integers would have the same benefit with double precision hardware. Having such bounded size integers could boost performance, as it appears the glsl spec requires integer and uint types to be of size 32bits.

Actually reading this second paragraph, if you'd like to request this, please raise separate issue - it's sufficiently detached from the original request that I don't think we'd accomodate it in the same extension.

[Tobski]

Just to clarify, despite me also working for AMD, @amdrexu and I are talking about different things. @amdrexu is talking about the AMD extensions, whereas I'm talking about working group efforts to standardise. Sorry for any confusion!

[Tobski]

@Cazadorro I really should've known this, but we can't talk about specifics of internal extension development for various non-technical reasons. Suffice to say thanks for your input, and we'll definitely be considering it :)

[Cazadorro]

@Tobski That last reply was in response to my first paragraph correct? Also where do I raise a separate issue? Vulkan-Docs git doesn't seem to be the correct place to do that.

[Tobski]

@Cazadorro Yep it was.

You can raise feature requests either here on Vulkan-Docs or in the ecosystem forum https://github.com/KhronosGroup/Vulkan-Ecosystem

If your request is vendor specific, it might be worth talking to them as well via whatever developer program they have.

[krOoze]

So, this seems to have been fixed in 1.1.79.

[Cazadorro]

@krOoze Wait really? I guess you are right, it looks like this was accomplished here https://github.com/KhronosGroup/Vulkan-Docs/commit/ff9357a4bad5fa49b99b9d3f69d254e2a3f0f575

• Add code:Float16 to the < SPIR-V capabilities>> for the VK_AMD_gpu_shader_half_float extension.

[krOoze]

@Cazadorro Really really, it's a true true story. VK_AMD_gpu_shader_half_float was patched to enable support for Float16 (as it was suspected it already works that way in reality).
They apparently closed the issue in their internal tracker, but not here (though it is beyond me why they keep one; except new extensions and versions, there is assumably nothing top secret going on there...)

[Tobski]

The issue here is still open because, as per my above comments, we're working on a cross-vendor solution.

[alegal-arm]

VK_KHR_shader_float16_int8 should cover the remaining bits of this issue.

[oddhack]

@NicolBolas please close this if you think the original issue has been addressed now.

[NicolBolas]

Yes, this should pretty much cover things.