Onednn: How does int8 winograd convolution use avx512_vnni instructions?

[Gxllii]

I found that u8s8s32x_wino_convolution has been supported in oneDNN. The operands need to be u8 and s8 in the VPDPBUSD vnni instruction according to the documentation. However, the transformed input (activation) in Winograd convolution is not guaranteed to be non-negative values. How does oneDNN resolve and implement it? Thanks.

[kwiersch]

Hi @Gxllii ,

I believe that during the src transform, the potentially signed values are shifted by 128 (see this line in the src transform kernel). But that changes the meaning of the resulting GeMM from tsrc * twei to (tsrc + 128) * twei, so to get back the intended result the extra factor of 128 * twei must removed (this is typically called the "compensation" and is calculated at this line during the weights reorder).

Thanks,
Keola

[Gxllii]

Thanks! @kwiersch.

[Gxllii]

Hi @kwiersch,
I'm a little confused about the weights reorder operation. Was it done before int8 GEMM?

[emfomenk]

Yes, a reorder happens before executing the convolution.
It applies the winograd transform (G_2x2_3x3) and quantizes the weights (see this line).

[Gxllii]

So the result of gemm is initial by -128 * twei?

[emfomenk]

If I read the code correctly (@kwiersch please correct me if I am wrong) the registers are initialized with -128 * twei (in the pointed code reg_aux_dst_b contains the address of these values) from the very beginning, and then gemm adds the result of (tsrc + 128) * twei to them.