Onednn: which format should i choose for convolution?

[xiechao4]

It seems that convolution primitive only accept weight data format as [oihw].
the weight data format of my model is [ohwi], and i want to add a reorder operation before convolution.
which kind of weight data format should i choose for my data? memory::format::Ohwi8o or memory::format::Ohwi16o?

[rsdubtso]

The best approach is to create a convolution primitive descriptor with 'any' specified as the format and then query it for the format. Please let me know if you need a code sample.

[xiechao4]

when set the primitive description with "any" and query it, it will return the destination data format. As i understand that i need a reorder primitive to reorder the source data format to this destination data format. Now, The problem is that the source data format is [ohwi], i can not find this format in the format definitions, only find Ohwi8o and Ohwi16o. Then what data format i should define for the source data format?

[emfomenk]

Hi @kriszhangyf,

// sorry for the long answer
Indeed MKL-DNN doesn't have ohwi alias for blocked layout...
You have two options to handle that right now:

• (difficult) instead of ohwi fill the memory descriptor your self
• (easy) instead of ohwi use mkldnn_nhwc
• (interesting) add ohwi format in mkl-dnn via RP

The first approach is very correct, but requires the understanding of how memory descriptor works.
Basically it describes dimensions, data type, the layout structure (which is blocked). The blocked layout describes the physical data layout. It has block_dims which describe how we block the dimensions. In you case there is no blocks at all, hence all the values are 1. The strides[0] field describes the strides between dimensions, the strides[1] -- strides within the blocks (which do not exist -- hence no matter what is inside). padding_dims is the physical padding for dimensions. Typically it equals to dims themselves. offset_padding_to_data and offset_padding set to zero.
The complete example:

mkldnn_memory_desc_t ohwi_desc = {
  .primitive_kind = mkldnn_memory,
  .ndims = 4,
  .dims = {oc, ic, kh, kw}, // logical dimensions are always in oihw format
  .data_type = mkldnn_f32,
  .format = mkldnn_blocked,
  .layout_desc.blocking = {
    .block_dims = {1, 1, 1, 1}, // no blocks
    .strides = {
      {kh*kw*ic, 1, kw*ic, ic}, // the strides between dims
      {1, 1, 1, 1}, // doesn't matter, since no blocking happens
    },
    .padding_dims = {oc, ic, kh, kw}, // no phys padding
    .offset_padding_to_data = {0, 0, 0, 0}, // no offset
    .offset_padding = 0, // no initial offset
  }
}

The second approach will do the same for you, using mkldnn_nhwc alias.
The thing is that for MKL-DNN (ideally) the actual name for the physical layout doesn't matter. The only reason we have those layout names is to simplify user's life with describing their layout. So those names just explain the correct permutation between logical (which is always nchw/oihw) and physical layout. E.g.

nchw == {0, 1, 2, 3,} -- identity permutation, logical and physical layout are the same
oihw == {0, 1, 2, 3,} -- the same, but other characters used to help user distinguish  between data and weights
nhwc == {0, 2, 3, 1,} -- n stays at the same place, hence first 0, then h and w goes -- hence 2, 3, finally c comes, hence 1 at the last position
...

So your ohwi is the same as nhwc, since:

• o is logical dim 0 in weights, as well as n in data,
• i is logical dim 1 in weights, as well as c in data,
• h and w are logical dims 2 and 3, as well as h and w in data

Finally, the third approach is the one, that will solve your problem in a very accurate way. If you are interesting in that you may refer to the 53e88ab2ac9f31c3c4a739ad347b07b84b7e30a4 commit that adds new layout aliases.