Faiss: Column-major inputs to brute force computation

[cjnolet]

We are using bruteForceKnn to perform a zero-copy brute-force kNN search using FAISS.

Because the inputs to that that function require row-major arrays, we end up having to do a transpose when cuDF is providing the initial input array.

It would be really useful if we could somehow set a flag on this function to denote that the array we're passing to it is column-major.

[mdouze]

Same request from @glample
I believe this should not be too hard because GpuIndexFlatL2 also supports column-major storage on the database side.

[wickedfoo]

I will work on this.

[wickedfoo]

@cjnolet You want both vectors and queries to be column major? What about outDistances and outIndices as well?

https://github.com/facebookresearch/faiss/blob/master/gpu/GpuDistance.h#L34

[cjnolet]

@wickedfoo , it would be ideal if they were all column-major.

[wickedfoo]

I'll add an option to let you specify the memory layout for each argument independently. At the moment deep inside the internals there is an option for column-major data but not column-major queries. I'll add an option for the queries as well and expose both.

I will natively and efficiently handle column major for the two input data arguments, but the output I'll transpose as needed (the kernels that write the output data I'll only have in row-major form).

[mdouze]

While you are at it, maybe it would be useful to support a stride / lda argument as well. Then we can input all gemm-supported matrix formats to the function (but not all numpy/torch matrix formats which in addition can have non-contiguous elements).

[cjnolet]

This feature will make a huge performance impact in cuML, since we will be able to use FAISS with memory directly from cuDF without the need for a transpose.

Any idea when this feature might be made available in a FAISS release?

[beauby]

It will be available in the next release (but no column-major output AFAIK
– cc @wickedfoo).

On Tue 14 May 2019 at 19:52, Corey J. Nolet [e-mail]
wrote:

This feature will make a huge performance impact in cuML, since we will be
able to use FAISS with memory directly from cuDF without the need for a
transpose.

Any idea when this feature might be made available in a FAISS release?

[wickedfoo]

I added support for row vs. column major for the input matrices which does the tiled matrix multiplication in the proper way, but the final k-selection output kernels are only in row-major form. I won't specialize those kernels for column major since it would have to do all of the ranges of k-values and would tack on a bunch more time in the compile.

If you really want an option for the output to be transposed, I can add it, but I'd be performing the transposition on a region of temporary memory versus you doing that. The size of the output data is usually quite minimal compared to the input data though.

[mdouze]

Supported in 1.5.2, closing.