Faiss: Support `float16` data in numpy

[mdouze]

I have the same problem when using CPU version. I am building a float16 numpy array with the data I want to index and then when I am trying to use flat index I get an exception:

index = faiss.IndexFlatL2(dimensionality)
index.add(data)  # data is of dtype('float16')

The last command raises the following exception:

     98         n, d = x.shape
     99         assert d == self.d
--> 100         self.add_c(n, swig_ptr(x))
    101 
    102     def replacement_add_with_ids(self, x, ids):

ValueError: did not recognize array type

Update: If I cast array to 'float32' there is no exception!

_Originally posted by @alup in https://github.com/facebookresearch/faiss/issues/590#issuecomment-449333404_

[mdouze]

To be honest I was not aware that numpy had a float16 dtype.
However, float16 arrays are supported internally on CPU by IndexScalarQuantizer.

Unfortunately the add and search methods take float*s in C++ so it is hard to interface as is.

So I will make a feature request of this.

[beauby]

We decided against doing implicit conversions inside Faiss, and we do not plan to support other python datatypes than float32 for the time being. As @mdouze mentioned, if you want your vectors to be stored as float16 within the index, it is already supported with IndexScalarQuantizer on CPU, and in the regular flat GPU index, but the interface remains float32.