Faiss: 3 arguments asked for add_with_id, but only two shown in wiki

[jukaradayi]

Hi,

I have been using the FAISS library for ~ a year now, in an algorithm I'm working on.
I haven't touched my code since ~november/december 2017, and I tried to reuse it now, after downloading
the latest version of Faiss and I'm running into a problem with IndexIDMap.
I'm using Python.

Here's the part of my code where it fails:

# use FlatIP index with normalization for use cos similarity
self.sub_index = faiss.IndexFlatIP(self.pca_dim)
# apply pca to reduce dimension (to 128 by default)
self.pca_matrix = faiss.PCAMatrix(self.d_fea,
                                              self.pca_dim, 0, True)

self.index_without_id = faiss.IndexPreTransform(self.pca_matrix,
                                                            self.sub_index)
self.index = faiss.IndexIDMap(self.index_without_id)

[...]
ids = np.arange(features.shape[0])
[...]
# compute PCA
self.index.train(all_features)

# add features in chunks to avoid memory pb and to do it faster
n_chunks = np.ceil(float(all_features.shape[0]) / self.chunk_size)

for ch in range(int(n_chunks)):
    n_id = len(all_features[ch * self.chunk_size:min(
                     (ch+1) * self.chunk_size, all_features.shape[0])])

    self.index.add_with_ids(
                    all_features[ch * self.chunk_size:min(
                        (ch+1) * self.chunk_size, all_features.shape[0])],
                    self.ids[ch * self.chunk_size:min(
                        (ch+1) * self.chunk_size, all_features.shape[0])])

This piece of code used to work fine, but now I have an error:

    self.index.train(all_features)
TypeError: train() takes exactly 3 arguments (2 given)

And actually, if I disable the PCA, I have pretty much the same error with add_with_ids:

TypeError: add_with_ids() takes exactly 4 arguments (3 given)

I tried to look at the wiki, but unless I'm missing something, on the pre/post processing page, both methods are called the same way.

I also tried to look at the source code, and in the MetaIndexes.cpp file, in the IndexIDMap implementation, both methods I called indeed have one more argument idx_t n.
Looking at the add_with_ids source code, it seems that this argument might be the total size of the features, but if I pass n=all_features.shape[0] as first argument of train, I end up with this error:

in method 'IndexIDMap_train', argument 3 of type 'float const *'

But all_features is of type <type 'numpy.ndarray'>, with dtype float32.

Did the code for IndexIDMap change ? Am I missing something, should I change the way I handle this index ?

Thanks in advance for your answer !

[jukaradayi]

I'm closing it, turns out I had a mix up in my conda environment, and I there was a conflict between compiled+swig faiss, and conda install faiss-cpu. I created a new env and my problem was resolved.

[jasstionzyf]

@jukaradayi i also had such problem , what do you mean?

[oorobets]

Same problem here! I created fresh conda environment and installed faiss-cpu from pytorch channel. Is there any resolution?