Hello,
first of all: great work. I like this library!
Second: I am running into problems using the IndexFlatIP for cosine similarity search.
The cosine similarity is just the dot product, if I normalize the vectors according to the l2-norm. Cosine similarity is 1, if the vectors are equal and -1 if they point in opposite direction.
Now my problem/question is:
How do I get the values closest to cosine similarity=1, which would mean they are equal.
I tried to search with the normalized negative vector like:
distPos, indexPos = faissPositive.search(-1.0 * findKNNToNormalized, k)
This just gives me the negative values in DECREASING order. For example beginning with -0.58 to -0.66. This is bad, because with the negtive vector I would like to have the ones closest to -1 and it seems to break something internally. Can someone please explain this to me?
Is it searching for the absolute minimum distance? Do I miss a point here?
I hope you get my point, because I would like to search for the cosine distance , which is kind of the inverted cosine similarity.
For IndexFlatIP the values are ordered by decreasing dot product, so you don't need to multiply findKNNToNormalized by -1.
No activity. Closing.
Most helpful comment
For IndexFlatIP the values are ordered by decreasing dot product, so you don't need to multiply
findKNNToNormalizedby -1.