Faiss: How to set useFloat16LookupTables for GPU IVFPQ

[gyuwankim]

Summary

I tried to use 8 bits per code and 64 sub-quantizers on GPU IVFPQ index.
It results in the following error:
RuntimeError: Error in void faiss::gpu::GpuIndexIVFPQ::verifySettings_() const at gpu/GpuIndexIVFPQ.cu:431: Error: 'requiredSmemSize <= getMaxSharedMemPerBlock(device_)' failed: Device 0 has 49152 bytes of shared memory, while 8 bits per code and 64 sub-quantizers requires 65536 bytes. Consider useFloat16LookupTables and/or reduce parameters.

I worry about the performance degradation coming from reducing parameters.
Therefore, I want to set useFloat16LookupTables to handle this issue.
However, I could not find an example of how to set this option.
Could you elaborate on this?

Thanks in advance!

Platform

OS: Ubuntu 18.04 (nvidia - pytorch docker)

Faiss version: 1.5.3

Faiss compilation options:

Running on:

• [ ] CPU
• [x] GPU

Interface:

• [ ] C++
• [x] Python

Reproduction instructions

dimension = 1024
ncentroids = 4096
code_size = 64
nprobe = 8

quantizer = faiss.IndexFlatL2(dimension)
index = faiss.IndexIVFPQ(quantizer, dimension, ncentroids, code_size, 8)
res = faiss.StandardGpuResources()
index = faiss.index_cpu_to_gpu(res, 0, index)
index.nprobe = nprobe

[mdouze]

co = faiss.GpuClonerOptions()
co. useFloat16LookupTables = True
index = faiss.index_cpu_to_gpu(res, 0, index, co)

[ofirpress]

Hi @mdouze - I've encountered the same error, I added the code you wrote and I still get the error:

RuntimeError: Error in void faiss::gpu::GpuIndexIVFPQ::verifySettings_() const at gpu/GpuIndexIVFPQ.cu:431: Error: 'requiredSmemSize <= getMaxSharedMemPerBlock(device_)' failed: Device 0 has 49152 bytes of shared memory, while 8 bits per code and 64 sub-quantizers requires 65536 bytes. Consider useFloat16LookupTables and/or reduce parameters`

[ofirpress]

I think I figured it out. In order to use useFloat16LookupTables the command is:
co.useFloat16 = True
instead of
co.useFloat16LookupTables = True

[mdouze]

No activity, closing.