Faiss: how to assign gpu memory manually in python

[naah69]

Summary

Platform

OS: ubuntu 16.04

Faiss version: 1.5.0-GPU

Faiss compilation options:

Running on:

• GPU

Interface:

• Python

Reproduction instructions

how to assign gpu memory manually in python?

i use faiss-gpu in multi-thread.
so i need faiss.StandardGpuResources() every times.
but i found it allocation 1613MiB every times.
up to a few times,it will be errors.
i wanna know how to assign gpu memory manually in python

Faiss assertion 'err == cudaSuccess' failed in faiss::gpu::StackDeviceMemory::Stack::Stack(int, size_t) at utils/StackDeviceMemory.cpp:33

[wickedfoo]

i use faiss-gpu in multi-thread.
how do you use it with multiple threads?

Only a single CPU thread can control a single GPU index instance, or a single sharded/replicated index.
If you call an index from multiple threads, you must handle mutual exclusion yourself.

so i need faiss.StandardGpuResources() every times.
when are you creating one of these objects?

A StandardGpuResources object should be used once per CPU thread. It can be shared among multiple GPU indices but all of those indices must be used within a single thread, so that none of the usage from multiple indices overlap.

[naah69]

i use faiss-gpu in multi-thread.
how do you use it with multiple threads?

Only a single CPU thread can control a single GPU index instance, or a single sharded/replicated index.
If you call an index from multiple threads, you must handle mutual exclusion yourself.

so i need faiss.StandardGpuResources() every times.
when are you creating one of these objects?

A StandardGpuResources object should be used once per CPU thread. It can be shared among multiple GPU indices but all of those indices must be used within a single thread, so that none of the usage from multiple indices overlap.

yeah.i know .
i had found the api named setTempMemory and setPinnedMemory of StandardGpuResources
so how i set them .
is there any demo?

[LiberiFatali]

If you have res = faiss.StandardGpuResources() then

res.noTempMemory() will disable temporary memory on GPU
res.setTempMemory(200 * 1024 * 1024) will allocate 200 MB of temporary memory on GPU

More info in comments:

https://github.com/facebookresearch/faiss/blob/master/gpu/StandardGpuResources.h

[naah69]

ok. i has a new problem.
when i just use setTempMemory and del it after used,i find the gc will not collect whatever manual and the memory(not GPU) will cumulative constantly.

but when i use setPinnedMemory,it will not use memory but virtual memory , and virtual memory will cumulative constantly.

can u tell me what is the meaning of setPinnedMemory and the bad consequence that is leaded by more and more virtual memory

[naah69]

can u tell me that is the more gpu make the less calculate time is right ?

[wickedfoo]

0.5 GiB is reserved for <= 4 GiB memory GPUs,
1.0 GiB is reserved for <= 8 GiB memory GPUs and
1.5 GiB is reserved for all other GPUs by default.

It should not be necessary to change this. Changing setPinnedMemory should not be required either, and in any case, that does not change the memory allocated on GPUs. This memory is used for async CPU <-> GPU data copies.

If you have multiple CPU threads, you should create a single StandardGpuResources object per CPU thread.

If you have multiple GPU indices per each GPU, you should have them be managed from the same CPU thread, such that they can share the same StandardGpuResources object.

The only case where having multiple indices across multiple CPU threads usually makes sense is if each CPU thread is dedicated to a single GPU, as seen in IndexShards and IndexReplicas.

[mdouze]

No activity, closing.