Faiss: Got an error when training index

[luochenxi]

Hi,
@beauby @mdouze
This issue#712 did not solve the problem I encountered.

Platform

OS: ubuntu:16.04
ENV:

    package                    |            build
    ---------------------------|-----------------
    blas-1.0                   |              mkl           6 KB
    faiss-cpu-1.5.0            |           py27_1         850 KB  pytorch
    intel-openmp-2019.3        |              199         886 KB
    libgfortran-ng-7.3.0       |       hdf63c60_0         1.3 MB
    mkl-2019.3                 |              199       203.3 MB
    mkl_fft-1.0.10             |   py27ha843d7b_0         161 KB
    mkl_random-1.0.2           |   py27hd81dba3_0         383 KB
    numpy-1.16.2               |   py27h7e9f1db_0          49 KB
    numpy-base-1.16.2          |   py27hde5b4d6_0         4.3 MB
    ------------------------------------------------------------

Running on:

• [x] CPU
• [ ] GPU

Interface:

• [ ] C++
• [x] Python

Dockerfile:

FROM ubuntu:16.04
MAINTAINER luochenxi

ARG FAISS_CPU_OR_GPU=cpu
ARG FAISS_VERSION=1.5.0

RUN apt-get update && \
    apt-get install -y curl bzip2  && \
    curl https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh > /tmp/conda.sh && \
    bash /tmp/conda.sh -b -p /opt/conda && \
    /opt/conda/bin/conda update -n base conda && \
    /opt/conda/bin/conda install -y -c pytorch faiss-${FAISS_CPU_OR_GPU}=${FAISS_VERSION} && \
    apt-get remove -y --auto-remove curl bzip2 && \
    apt-get clean && \
    rm -fr /tmp/conda.sh

ENV PATH="/opt/conda/bin:${PATH}"

instructions

I used the following code to train an index:

m = 16                                   # number of subquantizers
n_bits = 8                               # bits allocated per subquantizer
pq = faiss.IndexPQ (d, m, n_bits)        # Create the index
pq.train (x_base)                       # Training
pq.add (x_base)                          # Populate the index
D, I = pq.search (x_query, k)            # Perform a search

However, the error occurred:

Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so.

An optional solution I tried is:#712
However, the same error still happen.

What can I do for this?
Thank you very much

[sankovalev]

+1
the same garbage, none of the solutions found did not help

@luochenxi
UPD:

import mkl
mkl.get_max_threads()

before all imports worked for me

[tfriedel]

+1
the same garbage, none of the solutions found did not help

@luochenxi
UPD:

import mkl
mkl.get_max_threads()

before all imports worked for me

same problem, fix worked for me. I don't understand why this works. Guess there's two versions of the same library and I pin the correct one down by running this command. This is disappointingly ugly.

[luochenxi]

+1
the same garbage, none of the solutions found did not help

@luochenxi
UPD:

import mkl
mkl.get_max_threads()

before all imports worked for me

Thank you very much. fix worked for me. I don't understand why this works,too.

[DogeWatch]

But when I import mkl, there were a segmentation fault (core dumped) happened.

[beauby]

Could you try with faiss-cpu version 1.5.1?

[UpCoder]

@beauby yes, I try. python2.7, faiss-cpu 1.5.1. Still meet the problem. and
import mkl mkl.get_max_threads()
help me to solve the problem.