Incubator-mxnet: How to run mxnet(C++) in single-thread mode?

[xianyujie]

Hey, I want to run mxnet in single-thread mode, but I find my programs have many threads when using mxnet.

Using mxnet

ubuntu@ubuntu-MW50-SV0:~$ top -H -b -n1 | grep validate11 | wc -l
33

Without mxnet

ubuntu@ubuntu-MW50-SV0:~$ top -H -b -n1 | grep validate11 | wc -l
2

So, when I compile mxnet, if there are some options to prevent multi-threads?

[xianyujie]

@pengzhao-intel do you have some tips? many thanks.

[xianyujie]

I want to take part in the FRVT competition. Here are some requirements:

Implementations must run in single-threaded mode, because NIST will parallelize the test 
by dividing the workload across many cores and many machines. Implementations must ensure 
that there are no issues with their software being parallelized via the fork() function. 
Developers should take caution with checking threading when using third-party frameworks 
(e.g., TensorFlow, MXNet, etc.).

[pengzhao-intel]

Could you try below setting?

export KMP_AFFINITY=granularity=fine,noduplicates,compact,1,0
export OMP_NUM_THREADS=1
numactl --physcpubind=0 --membind=0 python ... 

[xianyujie]

@pengzhao-intel I tried, and it's still multi-threads.
By recompiling mxnet with OPENMP=0, the number of threads reduced from 33 to 11. It does not completely prohibit multi-threads.

[pengzhao-intel]

Could you help provide the reproducible case?

BTW, please try naive engine :)
export MXNET_ENGINE_TYPE=NaiveEngine

[leleamol]

@mxnet-label-bot add [Question, Pending Requester Info]

[ElaineBao]

Hi @xianyujie, I tried the solution mentioned in your email, and confirm that with:

compile mxnet with OPENMP=0
export MXNET_ENGINE_TYPE=NaiveEngine

mxnet is running with minimum number of threads.

[xianyujie]

@ElaineBao yes, it works, but sometimes the number of threads increased, I'm retesting if it was an error caused by my test.

[xianyujie]

compile mxnet with OPENMP=0 
export OMP_NUM_THREADS=1
export MXNET_ENGINE_TYPE=NaiveEngine

solve it! Other environment variables are as follows:
http://mxnet.incubator.apache.org/versions/master/faq/env_var.html?highlight=naive
thanks @pengzhao-intel

[pengzhao-intel]

compile mxnet with OPENMP=0 
export OMP_NUM_THREADS=1
export MXNET_ENGINE_TYPE=NaiveEngine

solve it! Other environment variables are as follows:
http://mxnet.incubator.apache.org/versions/master/faq/env_var.html?highlight=naive
thanks @pengzhao-intel

Cool!

Thanks, @ElaineBao as well, and I am going to close this issue :)
Feel free to let us know if you encounter any issue in the future.

[pengzhao-intel]

Resolved and closing now.

[igor-byel]

compile mxnet with OPENMP=0 
export OMP_NUM_THREADS=1
export MXNET_ENGINE_TYPE=NaiveEngine

solve it! Other environment variables are as follows:
http://mxnet.incubator.apache.org/versions/master/faq/env_var.html?highlight=naive
thanks @pengzhao-intel

we used such solution also for frvt
setting those variable in code

std::shared_ptr<Interface>
Interface::getImplementation() {
//set environemnt variable for mxnet to run single thread
    try {
        char mxnetConfig1[] = "MXNET_ENGINE_TYPE=NaiveEngine";
        putenv(mxnetConfig1);

        char mxnetConfig2[] = "OMP_NUM_THREADS=1";
        putenv(mxnetConfig2);
    }
    catch (const std::exception& e)
    {

    }

it worked for frvt test on our local machine
but when we send it to frvt for check they respond that our lib is threading.
Did you succeed with mxnet configuration?
Thank you.

[cyrusbehr]

@igor-byel I experienced the same issue you are facing, and here is my fix. The issue is likely due to the way you are using the putenv function. From the linux manual:

The putenv() function adds or changes the value of environment
variables. The argument string is of the form name=value. If name
does not already exist in the environment, then string is added to
the environment. If name does exist, then the value of name in the
environment is changed to value. The string pointed to by string
becomes part of the environment, so altering the string changes the
environment.

In your above implementation, your mxnetConfig1 and mxnetConfig2 variables are no longer defined once they go out of scope (which is at the end of the getImplementation function). You therefore have a dangling pointer in your environment. Make mxnetConfig1 mxnetConfig2 global vars, member vars, or allocate on heap for the changes to persist.

For more info, refer to this stack overflow question.

[jaisanant0]

@cyrusbehr

as you mentioned i have moved the variables mxnetConfig1 mxnetConfig2 into class member but the number of threads spawned is qual to number of cores.
Any solution

[cyrusbehr]

@jaisanant0 As a sanity check, can you export those environment variables directly into your shell where you are running your program (export them before running the program). That way we can isolate the problem.

[jaisanant0]

@cyrusbehr
I did export omp_num_threads = 1 is working.
Could you please tell me how to do in shared library as required by NIST?

[AyushP123]

It might be an OpenBlas issue instead of OpenMP (Setting OMP_NUM_THREADS also sets OPENBLAS_NUM_THREADS):
Relevant links:

1. https://github.com/apache/incubator-mxnet/issues/16961
2. https://github.com/bgeneto/build-install-compile-openblas