Drake: RBM/RBS Dynamics performance differences between Matlab and C++

[wxmerkt]

I am currently doing a very large number of calls to .dynamics(t, x, u) and so briefly benchmarked the Matlab and C++ implementations. To my surprise, the Matlab implementation is faster than the C++ one (3.2s vs 5.1s for 1000 calls).

I've uploaded a basic example for both here: https://github.com/wxmerkt/benchmark_drake_rbm_matlab_cpp

I am probably missing something here, would you mind having a look?

Thank you very much, any input or pointers are highly appreciated,
Wolfgang

[sherm1]

There are two calls in the loop -- dynamics and a random configuration generator. I think the dynamics should be identical but I wonder whether different random number generators are being used. If you replace that with something trivial do you still see a difference? (Removing it altogether probably won't work because of caching, but I think you could do something like flip one of the x's from 0 to 1 for each call to defeat the cache.)

[wxmerkt]

Thanks for looking into it @sherm1. Here are some more results in C++:

1. getRandomConfiguration (original as in repository): 5.08688s
2. Replaced the getRandomConfiguration with x0.head(rigid_body_sys.getNumInputs()).setRandom();: 5.01393s.
3. Removed that line entirely, so C++ calls only dynamics: 5.07758s

Or, in other words, there isn't that much difference/influence. So it appears as if the difference is in the dynamics function.

[jwnimmer-tri]

Before going further, I would suggest evicting the "generate some random numbers" from the timing measurements, such that only dynamics is being timed.

[wxmerkt]

@jwnimmer-tri see (3.) in the comment above

[jwnimmer-tri]

Right. I see I was behind in the discussion. Thanks.

[liangfok]

I ran the C++ version of the benchmark on my computer using the following commands:

$ cd ~/dev
$ git clone [email protected]:wxmerkt/benchmark_drake_rbm_matlab_cpp.git
$ cd benchmark_drake_rbm_matlab_cpp
$ PKG_CONFIG_PATH=$HOME/dev/drake-build/install/lib/pkgconfig/:$PKG_CONFIG_PATH make
$ LD_LIBRARY_PATH=$HOME/dev/drake-build/install/lib:$LD_LIBRARY_PATH ./pod-build/bin/benchmark_dynamics
...
Time elapsed: 0.079498
RigidBodySystem has 64 states and 32 inputs
Time elapsed: 1.08254

The latency is much less than 5s. Here's my CPU info:

$ cat /proc/cpuinfo | grep "model name"
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
model name  : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz

Unfortunately, I'm having trouble running the MATLAB version:

$ matlab -nodisplay -nosplash -nodesktop -r "cd /home/liang/dev/drake-distro/drake;run('addpath_drake.m');run('/home/liang/dev/benchmark_drake_rbm_matlab_cpp/benchmark_dynamics.m');"

                                                                                                          < M A T L A B (R) >
                                                                                                Copyright 1984-2016 The MathWorks, Inc.
                                                                                                R2016a (9.0.0.341360) 64-bit (glnxa64)
                                                                                                           February 11, 2016


To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

Error using addpath_drake (line 24)
You must run make first (and/or add your pod build/matlab directory to the matlab path)

Error in run (line 96)
evalin('caller', [script ';']);

It looks like Drake-MATLAB is only compatible with the in-source build method (the version with a pod-build directory). I'll return to trying to get the MATLAB version of this test working later and report back.

[david-german-tri]

It looks like Drake-MATLAB is only compatible with the in-source build method

2599 will fix that.

[jwnimmer-tri]

FYI per #6369 the MATLAB implementation is going to be removed from master, so in order to reproduce this benchmark, we'll have to pull up a prior revision from https://github.com/RobotLocomotion/drake/tree/last_sha_with_original_matlab.

[EricCousineau-TRI]

I don't believe this benchmark is relevant given that RBM / RBS are part of the older MATLAB release; closing for now.