Osrm-backend: c++ slower then php?

[gabrielozaki]

The test
8 diferents points and making a route using all
using a for loop to repeat the process 5000 times
The osrm file is from my region(i get my country osm and use the osmconvert with a bounding box of my region), all the files have 11 MB

The php code runs in the same machine of osrm-routed to avoid network problems
The c++ code is a variation of the code in example folder

VM
Virtualbox
4x 2.0 ghz
4 Gb ram
10 Gb disk(the host disk are a 500 Gb 7200rpm)

VPS
4x2.3 Ghz
1 Gb ram
10 Gb disk(high speed read and write, probably some raid 0 configuration)

Results
Virtualbox
C++ near of 5/6 minutes
php near of 3/4 minutes

VPS
C++ 1m49.512s
PHP 1m0.938s

Codes

<?php
$lat[0] = "-22.121093"; 
$long[0] = "-51.404997";

$lat[1]     = "-22.140927";
$long[1]    = "-51.400867";

$lat[2]     = "-22.134668";
$long[2]    = "-51.362715";

$lat[3]     = "-22.153748";
$long[3]    = "-51.370611";

$lat[4]     = "-22.153589";
$long[4]    = "-51.391039";

$lat[5]     = "-22.153748";
$long[5]    = "-51.412840";

$lat[6]     = "-22.154225";
$long[6]    = "-51.415071";

$lat[7]     = "-22.123748";
$long[7]    = "-51.212840"; 

//$lat[8]       = "-22.146682";
//$long[8]  = "-51.312715";

for($i=0;i<5000;i++){
   $jsonurl = "http://127.0.0.1:5000/route/v1/driving/$long[0],$lat[0];$long[1],$lat[1];$long[2],$lat[2];$long[3],$lat[3];$long[4],$lat[4];$long[5],$lat[5];$long[6],$lat[6];$long[7],$lat[7]";
$json = file_get_contents($jsonurl);
$array = json_decode($json,true);
}
echo "End"

#include "osrm/route_parameters.hpp"
#include "osrm/table_parameters.hpp"
#include "osrm/nearest_parameters.hpp"
#include "osrm/trip_parameters.hpp"
#include "osrm/match_parameters.hpp"

#include "osrm/coordinate.hpp"
#include "osrm/engine_config.hpp"
#include "osrm/json_container.hpp"

#include "osrm/status.hpp"
#include "osrm/osrm.hpp"

#include <string>
#include <utility>
#include <iostream>
#include <exception>

#include <cstdlib>

int main(int argc, const char *argv[]) try
{
    int i;

    if (argc < 2)
    {
        std::cerr << "Usage: " << argv[0] << " data.osrm\n";
        return EXIT_FAILURE;
    }

    using namespace osrm;

    // Configure based on a .osrm base path, and no datasets in shared mem from osrm-datastore
   EngineConfig config;
    config.storage_config = {argv[1]};
    config.use_shared_memory = false;


    // Routing machine with several services (such as Route, Table, Nearest, Trip, Match)
    OSRM osrm{config};

    // The following shows how to use the Route service; configure this service
    RouteParameters params;

    // Route in monaco
    //0
    params.coordinates.push_back({util::FloatLongitude(-51.404997), util::FloatLatitude(-22.121093)});
//1
    params.coordinates.push_back({util::FloatLongitude(-51.400867), util::FloatLatitude(-22.140927)});
//2
    params.coordinates.push_back({util::FloatLongitude(-51.362715), util::FloatLatitude(-22.134668)});
//3
    params.coordinates.push_back({util::FloatLongitude(-51.370611), util::FloatLatitude(-22.153748)});
//4
    params.coordinates.push_back({util::FloatLongitude(-51.391039), util::FloatLatitude(-22.153589)});
//5
    params.coordinates.push_back({util::FloatLongitude(-51.412840), util::FloatLatitude(-22.153748)});
//6
    params.coordinates.push_back({util::FloatLongitude(-51.415071), util::FloatLatitude(-22.154225)});
//7
    params.coordinates.push_back({util::FloatLongitude(-51.212840), util::FloatLatitude(-22.153748)});

    // Response is in JSON format
    json::Object result;
    // Execute routing request, this does the heavy lifting
    for(i=0;i<5000;i++)
        const auto status = osrm.Route(params, result);

        std::cerr << "END\n";
}
catch (const std::exception &e)
{
    std::cerr << "Error: " << e.what() << std::endl;
    return EXIT_FAILURE;
}

[danpat]

@gabrielozaki There is a startup time in the C++ version - the data file needs to be loaded into memory. 11MB is not a lot and should load fast (<1 second), but your measurements are not equal.

Additionally, what optimization flags did you compile your test code with? If osrm-routed is built with -O3 (the default for a Release build) and your C++ code is built with -O1, then you'll see a significant difference in performance.

[gabrielozaki]

Hi, i have use the O3 on compilation process
with default options :

date && time ./osrm-example.O0 /root/prudente/prudente.osrm && date
Tue May 10 23:48:48 EDT 2016
FIM

real    1m4.636s
user    1m1.410s
sys 0m2.761s
Tue May 10 23:49:53 EDT 2016

with -O3:

date && time ./osrm-example /root/prudente/prudente.osrm && date
Tue May 10 23:47:07 EDT 2016
FIM

real    0m36.413s
user    0m33.393s
sys 0m2.736s
Tue May 10 23:47:44 EDT 2016

PHP code

date && time php teste.php && date
Tue May 10 23:45:27 EDT 2016
real    1m1.304s
user    0m0.917s
sys 0m0.640s
Tue May 10 23:46:29 EDT 2016

Is possible get better results for this test?

[daniel-j-h]

Try

#include <chrono>

...

auto t0 = std::chrono::high_resolution_clock::now();

for (...)
    osrm.Route(...);

auto t1 = std::chrono::high_resolution_clock::now();

std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(t1 - t0).count() << " ms\n";

and compile the example with

cmake .. -DCMAKE_BUILD_TYPE=Release

too.

[gabrielozaki]

@daniel-j-h Hi

The result

./osrm-example /root/prudente/prudente.osrm        
65892 ms

[danpat]

@gabrielozaki It doesn't look like you're doing any requests in parallel.

osrm-routed is multi-threaded, and the OSRM C++ interface is thread-safe, so you could update your code to run many queries in parallel and it would probably be quite a lot faster overall. Each request won't be quicker, but the total runtime should be faster.

[daniel-j-h]

You can do a quick test running requests explicitly in parallel:

#include <tbb/parallel_for.h>
#include <tbb/blocked_range.h>

...

using range = tbb::blocked_range<size_t>;
const constexpr auto batch = 500;

tbb::parallel_for(range{0, 5000, batch}, [&] (const range&) {
  json::Object o;  // you probably can optimize this by caching a thread local result object
  osrm.Route(params, o);
});

and then play with batch to set the min. number of requests per threads.

[gabrielozaki]

Using @daniel-j-h modifications and -O3 optimization flag i get this results

date && time ./osrm-example /root/prudente/prudente.osrm && date
Wed May 11 13:27:11 EDT 2016

real    0m0.773s
user    0m0.154s
sys 0m0.059s
Wed May 11 13:27:11 EDT 2016

This is a really amazing result, thanks @danpat and @daniel-j-h

[danpat]

@gabrielozaki You should check that you're getting a valid response - make sure there is no error code being returned.

While I expect 5000 routes should be a lot faster than you were getting, 700ms seems a little suspicious.

I'd expect roughly 5ms per request. With 4 cores and 5000 requests, that means it should take somewhere in the order of 5 seconds. 700ms seems a bit too fast.

[daniel-j-h]

Yep, checking the result object for error is probably a good idea. Could totally be the case that the optimizer removes all of the code as dead code, since we're not really using it any further.

Maybe accumulate all distances or something like that.

[TheMarex]

I think we are good to close here.