Osrm-backend: How to improve osrm-backend docker performance.

[xydinesh]

I'm using Georgia (US State) from geofabrik, http://download.geofabrik.de/north-america/us/georgia-latest.osm.pbf (172MB) and car profile from OSRM v5.12.0. I ran osrm-extract and osrm-contract on ec2 instances (m4.4xlarge) and using a docker instance in the same EC2 instance.

Run time from EC2 instance:

osrm-extract: 34sec
osrm-contract: 2min

Run time from docker

osrm-extract: 2min
osrm-contract: 136 min

I'm running another test with docker with stxxl

echo disk=/data/stxxl,20G,syscall > .stxxl
time docker run -v $PWD:/data -e STXXLCFG=/data/.stxxl -ti osrm/osrm-backend:v5.12.0 /usr/local/bin/osrm-contract /data/georgia-latest.osrm

osrm-contract: 50 min

Do you have any suggestion on how to improve docker instance performance ?

[xydinesh]

I had better luck with ubuntu:16.04 based docker image. Contraction finished about 5min. I would still love to figure out how to enhance alpine linux performance.

time docker run -v $PWD:/data -ti xydinesh/osrm /usr/local/bin/osrm-contract /data/georgia-latest.osrm

...
[info] Getting edges of minimized graph . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Contracted graph has 10597751 edges.
[info] Contraction took 305.437 sec
[info] Preprocessing : 305.83 seconds
[info] finished preprocessing
[info] RAM: peak bytes used: 841830400

Dockerfile

FROM ubuntu:16.04
RUN apt-get update && apt-get -y install build-essential git cmake pkg-config \
libbz2-dev libstxxl-dev libstxxl1v5 libxml2-dev \
libzip-dev libboost-all-dev lua5.2 liblua5.2-dev libtbb-dev wget
RUN mkdir -p /src && mkdir -p /opt
COPY . /src
WORKDIR /src
RUN mkdir -p build
WORKDIR build
RUN cmake .. -DCMAKE_BUILD_TYPE=Release
RUN cmake --build .
RUN cmake --build . --target install

[daniel-j-h]

Yeah, we had performance problems with Alpine before - but given that these Docker images should be reasonable defaults only (e.g. without stxxl) we were fine with it.

There's no reason not to switch to an Ubuntu 16.04 base image other than image size.

Feel free to switch it over and make a pull request for

https://github.com/Project-OSRM/osrm-backend/blob/master/docker/Dockerfile

and check performance / image size for changes.

[daniel-j-h]

Closing as not actionable. Feel free to change to Ubuntu and send us a pull request if you have the time for it.

[stvno]

It might be nice to add a footnote to the 'Using Docker' paragraph that the docker image comes with a (massive) performance hit for large regions. I tried to contract Africa with docker and after 24 hours it was about a quarter done so I killed it, build the same OSRM version locally and ran the exact same contract in a couple of hours

[daniel-j-h]

Good idea - want to send in a pull request for that?

[Nicien]

Hello guys, i'm running a dockerized osrm-prepare/contract on an ec2 m4.16xlarge (64 cores, 256G) .

I replaced alpine by ubuntu, the improvement is epic.

alpine: after 20 hours of processing... still running.
ubuntu: 2 hours. Done.

@daniel-j-h I could make a pull request :)

[daniel-j-h]

:+1: happy to review it.

[xydinesh]

@daniel-j-h added a pull request for debian based docker image. Appreciate if you can review it.