Osrm-backend: Shared memory multiple issues

[RobertSellers]

Hi to anyone here who can help out with this. I'm unable to run osrm-routed on a shared-memory state, despite having gone through all of the various steps in the shared memory guide. Likewise osrm-datastore is not behaving in an understandable way. I am running Linux 16.04.

uname -a
Linux ip-172-31-48-27 4.4.0-109-generic #132-Ubuntu SMP Tue Jan 9 19:52:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

ulimit -a|grep max

max locked memory       (kbytes, -l) 68719476736
max memory size         (kbytes, -m) unlimited
max user processes              (-u) 128532

```sh
free -h
total used free shared buff/cache available
Mem: 31G 349M 12G 9.7G 18G 20G
Swap: 101G 0B 101G

```sh
sudo sysctl -p

kernel.shmall = 1152921504606846720
kernel.shmmax = 18446744073709551615

cat /proc/sys/kernel/shmmax
18446744073709551615

Running osrm-datastore gives me the following bug, however this runs "normally" using sudo.

osrm-datastore osrm_data/ecar_lua/file.osrm

terminate called after throwing an instance of 'boost::interprocess::interprocess_exception'

```sh
sudo osrm-datastore osrm_data/ecar_lua/file.osrm

[info] Data layout has a size of 1894 bytes
[info] Allocating shared memory of 1024845484 bytes
[info] Data layout has a size of 1661 bytes
[info] Allocating shared memory of 2369514796 bytes
[info] All data loaded. Notify all client about new data in:
[info] /static 0
[info] /updatable 1
[info] All clients switched.

Running ipcs -a aafter osrm-datastore yields the following:

------ Message Queues --------
key msqid owner perms used-bytes messages

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x0052e2c1 0 postgres 600 56 5
0x00000000 98305 lightdm 600 524288 2 dest
0x00000000 196610 lightdm 600 524288 2 dest
0x00000000 229379 lightdm 600 33554432 2 dest
0x000104a1 393220 root 644 1024845886 0 locked
0x010104a1 425989 root 644 2369721773 0 locked
0x020104a1 327686 root 644 1024845886 0 locked
0x030104a1 360455 root 644 2369721773 0 locked
0x060104a1 524298 root 644 1024845886 0 locked
0x070104a1 557067 root 644 2369721773 0 locked

------ Semaphore Arrays --------
key semid owner perms nsems
0x0052e2c1 0 postgres 600 17
0x0052e2c2 32769 postgres 600 17
0x0052e2c3 65538 postgres 600 17
0x0052e2c4 98307 postgres 600 17
0x0052e2c5 131076 postgres 600 17
0x0052e2c6 163845 postgres 600 17
0x0052e2c7 196614 postgres 600 17
0x0052e2c8 229383 postgres 600 17
0x0052e2c9 262152 postgres 600 17
0x0052e2ca 294921 postgres 600 17
0x0052e2cb 327690 postgres 600 17
0x0052e2cc 360459 postgres 600 17
0x0052e2cd 393228 postgres 600 17
0x0052e2ce 425997 postgres 600 17

Running osrm-routed does not work with the -s flag due to "Shared memory settings conflicts".  I can get it to run without -shared-memory tag...

```sh
 osrm-routed --port 5001 --algorithm mld osrm_data/ecar_lua/file.osrm --shared-memory

[warn] Shared memory settings conflict with path settings.

[RobertSellers]

Nobody else get past this error?

[warn] Shared memory settings conflict with path settings.

[RobertSellers]

Solved by not including the .osrm --base target in the osrm-routed function...please be more explicit in the docs.

[johnjinwoua]

Hello @RobertSellers please could you explain how you solved the issue ?
I am facing the same problem.

Thanks,

Regards

[danpat]

When using osrm-routed in shared memory mode, you do not pass it a filename.

Assuming your user has sufficient permission to create a shared memory block, usage is typically:

$ osrm-datastore path/to/your/map.osrm
$ osrm-routed --shared-memory

Depending on how you prepared the datafiles, you may need to add --algorithm mld to the osrm-routed invocation.

[johnjinwoua]

I am working with docker. I have this error message No shared memory block 'osrm-region' found, have you forgotten to run osrm-datastore?include/storage/shared_monitor.hpp:83

This are the different commands I launch for docker :

• docker run -t -v $(pwd):/data osrm/osrm-backend:latest osrm-extract -p /opt/car.lua /data/luxembourg-latest.osm.pbf
• docker run -t -v $(pwd):/data osrm/osrm-backend:latest osrm-partition /data/luxembourg-latest.osrm
• docker run -t -v $(pwd):/data osrm/osrm-backend:latest osrm-customize /data/luxembourg-latest.osrm
  -docker run --ipc host -v ${pwd}:/data osrm/osrm-backend:latest osrm-datastore /data/luxembourg-latest.osrm
• docker run -t -i -p 5000:5000 -v $(pwd):/data osrm/osrm-backend:latest osrm-routed --algorithm=MLD -s

When I do not use shared memory, it works fine.

Thanks for helping.

Regards

[danpat]

You also need to pass the --ipc host to the docker run command that launches osrm-routed.

[johnjinwoua]

Thanks. It works 👍
I have to try now to include the traffic.csv file.

Regards.

[johnjinwoua]

Sorry for reopening.

I can't actually curl when I use shared memory. This is what I obtain :

[info] starting up engines, v5.22.0
[info] Loading from shared memory
[info] Threads: 14
[info] IP address: 0.0.0.0
[info] IP port: 5000

It should normally be :

[info] starting up engines, v5.22.0
[info] Threads: 14
[info] IP address: 0.0.0.0
[info] IP port: 5000
[info] http 1.1 compression handled by zlib version 1.2.8
[info] Listening on: 0.0.0.0:5000
[info] running and waiting for requests

Someone has an idea ?

Thanks

[johnjinwoua]

I restarted my computer and now I am getting this error message after running the orsm-routed (docker run -t -i -p 5000:5000 -v $(pwd):/data osrm/osrm-backend:latest osrm-routed --algorithm=MLD -s) :

terminate called after throwing an instance of 'osrm::util::exception' what(): No shared memory block 'osrm-region' found, have you forgotten to run osrm-datastore?include/storage/shared_monitor.hpp:83

I ran the osrm-datastore before the osrm-routed through:

docker run --ipc host -v ${pwd}:/data osrm/osrm-backend:latest osrm-datastore /data/luxembourg-latest.osrm

And I get this message :

panic: runtime error: index out of range

goroutine 1 [running]:

github.com/docker/cli/cli/compose/loader.isFilePath(0xc000331118, 0x2, 0x700a508) /go/src/github.com/docker/cli/cli/compose/loader/volume.go:121 +0x11b

github.com/docker/cli/cli/compose/loader.populateType(0xc0001eb2a8) /go/src/github.com/docker/cli/cli/compose/loader/volume.go:102 +0x74

github.com/docker/cli/cli/compose/loader.ParseVolume(0x7ffeefbffbc8, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /go/src/github.com/docker/cli/cli/compose/loader/volume.go:44 +0x3f6

github.com/docker/cli/cli/command/container.parse(0xc000140400, 0xc000456580, 0xc0001c2bb0, 0x5, 0x0, 0x0, 0xc0005ca000) /go/src/github.com/docker/cli/cli/command/container/opts.go:344 +0x28f

github.com/docker/cli/cli/command/container.runRun(0x5e0b440, 0xc00026a000, 0xc000140400, 0xc0000b2800, 0xc000456580, 0x55a8370, 0x48ce4b0) /go/src/github.com/docker/cli/cli/command/container/run.go:80 +0x4be

github.com/docker/cli/cli/command/container.NewRunCommand.func1(0xc000490f00, 0xc0004fa180, 0x3, 0x6, 0x0, 0x0) /go/src/github.com/docker/cli/cli/command/container/run.go:46 +0xfd

github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc000490f00, 0xc000327970, 0x6, 0x6, 0xc000490f00, 0xc000327970) /go/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:762 +0x465

github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0002fec80, 0xc000327960, 0x7, 0x7) /go/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:852 +0x2ec

github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).Execute(...) /go/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:800

main.runDocker(0xc00026a000, 0x5d5aae0, 0xc000010018) /go/src/github.com/docker/cli/cmd/docker/docker.go:281 +0x1d3

main.main() /go/src/github.com/docker/cli/cmd/docker/docker.go:292 +0xf1

Am I doing something wrong ?

Thanks :)