Odm: Getting error while running from docker

[Sharathmk99]

Hi All,

I'm getting below error while trying to run from Docker,
Please Help,
Machine Configuration: 8GB Ram, 4Core

DJI_0079.JPG - DJI_0096.JPG has 31 candidate matches
Robust matching time : 0.000393867492676s
Full matching 27 / 31, time: 1.43784499168s
DJI_0079.JPG - DJI_0087.JPG has 225 candidate matches
Robust matching time : 0.000363111495972s
Full matching 214 / 225, time: 1.4198319912s
DJI_0079.JPG - DJI_0102.JPG has 28 candidate matches
Robust matching time : 0.000262022018433s
Full matching 26 / 28, time: 1.40974402428s
DJI_0079.JPG - DJI_0086.JPG has 1201 candidate matches
Robust matching time : 0.00075101852417s
Full matching 1166 / 1201, time: 1.42681789398s
2016-07-22 07:11:36,245 reading features
2016-07-22 07:11:36,745 Merging features onto tracks
2016-07-22 07:11:38,044 Good tracks: 38609
/code/SuperBuild/src/opensfm/bin/run_all: line 11: 32 Illegal instruction (core dumped) $DIR/reconstruct $1
[ERROR] quitting cause:
PYTHONPATH=/code/SuperBuild/install/lib/python2.7/dist-packages /code/SuperBuild/src/opensfm/bin/run_all /images/opensfm
returned with code 33792.

An error occurred. Check stdout above or the logs.

[dakotabenjamin]

How many images are you trying to process? We have as similar issue here: #332

[Sharathmk99]

Thank you for the response.

I'm trying to process 30 images.

On 22 Jul 2016 11:23 p.m., "Dakota Benjamin" [email protected]
wrote:

How many images are you trying to process? We have as similar issue here:

332 https://github.com/OpenDroneMap/OpenDroneMap/issues/332

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenDroneMap/OpenDroneMap/issues/346#issuecomment-234611157,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADyVJJIYcz1QoybDWORzeK7PXPG3jiqBks5qYQN_gaJpZM4JSgHL
.

[Sharathmk99]

I tried with all argument in the #332 thread. Still same error.
Please help

[dakotabenjamin]

Can you share the images via a dropbox link or something similar? You can do so privately if you need to.

[Sharathmk99]

I used sample odm data,
https://github.com/merkato/odm_boruszyn_kap
And
https://github.com/OpenDroneMap/OpenDroneMap/tree/master/tests/test_data

On 22 Jul 2016 11:46 p.m., "Dakota Benjamin" [email protected]
wrote:

Can you share the images via a dropbox link or something similar? You can
do so privately if you need to.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenDroneMap/OpenDroneMap/issues/346#issuecomment-234617058,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADyVJGLKAxwbtfocBfZa_3H4QR_xw5RYks5qYQjzgaJpZM4JSgHL
.

[dakotabenjamin]

OK I need more information. If you can provide it, what steps did you take specifically to reproduce this issue? Start with your machine information and what you did to run the docker image, etc. Thank you

[Sharathmk99]

I downloaded the image using
docker pull opendronemap/opendronemap.

And I downloaded images as described from above links.

And setup images directory using,
export IMAGES=path of image directory.

Now I ran docker using below command,

docker run -v $IMAGES:/images opendronemap/opendronemap

After some time around 4 to 5 min I got above error.

System configuration,

8GB ram
I3 processor
2core machine.

If any things needed please let me know.

On 22 Jul 2016 11:58 p.m., "Dakota Benjamin" [email protected]
wrote:

OK I need more information. If you can provide it, what steps did you take
specifically to reproduce this issue? Start with your machine information
and what you did to run the docker image, etc. Thank you

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenDroneMap/OpenDroneMap/issues/346#issuecomment-234620095,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADyVJJibI2-QUz1XTW9jjDdx3oDoQyEHks5qYQu6gaJpZM4JSgHL
.

[dakotabenjamin]

So do you have 2 cores or 4? Or is it 2 threaded cores?

I think that could be a problem. I'm going to find a slow machine and try it on Monday.

[Sharathmk99]

I have 2cores machine.

What is the recommended configuration.?

On 23 Jul 2016 1:14 a.m., "Dakota Benjamin" [email protected]
wrote:

So do you have 2 cores or 4? Or is it 2 threaded cores?

I think that could be a problem. I'm going to find a slow machine and try
it on Monday.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenDroneMap/OpenDroneMap/issues/346#issuecomment-234638039,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADyVJBcU_Yp1mWN82KAah_DfBEjgnxvfks5qYR26gaJpZM4JSgHL
.

[dakotabenjamin]

The more the better, for sure. I haven't been able to test minimum requirements yet but I would say based on experience 4 cores is minimum

[Sharathmk99]

Ok I'll try to get 4cores machine.

On 23 Jul 2016 1:18 a.m., "Dakota Benjamin" [email protected]
wrote:

The more the better, for sure. I haven't been able to test minimum
requirements yet but I would say based on experience 4 cores is minimum

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenDroneMap/OpenDroneMap/issues/346#issuecomment-234639084,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADyVJInOcRWcORU48WiF-ECH_YxCyrHbks5qYR6XgaJpZM4JSgHL
.

[Fi156]

Some time ago I tested the non docker version on an I5 520m (2cores/4threads) with 4gb RAM without a problem.

[waldheinz]

The fact that it complains about an illegal instruction means that either something in the docker image uses instructions that the CPU does not support (AVX or AVX2 maybe?). Building the docker image on the target machine might help in case there is some software is compiled with -march=native. Otherwise faulty hardware is also a possibility.

@Sharathmk99 can you provide the exact processor type you're using? Maybe the contents of /proc/cpuinfo?

[mojodna]

Just saw this on a 2009 iMac (2.8 GHz Intel Core i7, Lynnfield/Nehalem) with 4 cores + 8GB RAM allocated to a VMware VM provisioned with docker-machine when processing 13 images (https://github.com/mojodna/odm_seattle_university):

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 30
model name  : Intel(R) Core(TM) i7 CPU         860  @ 2.80GHz
stepping    : 5
microcode   : 0x5
cpu MHz     : 2791.237
cache size  : 8192 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lm ida dtherm
bugs        :
bogomips    : 5582.47
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

In this case, the Docker image was built on a Skylake i7, so the CPU instruction limitation seems very possible.

[mojodna]

Skylake /proc/cpuinfo:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 94
model name  : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
stepping    : 3
microcode   : 0x74
cpu MHz     : 799.929
cache size  : 8192 KB
physical id : 0
siblings    : 8
core id     : 0
cpu cores   : 4
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 22
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp
bugs        :
bogomips    : 6815.78
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

[dakotabenjamin]

Is the first /proc/cpuinfo in the VM or the docker-machine?

I've gotta say, Docker is very confusing as far as terms go. Even after reading the overview I don't really understand what a docker-machine really is or how cpu allocation works in that regard.

[mojodna]

Both were extracted from within a Docker container. The first was a container within a boot2docker VM (VMs are necessary to run Docker on OS X / Windows), the second was from a container running on a bare-metal Linux machine.

When run from a VM, there are 2 layers where CPU / memory can be allotted: what's available to the virtual machine and what docker makes available to containers (using cgroups).

When run on a Linux machine, it's as though it's running within a VM, so the latter is the only way CPU / memory allocation can occur.

[dakotabenjamin]

This kind of problem is outside my wheelhouse, unfortunately... :disappointed:

Did you run into any issues/errors, or simply the normal effects of a single cpu?

[dakotabenjamin]

Does running docker-machine create with --virtualbox-cpu-count 4 change it?

from here: https://docs.docker.com/machine/drivers/virtualbox/

[mojodna]

It changes the number of CPUs available, but I think @waldheinz is right about the instructions available when it was built vs the instructions available when it's run. I'll rebuild on the older machine and see if it solves the issue. If so, we may be able to provide specific flags when building opensfm in a Docker image intended for redistribution.

[mojodna]

Yup, that appears to have been the problem.

Next action (for me or whomever gets to it first): look into limiting the CPU instructions generated when compiling opensfm for compatibility (at the probable expense of performance) with older hardware.

[dakotabenjamin]

@paulinus do you know of this?

[mojodna]

Possibly helpful CFLAGS incantations for building generic images: https://wiki.gentoo.org/wiki/Safe_CFLAGS

[dakotabenjamin]

This issue is outdated and will be closed. Please open a new thread in http://community.opendronemap.org and reference this issue if you would like to continue the conversation.