Open3d: Headless rendering does not work

[ykaganov]

Building open3D using OSMesa for headless rendering as described in http://www.open3d.org/docs/tutorial/Advanced/headless_rendering.html does not work.
I tried installing osmesa using "sudo apt-get install libosmesa6-dev" and also by building from source as described in the link above. Both produce the following error message when running headless_rendering.py from the open3d examples:
"GLFW Error: OSMesa: Foward-compatible contexts not supported
Failed to create window". All glfw and glew were installed from the 3rdparty folder in the repository ( https://github.com/intel-isl/Open3D-3rdparty/tree/adac428ffa79f9e0e0a17878b5b246295d53dbb3). I tried on several different machines all running Ubuntu 16.04 and all give the same error.

[lucabergamini]

Same here, Ubuntu 18.04.

[JackLangerman]

I am also experiencing this.

I am trying to set up the ability to render images from a point cloud on a google cloud vm.

But I see this error:

GLFW Error: OSMesa: Foward-compatible contexts not supported Failed to create window

Here is my setup script which I run on a fresh Ubuntu 16.04 LTS VM:

```{bash}

compile open3d

update packages

sudo apt-get update -y
sudo apt-get upgrade -y

install OSMesa

sudo apt-get install libosmesa6-dev -y

setup virtualenv

sudo apt-get install virtualenv python-pip -y
virtualenv -p /usr/bin/python3 py3env

enter python env

source py3env/bin/activate

install numpy / matplotlib

pip install numpy matplotlib

download open3d source

git clone --recursive https://github.com/intel-isl/Open3D

install deps

cd Open3D
bash util/scripts/install-deps-ubuntu.sh
sudo apt-get install cmake -y

build

cmake -DENABLE_HEADLESS_RENDERING=ON \
-DBUILD_GLEW=ON \
-DBUILD_GLFW=ON \
-DBUILD_PYBIND11=ON \
-DPYTHON_EXECUTABLE:FILEPATH=$(which python) \
CMakeLists.txt

make

install

make install-pip-package

this should now work: (and it does)

python -c "import open3d"

this should also work: (and it doesn't)

cd examples/Python/Advanced
python headless_rendering.py

```{bash}
(py3env) a_-_@dev-vm:~/Open3D/examples/Python/Advanced$ ls
camera_trajectory.py                customized_visualization.py  global_registration.py  interactive_visualization.py  non_blocking_visualization.py  rgbd_integration.py
colored_pointcloud_registration.py  downsampling_and_trace.py    headless_rendering.py   load_save_viewpoint.py        pointcloud_outlier_removal.py  trajectory_io.py
color_map_optimization.py           fast_global_registration.py  __init__.py             multiway_registration.py      remove_geometry.py
(py3env) a_-_@dev-vm:~/Open3D/examples/Python/Advanced$ python headless_rendering.py 
Reading PLY: [========================================] 100%
Customized visualization playing a camera trajectory. Ctrl+z to terminate
GLFW Error: OSMesa: Foward-compatible contexts not supported
Failed to create window
Traceback (most recent call last):
  File "headless_rendering.py", line 69, in <module>
    custom_draw_geometry_with_camera_trajectory(pcd)
  File "headless_rendering.py", line 57, in custom_draw_geometry_with_camera_trajectory
    vis.get_render_option().load_from_json("../../TestData/renderoption.json")
AttributeError: 'NoneType' object has no attribute 'load_from_json'

Happy to work on this / help update the docs if someone can point me in the right direction. For now I am following the instructions here:
http://www.open3d.org/docs/tutorial/Advanced/headless_rendering.html
http://www.open3d.org/docs/compilation.html

If there was a way to have this work inside conda / be setup with conda that would be ideal, but again, happy to work on this.

In an ideal world I could just
conda install open3d
and if I happen to be on a local machine with a display it should work to do things like this:

```{python}
vis = o3d.Visualizer()
vis.create_window(visible=True)

while in a headless environment that could throw an exception, but this would still work:
```{python}
vis = o3d.Visualizer()
vis.create_window(visible=False)

PS
thanks for the awesome library!

[qianyizh]

@syncle @yxlao take a look at this?

[JackLangerman]

related to #972

[yxlao]

~Probably, this is due to (OS)MESA only supports up to OpenGL 3.1 (see their FAQ), while we're using OpenGL 3.3.~

Edit: from the release note, OSMESA does support later versions of OpenGL

[JackLangerman]

Just wanted to circle back on this. Does the support for OpenGL 3.3 in OSMESA imply a course of action? Any more thoughts on this? Happy to do some work on this issue (a solution would really help me), but I think I have a much better chance to make substantial progress if someone can get me pointed in the right direction.

I'm pretty sure I'm not the only one who would benefit from this. I think it is a relatively common use case to work on a remote headless server.

Thoughts?

[SapphireEmbers]

Hi - Any updates here? I have run into the same issue and it would also _really_ help me to have a solution.

Thoughts?

[Sireer]

Same for me on Ubuntu 16.04.6 LTS.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: [Open3D ERROR] GLFW Error: OSMesa: Foward-compatible contexts not supported

[Chandler]

Same here.

When I run this on a remote machine over ssh -X
python -c "import open3d; open3d.draw_geometries([])"

I see

GLFW Error: GLX: Forward compatibility requested but GLX_ARB_create_context_profile is unavailable
Failed to create window
[DrawGeometries] Failed creating OpenGL window.

[maxjaritz]

I still get the same error as described by @Chandler above with open3D 0.9.

[nicolasugrinovic]

@ykaganov how do you manage to build with cmake? I get the following error:
NASM assembler not found

[maxjaritz]

I simply installed with pip.

[JackLangerman]

Hey all,

If anyone has thoughts I'd really be stoked to get good documentation on how to do this (I am happy to write it if I / someone can get it figured out). It appears to have been fixed in #1358, but I am still having trouble. I posted a detailed followup here -> https://github.com/intel-isl/Open3D/pull/1358#issuecomment-594763459 .

Thanks!

[germanros1987]

Did you build it from source using -DENABLE_HEADLESS_RENDERING=ON ?

[JackLangerman]

Hi @germanros1987,

Thanks for getting back to me.

Did you build it from source using -DENABLE_HEADLESS_RENDERING=ON ?

Yes I did.

You can see my whole process in my comment on your PR: https://github.com/intel-isl/Open3D/pull/1358#issuecomment-594763459 .

but here is my build command:

...

# configure build
cmake -DENABLE_HEADLESS_RENDERING=ON \
      -DBUILD_GLEW=ON \
      -DBUILD_GLFW=ON \
      -DBUILD_PYBIND11=ON \
      -DBUILD_SHARED_LIBS=ON \
      -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 \
      CMakeLists.txt

...

Thanks for your help!

[JackLangerman]

@germanros1987 one other thing:

if you would be willing to post your system specs and build/install script which you used to get headless rendering working on linux I am on gcp and could follow it precisely (and write it into the docs). Would that work for you?

(additional followup to https://github.com/intel-isl/Open3D/pull/1358#issuecomment-594763459 and https://github.com/intel-isl/Open3D/issues/940#issuecomment-595221921)

Thanks!

[rajaneeshdwivedi]

For what it’s worth I’m experiencing the same outcome on Centos 7 with same build options and a GLFW built with an OSMesa profile.

As an aside, am absolutely loving this library and very grateful such outstanding work has been open-sourced.

[Xardvor]

Hi @JackLangerman! I'm now trying to solve problem with headless rendering.
Could you please run glxinfo on your VM and pin its output here using gist?
In case your VM doesn't have glxinfo you need to install mesa-utils package. For Ubuntu, use sudo apt-get install mesa-utils

[Xardvor]

@JackLangerman sorry, glxinfo wouldn't work on headless machine, I forgot that it needs a display. I need info about OpenGL version supported by your VM, will try to figure out how you may get it.

[Xardvor]

About environment - it looks exactly the same as mine. For the setup steps, I didn't use shared libs (DBUILD_SHARED_LIBS=ON), but it shouldn't make any difference.

[rajaneeshdwivedi]

I have glxinfo on a headless machine using Xvfb as a virtual display. Happy to provide output if that's helpful.

[Xardvor]

@rajaneeshdwivedi yeah, sure, I will take a look. It's not exactly that OSMesa uses, but still it's helpful to have output from glxinfo under xvfb.

[rajaneeshdwivedi]

glxinfo output under xvfb

[Xardvor]

Great! Thank you!

[JackLangerman]

Hi all!

Thanks so much for working on this!

As you said glxinfo does require a display

(py3env) jack@headless-rendering-2:~/Open3D$ sudo apt-get install mesa-utils
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  grub-pc-bin libboost-filesystem1.58.0 libboost-system1.58.0 libcapnp-0.5.3 libdmx-dev libdmx1 libfontenc-dev libfontenc1 libfs-dev libfs6 libice-dev libice6
  libmirclient-dev libmirclient9 libmircommon-dev libmircommon7 libmircookie-dev libmircookie2 libmircore-dev libmircore1 libmirprotobuf3 libpciaccess-dev
  libpixman-1-0 libpixman-1-dev libprotobuf-dev libprotobuf-lite9v5 libprotobuf9v5 libsm-dev libsm6 libxaw7 libxaw7-dev libxcomposite-dev libxcomposite1
  libxfont1 libxfont2 libxkbcommon-dev libxkbcommon0 libxkbfile-dev libxkbfile1 libxmu-dev libxmu-headers libxmu6 libxmuu-dev libxpm-dev libxpm4 libxres-dev
  libxres1 libxss-dev libxt-dev libxt6 libxtst-dev libxtst6 libxv-dev libxv1 libxvmc-dev libxvmc1 libxxf86dga-dev libxxf86dga1 mir-client-platform-mesa-dev
  x11proto-bigreqs-dev x11proto-composite-dev x11proto-dmx-dev x11proto-dri3-dev x11proto-fonts-dev x11proto-present-dev x11proto-record-dev
  x11proto-resource-dev x11proto-scrnsaver-dev x11proto-video-dev x11proto-xcmisc-dev x11proto-xf86bigfont-dev x11proto-xf86dga-dev x11proto-xf86dri-dev
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  mesa-utils
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 31.0 kB of archives.
After this operation, 118 kB of additional disk space will be used.
Get:1 http://us-east1.gce.archive.ubuntu.com/ubuntu xenial/universe amd64 mesa-utils amd64 8.3.0-1 [31.0 kB]
Fetched 31.0 kB in 0s (469 kB/s)      
Selecting previously unselected package mesa-utils.
(Reading database ... 83402 files and directories currently installed.)
Preparing to unpack .../mesa-utils_8.3.0-1_amd64.deb ...
Unpacking mesa-utils (8.3.0-1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up mesa-utils (8.3.0-1) ...
(py3env) jack@headless-rendering-2:~/Open3D$ glxinfo
Error: unable to open display 
(py3env) jack@headless-rendering-2:~/Open3D$ 

Do this or this seem relevant to you?

Here is some more info about my system (not sure if it is relevant):

(py3env) jack@headless-rendering-2:~/Open3D$ lspci -k
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
    Subsystem: Red Hat, Inc. Qemu virtual machine
00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI
    Subsystem: Red Hat, Inc. Virtio SCSI
    Kernel driver in use: virtio-pci
00:04.0 3D controller: NVIDIA Corporation GK210GL [Tesla K80] (rev a1)
    Subsystem: NVIDIA Corporation GK210GL [Tesla K80]
    Kernel modules: nvidiafb
00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device
    Subsystem: Red Hat, Inc. Virtio network device
    Kernel driver in use: virtio-pci
00:06.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG
    Subsystem: Red Hat, Inc. Virtio RNG
    Kernel driver in use: virtio-pci

Please let me know if there is anything else I can do to help get this solved.

Thanks so much!
Jack