Drake: RgbdCamera Unit Test Fails in CI

[kunimatsu-tri]

In the PR for RgbdCamera, the unit test failed in CI. There is no failure in my local machine.

It seems this is due to the lack of DISPLAY environment variable. The actual error message is:

190: ERROR: In /build/vtk-NWL3SN/vtk-5.10.1+dfsg/Rendering/vtkXOpenGLRenderWindow.cxx, line 539
190: vtkXOpenGLRenderWindow (0x1b820a0): bad X server connection. DISPLAY=ERROR: In /build/vtk-NWL3SN/vtk-5.10.1+dfsg/Rendering/vtkXOpenGLRenderWindow.cxx, line 343
190: vtkXOpenGLRenderWindow (0x1b820a0): bad X server connection. DISPLAY=
250/263 Test #186: rotary_encoders_test ...............................   Passed   12.33 sec
251/263 Test #219: acrobot_run_passive ................................   Passed   10.31 sec
252/263 Test #190: rgbd_camera_test ...................................***Exception: SegFault 11.60 sec
Running main() from gtest_main.cc
[==========] Running 5 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from RgbdCameraTest
[ RUN      ] RgbdCameraTest.InstantiateTest
ERROR: In /build/vtk-NWL3SN/vtk-5.10.1+dfsg/Rendering/vtkXOpenGLRenderWindow.cxx, line 539
vtkXOpenGLRenderWindow (0x1b820a0): bad X server connection. DISPLAY=ERROR: In /build/vtk-NWL3SN/vtk-5.10.1+dfsg/Rendering/vtkXOpenGLRenderWindow.cxx, line 343
vtkXOpenGLRenderWindow (0x1b820a0): bad X server connection. DISPLAY=

vtkXOpenGLRenderWindow.cxx is calling XOpenDisplay here and this needs DISPLAY.

In the #5210, @patmarion is suggesting to replace xvfb-run with Xvfb and this might solve this issue too, if the root cause is the same.

[kunimatsu-tri]

Now this issue is actually blocking #5201 to be merged.

@david-german-tri: Is it possible to set Xvfb in CI as patmarion is suggesting in #5210?

[david-german-tri]

from f2f conversation: I agree, we should try @patmarion's approach, and have sent a ping to Kitware. In the meantime, consider running your test under xvfb-run using logic similar to https://github.com/RobotLocomotion/director/blob/master/src/python/tests/CMakeLists.txt#L102

[kunimatsu-tri]

When I was trying to the director's approach, I noticed that Xvfb doesn't work for my case.
It gives me the following error:

ERROR: In /build/buildd/vtk-5.8.0/Rendering/vtkXOpenGLRenderWindow.cxx, line 404
vtkXOpenGLRenderWindow (0x19e8120): Could not find a decent visual

This is because Xvfb doesn't fully support GLX extensions. To cope with this problem, Xdummy is widely used.

So, I'm wondering now if we could use Xdummy instead of Xvfb, although a part of the unit tests for RgbdCamera fails even with Xdummy.

I'm debugging now and will update.

[patmarion]

Director tests use Xvfb (via xvfb-run) on the drake ci servers, it works. You might need to apt-get install a package (that might have been removed from drakes wiki instructions)

[kunimatsu-tri]

Can I see the output of glxinfo in CI environment somehow?

[kunimatsu-tri]

Never mind. Maybe I can open a dummy PR just to see glxinfo in CI.

[kunimatsu-tri]

I wrote a bash script that runs both Xvfb and a text executable:

Xvfb :88 -ac -screen 0 1280x1024x24 &
sleep 3
DISPLAY=:88 $1
test_result=$?
if [ "$test_result" == "0" ]
then
  echo "PASS"
else
  echo "FAIL"
fi
xvfb_pid="`pgrep -f "Xvfb :88"`"
kill -9 $xvfb_pid

and tried this script in CI.

The test passed on Trusty, but failed on Xenial. The error is:

00:29:23 195: X Error of failed request:  BadValue (integer parameter out of range for operation)
00:29:23 195:   Major opcode of failed request:  151 (GLX)
00:29:23 195:   Minor opcode of failed request:  3 (X_GLXCreateContext)
00:29:23 195:   Value in failed request:  0x0
00:29:23 195:   Serial number of failed request:  19
00:29:23 195:   Current serial number in output stream:  24

Also, the test with Xvfb didn't pass in my local Trusty machine neither as I reported above.
I guess this is because I'm using nvidia's libGL.so instead of mesa's.
The test passed in my local Trusty if I don't use Xvfb.

[kunimatsu-tri]

Calling vtkWindowToFilter::ReadFrontBufferOff() solved the error on Xenial.
Also, avoiding using Xvfb solved the error in my local machine when nvidia libGL.so is used.

So, there is no concern using Xvfb now.

[kunimatsu-tri]

By the face to face discussion with @sammy-tri and @jwnimmer-tri, we decided to wait for CI to have Xvfb. In the meantime, having manual test is going to be the temporal solution.

[david-german-tri]

@jamiesnape @BetsyMcPhail I think this issue (and the related #5210) is the highest-priority item in the Kitware queue that isn't already getting active development.

cc @stonier

[stonier]

@kunimatsu-tri I believe we can close this now. If you concur, please do :)

[kunimatsu-tri]

Before closing this issue, I think the manual test for 'RgbdCamera' should be replaced with automated test. See this: https://github.com/RobotLocomotion/drake/blob/master/drake/systems/sensors/BUILD#L324

I'll check if CI works or not when I have time (I'm on mobile now.)

[kunimatsu-tri]

Removed the manual test tag in #6773 to see if the PR passes CI, but failed.

Here is the error message I've got:

19:46:16 ==================== Test output for //drake/systems/sensors:rgbd_camera_test:
19:46:16 Running main() from gmock_main.cc
19:46:16 [==========] Running 13 tests from 3 test cases.
19:46:16 [----------] Global test environment set-up.
19:46:16 [----------] 2 tests from RgbdCameraTest
19:46:16 [ RUN      ] RgbdCameraTest.InstantiateTest
19:46:16 ERROR: In /root/Projects/vtk/vtk/Rendering/OpenGL2/vtkXOpenGLRenderWindow.cxx, line 468
19:46:16 vtkXOpenGLRenderWindow (0x56061c571070): bad X server connection. DISPLAY=external/bazel_tools/tools/test/test-setup.sh: line 168:     9 Aborted                 (core dumped) "${TEST_PATH}" "$@"

So, I think Xvfb has not been ready yet.

[jamiesnape]

Not yet. I will open a PR to enable the test when it is.