Mediapipe: Could not find type "type.googleapis.com/mediapipe.OpenCvVideoEncoderCalculatorOptions" stored in google.protobuf.Any.

[laoreja]

Platform: Linux, Ubuntu18.04 with GPU
Gcc G++ version: 7.5

I'm trying to run multi-hand tracking with input and output video files using https://github.com/google/mediapipe/blob/master/mediapipe/docs/multi_hand_tracking_desktop.md#tensorflow-lite-multi-hand-tracking-demo-with-webcam-gpu this tutorial.

Exact same building command:

bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \
    mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_gpu

And the building is successful.

When I try to run

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_gpu \
    --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_desktop.pbtxt --input_video_path=input.mp4 --output_video_path=output.mp4

The error I get is:

[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/text_format.cc:309] Error parsing text-format medi
apipe.CalculatorGraphConfig: 122:74: Could not find type "type.googleapis.com/mediapipe.OpenCvVideoEncoderCalculatorOp
tions" stored in google.protobuf.Any.
F0425 03:07:09.026211  4942 parse_text_proto.h:27] Check failed: google::protobuf::TextFormat::ParseFromString(input,
&result)
*** Check failure stack trace: ***
    @     0x55fae3a63a8f  (unknown)
    @     0x55fae3a62adb  (unknown)
    @     0x55fae3a64309  (unknown)
    @     0x55fae35b65d0  (unknown)
    @     0x55fae35a2619  (unknown)
    @     0x7f2840d31b97  __libc_start_main
    @     0x55fae35b1faa  (unknown)
    @              (nil)  (unknown)
[1]    4942 abort (core dumped)  GLOG_logtostderr=1   --input_video_path=input.mp4

When I try to run multi_hand_tracking_desktop_live.pbtxt or multi_hand_tracking_mobile.pbtxt

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_gpu \
    --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_desktop_live.pbtxt --input_video_path=input.mp4 --output_video_path=output.mp4

The error I get is:

I0425 03:25:26.505847  5908 demo_run_graph_main_gpu.cc:57] Initialize the calculator graph.
I0425 03:25:26.508827  5908 demo_run_graph_main_gpu.cc:61] Initialize the GPU.
I0425 03:25:26.519194  5908 gl_context_egl.cc:158] Successfully initialized EGL. Major : 1 Minor: 5
W0425 03:25:26.519245  5908 gl_context_egl.cc:163] Creating a context with OpenGL ES 3 failed: UNKNOWN: ; eglChooseConfig() returned no matching EGL configuration for RGBA8888 D16 ES3 request.
W0425 03:25:26.519259  5908 gl_context_egl.cc:164] Fall back on OpenGL ES 2.
E0425 03:25:26.523185  5908 demo_run_graph_main_gpu.cc:186] Failed to run the graph: ; eglChooseConfig() returned no matching EGL configuration for RGBA8888 D16 ES2 request.

[jiuqiant]

Could you verify that your OpenGL ES version is 3.1 or above? See this requirement at https://github.com/google/mediapipe/blob/master/mediapipe/docs/gpu.md#opengl-es-setup-on-linux-desktop.

[RLashofRegas]

@jiuqiant - I am getting the same error and I am on OpenGL ES version 3.2:

$ glxinfo | grep -i opengl
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: GeForce GTX 980/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 450.57
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6.0 NVIDIA 450.57
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 450.57
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

I am running on archlinux, below are some relevant package versions:

libuv 1.38.1-1
glu 9.0.1-2
mesa 20.1.3-1
mesa-demos 8.4.0-4
bazel 3.3.0-1
opencv 4.3.0-7

During build I did have to change a couple of things due to opencv4. Namely, srcs, hdrs, and includes for opencv_linux.BUILD

cc_library(
    name = "opencv",
    srcs = glob(
        [
            "lib/libopencv_core.so",
            "lib/libopencv_calib3d.so",
            "lib/libopencv_features2d.so",
            "lib/libopencv_highgui.so",
            "lib/libopencv_imgcodecs.so",
            "lib/libopencv_imgproc.so",
            "lib/libopencv_video.so",
            "lib/libopencv_videoio.so",
        ],
    ),
    hdrs = glob(["include/opencv4/opencv2/**/*.h*"]),
    includes = ["include/opencv4/"],
    linkstatic = 1,
    visibility = ["//visibility:public"],
)

[RLashofRegas]

@jiuqiant - I was able to get past this by adding ":desktop_offline_calculators" to mediapipe/graphs/hand_tracking:multi_hand_mobile_calculators. It seems strange to me that the desktop gpu examples are referencing the mobile calculators. Is this expected?

[RLashofRegas]

At least for me, I found that the problem was that I was trying to run "multi_hand_tracking_desktop.pbtxt" on gpu which is not correct. This graph seems to be designed for running tflite on CPU only. When I instead ran "multi_hand_tracking_mobile.pbtxt" everything worked correctly without adding the ":desktop_offline_calculators" as mentioned above (after adding this I ran into other runtime issues as would be expected seeing as though this graph is not designed for GPU).

[mcclanahoochie]

Yes, the multi_hand_tracking_gpu binary and the multi_hand_tracking_mobile.pbtxt is the correct pairing.
Similarly for most of the *_gpu desktop demos and the *_mobile.pbtxt graphs.

The naming isn't great, but GPU support for desktop is rather experimental and came much after mobile GPU support. This is something we are working to unify in the future.