Mediapipe: Accessing landmarks, tracking multiple hands, and enabling depth on desktop

[JECBello]

Hello,

I found out about Mediapipe after seeing Google's blog post regarding hand tracking. Currently, I am working on using Mediapipe to build a cross platform interface using gestures to control multiple systems. I am using the Desktop CPU example as a base for how to move forward, and I have successfully retrieved the hand landmarks. I just want to ensure that I am retrieving them in the most efficient and proper way.

The process I use is as follows:
1) Create a listener of class OutputStreamPoller which listens for the hand_landmarks output stream in the HandLandmark subgraph.
2) If there is an available packet, load the packet into a variable of class mediapipe::Packet using the .Next() method of the OutputStreamPoller class.
3) Use the .Get() method of the Packet class and load into another variable called hand_landmarks.
4) Loop through the variable and retrieve the x, y, and z coordinates and place them into a vector for processing.

Is this process correct or is there a better way to go about retrieving the coordinates of the hand landmarks?

I have additional questions, but I am unsure if I should place them in a separate issue. I will ask them here but please let me know if I should open a separate issue.

1) In the hand tracking examples, only a single hand is to be detected. How would I alter the build such that it can detect multiple hands (specifically 2)?
2) How would I enable the desktop implementations of hand tracking such that they can capture depth (similar to how the android/ios 3D builds can output z coordinates)?

[mgyong]

@JECBello For C++ desktop example, what you described is correct way to get result protos
I have created an example for Object Detection desktop CPU of how to print out detection protos
https://github.com/mgyong/mediapipe-issue200
I copied the demo_run_graph_main.cc into demo_run_graph_main_out.cc

1. Created a listener for detection stream
2. i polled the detection stream kDetectionsStream using Next() for the detection packet
3. Load detection_packet into variable output_detections
4. In my case, the output of the packet is of type std::vector<::mediapipe::Detection>. In your case, it should be landmark proto
   Once you have example of how you do it for hand tracking, appreciate if you could share your implementation with the community

A. For multi-hands, we are working on an example that will hopefully be available in early mid Nov.
B. Can you create a separate issue on the question for desktop implementations of hand tracking such that they can capture depth (similar to how the android/ios 3D builds can output z coordinates)

[alizahidraja]

can you tell us the details for " modify the graph so that it contains multiple branches of hand landmark localization (e.g., two branches for up to 2 hands)" and how to do this?

[Sara533]

@JECBello Hi, could you share your code of getting hand landmarks based on the Desktop CPU example? I also tried to do it but failed.

[shadowver]

I have tested what @mgyong had mentioned with hand_tracking. It built successfully, but failed running and appeared to be some sort of memory lick

[alizahidraja]

@shadowver @Sara533 it runs perfectly on ubuntu 18

[JECBello]

@mgyong Great, thank you for the clarification and example! I will definitely share my implementation when I have time.

[shadowver]

I just found out what went wrong with my code. I coded landmark_packet.Get>() when it really should be landmark_packet.Get>() as the graph takes normalized landmark as input

[alizahidraja]

@chuoling please let me know about the different branch of handmark localization, i have made another branch but it takes a single hand and detects it twice

[mgyong]

@alizahidraja We are looking into releasing something on multi-hands shortly. Pls monitor this thread @eknight7

[pablovela5620]

I created a gist showing how to do this for retrieve hand landmarks (Based on @mgyong example. Thank you!) You can find it HERE

There other important thing to note is that you need to modify the desktop/BUILD and I also added a new hand_tracking/BUILD that look like this

desktop

cc_library(
    name = "demo_run_graph_main_out",
    srcs = ["demo_run_graph_main_out.cc"],
    deps = [
        "//mediapipe/calculators/util:landmarks_to_render_data_calculator",
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework/formats:image_frame",
        "//mediapipe/framework/formats:image_frame_opencv",
        "//mediapipe/framework/formats:landmark_cc_proto",
        "//mediapipe/framework/port:commandlineflags",
        "//mediapipe/framework/port:file_helpers",
        "//mediapipe/framework/port:opencv_highgui",
        "//mediapipe/framework/port:opencv_imgproc",
        "//mediapipe/framework/port:opencv_video",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/port:status",
    ],
)

The important additions here are
"//mediapipe/calculators/util:landmarks_to_render_data_calculator" and "//mediapipe/framework/formats:landmark_cc_proto"

hand_tracking

cc_binary(
    name = "hand_tracking_out_cpu",
    deps = [
        "//mediapipe/examples/desktop:demo_run_graph_main_out",
        "//mediapipe/graphs/hand_tracking:desktop_tflite_calculators",
    ],
)

The build command looks as follows:
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_out_cpu

and the run command is:
bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_out_cpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt

[Sara533]

@mgyong @pablovela5620
Can I print out the hand landmarks of Hand Tracking desktop GPU?
Just using the same method above.
But it seems I need try to get out_landmark from GPU buffer.

[gitBitHubNet]

Thanks to @JECBello and especially @pablovela5620 I reproduced the code for how to printout the hand tracking landmarks.
Yet I have not been able to find anywhere in the official mediapipe site nor in the mediapipe folder a manual describing the mediapipe functions, other than what is mentioned in the readme files and as comment of the functions.

Is there a more analytical documentation of the mediapipe functions and their methods?

[JECBello]

@Sara533 In order to access the hand landmarks from the GPU buffer, you need to use a for loop.
Here is a snippet of the code I used to extract the landmarks, which was in a buffer I had named hand_landmarks, and print them to the console.

for (const auto &landmark : hand_landmarks) 
{ 
    LOG(INFO) << "x coordinate: " << landmark.x(); 
    LOG(INFO) << "y coordinate: " << landmark.y(); 
    LOG(INFO) << "z coordinate: " << landmark.z(); 
}

If anyone knows a more efficient way of accessing the landmark coordinates, please feel free to correct me!

[shadowver]

@JECBello I used the same method to output landmark coordinates, but the z value looks pretty strange to me. I guess because it was normalized. What is your result on the z value? Maybe I need to calibrate the camera first then use PnP to calculate the z value?

[JECBello]

@shadowver I believe I read in another thread that the z-values are determined with respect to the wrist(#156). I'm getting values ranging from -80 to 80 (I might have to double check this). Regarding calibrating the camera and using PnP, I believe @mgyong might be a better person to ask!

[shadowver]

@JECBello Thanks for the quick reply. Ya, I read that thread as well but so far it has not getting any update yet. I will keep an eye on it.

[gitBitHubNet]

@mgyong In the hand tracking example, I have been trying to get the info about the bounding box that surrounds the hand and I have added the following code lines, with respect to the code used to get the hand landmarks (clearly more code is needed, but the problem starts at the include files):

#include "mediapipe/calculators/util/rect_to_render_data_calculator.pb.h"
#include "mediapipe/framework/formats/rect.pb.h"

But I get the complile error:

workmule@workmule-HP-Compaq-6730b-GB991EA-B1A:~/mediapipe$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_nikout_cpu
INFO: Analyzed target //mediapipe/examples/desktop/hand_tracking:hand_tracking_nikout_cpu (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/workmule/mediapipe/mediapipe/examples/desktop/BUILD:54:1: C++ compilation of rule '//mediapipe/examples/desktop:demo_run_graph_main_nikout' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 72 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
mediapipe/examples/desktop/demo_run_graph_main_nikout.cc:33:10: fatal error: mediapipe/calculators/util/rect_to_render_data_calculator.pb.h: No such file or directory
 #include "mediapipe/calculators/util/rect_to_render_data_calculator.pb.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Target //mediapipe/examples/desktop/hand_tracking:hand_tracking_nikout_cpu failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.797s, Critical Path: 0.51s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

Even if I comment-out the first include, the second line does not pass the compilation either.

I see in the bazel-bin and bazel-genfiles folders that these pb.h files do exist, so what is wrong and how is it possible to get the info about the rectangular that surrounds the hand? I would guess that similar to the landmark proto files, the rect would also be available, but clearly I am missing something fundamental in the process.

Also, after a lot of reading on the mediapipe project, I am still puzzled on how to find information describing how to use it. Without any manuals the code is too overwelmingly complicated to understand how to use its functions and methods. Any suggestions?

[shadowver]

@gitBitHubNet did you try adding
"//mediapipe/calculators/util:rect_to_render_data_calculator",
"//mediapipe/framework/formats:rect_cc_proto","
in your BUILD file?

[gitBitHubNet]

Thumbs up to @shadowver.
Indeed I was missing something fundamental...

So to sum up, the correct steps to include the hand rectangular landmarks are:

a. follow the steps described by @pablovela5620 to add the hand landmarks

b. then, in order to add the rect landmarks, add the following lines in your demo_run_graph_main_out.cc file, each in the corresponding place according to @pablovela5620 's additions for the hand landmarks
i. 'include' code lines

#include "mediapipe/framework/formats/rect.pb.h"

note that as it turns out, the lines

#include "mediapipe/calculators/util/landmarks_to_render_data_calculator.pb.h"
#include "mediapipe/calculators/util/rect_to_render_data_calculator.pb.h"

are not necessary in the demo_run_graph_main_out.cc file

ii. in the streams definition section add

constexpr char kLandmarksRectStream[] = "hand_rect";

iii. after the other ASSIGN_OR_RETURN statements add

//hand landmarks_rect stream
  ASSIGN_OR_RETURN(mediapipe::OutputStreamPoller poller_landmark_rect,
            graph.AddOutputStreamPoller(kLandmarksRectStream));

iv. in the packet pollers section add

mediapipe::Packet landmark_rect_packet;
if (!poller_landmark_rect.Next(&landmark_rect_packet)) break;
auto& output_landmarks_rect = landmark_rect_packet.Get<::mediapipe::NormalizedRect>();

v. after printing out the hand landmarks, add the following to printout the rectangular info

//printout the hand rectangle
    std::cout << "Rectangle landmarks: " << "\n";
    std::cout << "x_center:" << output_landmarks_rect.x_center() << "\n";
    std::cout << "y_center:" << output_landmarks_rect.y_center() << "\n";
    std::cout << "height:" << output_landmarks_rect.height() << "\n";
    std::cout << "width:" << output_landmarks_rect.width() << "\n";
    std::cout << "rotation:" << output_landmarks_rect.rotation() << "\n";

c. finally add the following lines to the .BUILD file

"//mediapipe/calculators/util:rect_to_render_data_calculator",
"//mediapipe/framework/formats:rect_cc_proto",

[alxcpa01101]

So are z-values available for the desktop app? I am just getting zero values.

[Sara533]

@alxcpa01101
Please change the model to hand_landmark_3d.tflite in mediapipe/graphs/hand_tracking/subgraphs/hand_landmark_cpu.pbtxt.
Like this:
node {
calculator: "TfLiteInferenceCalculator"
input_stream: "TENSORS:image_tensor"
output_stream: "TENSORS:output_tensors"
node_options: {
[type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] {
model_path: "mediapipe/models/hand_landmark_3d.tflite"
}
}
}

[eknight7]

With the new release https://github.com/google/mediapipe/releases/tag/v0.6.4, we have multi-hand tracking examples:

• https://github.com/google/mediapipe/blob/master/mediapipe/docs/multi_hand_tracking_mobile_gpu.md
• https://github.com/google/mediapipe/blob/master/mediapipe/docs/multi_hand_tracking_desktop.md

[miaoYuanyuan]

@eknight7
hi, before I tune to the 3d model, which I got is 2D point (x,y), when I change to 3d model in subgraph, I also got the zero value in z-axis? (x,y,0), could you help me ? thank you!

[palmloftz]

Hi, Now I already have got x and y from Rectangle landmarks. So I have any question, if I need to get key points from hand, Can I do ? Thank you

[shubham0420]

I did exactly what @gitBitHubNet and @pablovela5620 said. Build was successfull. But i am still getting one more error when running code.

bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_out_cpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt

F0123 23:01:42.267542 11083 packet.h:674] Packet::Get() failed: The Packet stores "mediapipe::NormalizedLandmarkList", but "std::vector<mediapipe::NormalizedLandmark, std::allocator<mediapipe::NormalizedLandmark> >" was requested.
*** Check failure stack trace: ***
    @     0x562502839b6f  (unknown)
    @     0x562502838bbb  (unknown)
    @     0x56250283a3f9  (unknown)
    @     0x5625024f9fcf  (unknown)
    @     0x5625024ea4ce  (unknown)
    @     0x7f62ed1e1b97  __libc_start_main
    @     0x5625024f56da  (unknown)
    @              (nil)  (unknown)

Can someone please resolve this. I don't understand where to make change.

[pablovela5620]

@shubham0420 so this is because mediapipe changed the packet type when they added multi hand support. You have to change

auto& output_landmarks = landmark_packet.Get<std::vector<::mediapipe::NormalizedLandmark>>();

to

auto &output_landmarks = landmark_packet.Get < mediapipe::NormalizedLandmarkList > ();

[shubham0420]

@pablovela5620 great. It worked. But when i print the coordinates. It is giving error. I used below loop :-

for (const ::mediapipe::NormalizedLandmark& landmark : output_landmarks) {
          std::cout << landmark.DebugString();
    }

I have also changed the loop to the one below, but still error persist.

for (const ::mediapipe::NormalizedLandmarkList& landmark : output_landmarks) {
          std::cout << landmark.DebugString();
    }

Can you please help with the last step of implementation.

[ren0d1]

@shubham0420 The .DebugString() function is available for ::mediapipe::NormalizedLandmark not ::mediapipe::NormalizedLandmarkList.

Given that in your code sample the variable "output_landmarks" is of type std::vector<::mediapipe::NormalizedLandmarkList> and the variable "landmark" is of type ::mediapipe::NormalizedLandmarkList.

Then if you want to use the .DebugString() function, you will have to loop over the const ::mediapipe::NormalizedLandmarkList& landmark to retrieve the ::mediapipe::NormalizedLandmark objects from which you can then call the aforementioned function .DebugString().

[ChuyiZhong]

@shubham0420 so this is because mediapipe changed the packet type when they added multi hand support. You have to change

auto& output_landmarks = landmark_packet.Get<std::vector<::mediapipe::NormalizedLandmark>>();

to

auto &output_landmarks = landmark_packet.Get < mediapipe::NormalizedLandmarkList > ();

Hi which file should I make this change?

[shubham0420]

demo_run_graph_main_out.cc

[Shahraizz]

@shubham0420 I am having the same issue as you described, if you solved the problem can you please tell me what i need to change in this code.
auto &output_landmarks = landmark_packet.Get < mediapipe::NormalizedLandmarkList > ();

for (const ::mediapipe::NormalizedLandmark& landmark : output_landmarks) { std::cout << landmark.DebugString(); }

[razak-io]

Hello,

To print the landmarks you could do this:
for (int i = 0; i < output_landmarks.landmark_size(); ++i) { const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i); LOG(INFO) << "x coordinate: " << landmark.x(); LOG(INFO) << "y coordinate: " << landmark.y(); LOG(INFO) << "z coordinate: " << landmark.z(); }

[Shahraizz]

i tried what @pablovela5620 said and it worked for single hand-tracking, is there any way we can do it for multi hand-tracking.

[QBoulanger]

Great work here! I'm also looking for how to get the landmarks with mediapipe. Could you please share the final code for GPU and multi-hand tracking?

It would make us save a huge amount of time... ;-)

[QBoulanger]

multi_hand_tracking_landmarks_gpu.txt
I share here the code I made following your different comments to get the landmarks with multi_hand_tracking on gpu. Hope it will help and save time to some of you!

I put this file with .cc extension in examples/desktop.

I added to desktop/BUILD:

cc_library(
name = "multi_hand_tracking_landmarks_gpu",
srcs = ["multi_hand_tracking_landmarks_gpu.cc"],
deps = [
"//mediapipe/calculators/util:landmarks_to_render_data_calculator",
"//mediapipe/framework:calculator_framework",
"//mediapipe/framework/formats:image_frame",
"//mediapipe/framework/formats:image_frame_opencv",
"//mediapipe/framework/formats:landmark_cc_proto",
"//mediapipe/framework/port:commandlineflags",
"//mediapipe/framework/port:file_helpers",
"//mediapipe/framework/port:opencv_highgui",
"//mediapipe/framework/port:opencv_imgproc",
"//mediapipe/framework/port:opencv_video",
"//mediapipe/framework/port:parse_text_proto",
"//mediapipe/framework/port:status",
"//mediapipe/gpu:gl_calculator_helper",
"//mediapipe/gpu:gpu_buffer",
"//mediapipe/gpu:gpu_shared_data_internal",
],
)

And I added to multi_hand_tracking/BUILD:

cc_binary(
name = "multi_hand_tracking_landmarks_gpu",
deps = [
"//mediapipe/examples/desktop:multi_hand_tracking_landmarks_gpu",
"//mediapipe/graphs/hand_tracking:multi_hand_mobile_calculators",
],
linkstatic=False,
)

My command to build:

bazel-1.0.0 build -c opt --copt -DMESA_EGL_NO_X11_HEADERS mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_landmarks_gpu

My command to run:

./bazel-bin/mediapipe/examples/desktop/multi_hand_tracking//multi_hand_tracking_landmarks_gpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_mobile.pbtxt

If you find any way to improve performances, please let me know!

[RaaghavM]

@QBoulanger Thanks for the concise summary with multiple hand tracking, it was quite helpful!

Does anyone know in what order the landmarks are stored in the landmark list? Visually speaking, in what order do the landmarks correspond to the key-points on the hand?

[ninalutz]

Hi! I tried running the code from @pablovela5620 and keep getting this error:
F0309 12:02:10.959995 231630272 packet.h:682] Packet::Get() failed: The Packet stores "mediapipe::NormalizedLandmarkList", but "std::__1::vector >" was requested.
* Check failure stack trace: *
@ 0x107f2fa6f google::LogMessageFatal::~LogMessageFatal()
@ 0x107f2c189 google::LogMessageFatal::~LogMessageFatal()
@ 0x107b96582 mediapipe::Packet::Get<>()
@ 0x107b52c1f RunMPPGraph()
@ 0x107b54163 main
Abort trap: 6

I've been looking in the demo_run_graph_main_out.cc but I guess I'm still a little confused as to how we could iterate through the mediapipe::NormalizedLandmarkList and output it without vector.

For my purposes I am ideally trying to get the landmark points out in a file so I can do some visualizations with them.

Any recommendations, etc are appreciated! I'm on MacOS

[eknight7]

@ninalutz see this comment and this comment. The code has been updated a while back to output a mediapipe::NormalizedLandmarkList instead of a std::vector<mediapipe::NormalizedLandmark>.

[chillcloud-dev]

@eknight7
Hi!
I want to use mediapipe to extract the joint position of the user's hand and process it later on (after saving them into a txt file or so) and compare with my own code that also extracts the finger joints. (My code is not as fast as mediapipe so i want to switch it)

If I use

for (const ::mediapipe::NormalizedLandmark& landmark : output_landmarks) {
          std::cout << landmark.DebugString();
    }

I get a huge string that i need to process to get the x y for each joint - > but it works !

If i add the following code from this comment

  for (int i = 0; i < output_landmarks.landmark_size(); ++i) { 
    const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i); 
    LOG(INFO) << "x coordinate: " << landmark.x(); 
  LOG(INFO) << "y coordinate: " << landmark.y(); 
  LOG(INFO) << "z coordinate: " << landmark.z(); 
  }

i get the following error:

mediapipe/examples/desktop/demo.cc:185:40: error: 'const class std::vector' has no member named 'landmark_size'; did you mean 'max_size'?
for (int i = 0; i < output_landmarks.landmark_size(); ++i) {
^~~~~
max_size
mediapipe/examples/desktop/demo.cc:187:70: error: 'const class std::vector' has no member named 'landmark'
const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i);
^~~~
Target //mediapipe/examples/desktop/multi_hand_tracking:demo failed to build
Use --verbose_failures to see the command lines of failed build steps.

Could you tell me how I can fix this issue ?

Thanks :)

(I use ubuntu 18)

[eknight7]

@chillcloud-dev There seems to be an issue with the type of output_landmarks. Can you actually share your entire code in a gist somewhere? I want to see how you are getting output_landmarks.

Context: output_landmarks should be a mediapipe::NormalizedLandmarkList which will have a repeated field landmark as defined here: https://github.com/google/mediapipe/blob/master/mediapipe/framework/formats/landmark.proto#L46.
However, from there error

mediapipe/examples/desktop/demo.cc:185:40: error: 'const class std::vectormediapipe::NormalizedLandmarkList' has no member named 'landmark_size'; did you mean 'max_size'?
for (int i = 0; i < output_landmarks.landmark_size(); ++i) {
^~~~~

it seems the code is saying that output_landmarks is an std::vector<NormalizedLandmarkList> which seems to imply the code is collecting a set of landmark list objects, each of which will contain the repeated landmark field. My question is how is the output_landmarks generated in your code, so that we can determine the actual type, and hence the correct way to parse those landmarks. Thanks!

[chillcloud-dev]

@eknight7
thanks for your fast reply. I just followed the upper steps and ended up with my version.
you can find my running file in link . It prints the for each hand the finger joint coordinates.

What I wanted to achieve with the following codesnipplet:
for (int i = 0; i < output_landmarks.landmark_size(); ++i) { const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i); LOG(INFO) << "x coordinate: " << landmark.x(); LOG(INFO) << "y coordinate: " << landmark.y(); LOG(INFO) << "z coordinate: " << landmark.z(); }

is that i want to access every finger joint cleanly without processing the DebugString.
Thanks for your help!

[eknight7]

Here is some code to help you with this @chillcloud-dev: https://gist.github.com/eknight7/d4a57504c8f866fc80c0eb2c61ff6b4f.

[chillcloud-dev]

Is there any way to output the z-coordinate of the landmarks?

right now its always zero and i actually read that they are all normalized relative to wrist. But that would be fine for my application I have in mind.

I am using a realsense D415 camera that gives me good FPS ( currently around 25 FPS )

[Pan-zhaoyu]

Hello,

To print the landmarks you could do this:
for (int i = 0; i < output_landmarks.landmark_size(); ++i) { const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i); LOG(INFO) << "x coordinate: " << landmark.x(); LOG(INFO) << "y coordinate: " << landmark.y(); LOG(INFO) << "z coordinate: " << landmark.z(); }

it works to solve the above problem about the type of Landmark.

[Pan-zhaoyu]

I created a gist showing how to do this for retrieve hand landmarks (Based on @mgyong example. Thank you!) You can find it HERE

There other important thing to note is that you need to modify the desktop/BUILD and I also added a new hand_tracking/BUILD that look like this

desktop

cc_library(
    name = "demo_run_graph_main_out",
    srcs = ["demo_run_graph_main_out.cc"],
    deps = [
        "//mediapipe/calculators/util:landmarks_to_render_data_calculator",
        "//mediapipe/framework:calculator_framework",
        "//mediapipe/framework/formats:image_frame",
        "//mediapipe/framework/formats:image_frame_opencv",
        "//mediapipe/framework/formats:landmark_cc_proto",
        "//mediapipe/framework/port:commandlineflags",
        "//mediapipe/framework/port:file_helpers",
        "//mediapipe/framework/port:opencv_highgui",
        "//mediapipe/framework/port:opencv_imgproc",
        "//mediapipe/framework/port:opencv_video",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/port:status",
    ],
)

The important additions here are
"//mediapipe/calculators/util:landmarks_to_render_data_calculator" and "//mediapipe/framework/formats:landmark_cc_proto"

hand_tracking

cc_binary(
    name = "hand_tracking_out_cpu",
    deps = [
        "//mediapipe/examples/desktop:demo_run_graph_main_out",
        "//mediapipe/graphs/hand_tracking:desktop_tflite_calculators",
    ],
)

The build command looks as follows:
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_out_cpu

and the run command is:
bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_out_cpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt

I did as the above. There's no error. Just I can't get the landmarks.
codes are as below:
for (int i = 0; i const mediapipe::NormalizedLandmark& landmark = output_landmarks.landmark(i);
LOG(INFO) << "X COORDINATE:" << landmark.x();
LOG(INFO) << "Y COORDINATE:" << landmark.y();
LOG(INFO) << "Z COORDINATE:" << landmark.z();
}

Anyone have the idea to solve the problem?

[Akash-Dasyam]

@Pan-zhaoyu same here.. did u find out whats happening here?

edit: I'ma dumbass.. i forgot to use: std::cout

[Pan-zhaoyu]

@Pan-zhaoyu same here.. did u find out whats happening here?

edit: I'ma dumbass.. i forgot to use: std::cout
Hey, I solve this problem by deleting the following code. It seems that this word doesn't show the log file while running.
//google::InitGoogleLogging(argv[0]);

the above code is in the main function.
Hope it's not late to help you.

[Somyarani13]

How can I retrieve the coordinates of the keypoints of fingers in android FOR MULTI-HAND TRACKING? Please let me know. Any suggestions would be helpful.
@Pan-zhaoyu @pablovela5620

[tonywang531]

multi_hand_tracking_landmarks_gpu.txt
I share here the code I made following your different comments to get the landmarks with multi_hand_tracking on gpu. Hope it will help and save time to some of you!

I put this file with .cc extension in examples/desktop.

I added to desktop/BUILD:

cc_library(
name = "multi_hand_tracking_landmarks_gpu",
srcs = ["multi_hand_tracking_landmarks_gpu.cc"],
deps = [
"//mediapipe/calculators/util:landmarks_to_render_data_calculator",
"//mediapipe/framework:calculator_framework",
"//mediapipe/framework/formats:image_frame",
"//mediapipe/framework/formats:image_frame_opencv",
"//mediapipe/framework/formats:landmark_cc_proto",
"//mediapipe/framework/port:commandlineflags",
"//mediapipe/framework/port:file_helpers",
"//mediapipe/framework/port:opencv_highgui",
"//mediapipe/framework/port:opencv_imgproc",
"//mediapipe/framework/port:opencv_video",
"//mediapipe/framework/port:parse_text_proto",
"//mediapipe/framework/port:status",
"//mediapipe/gpu:gl_calculator_helper",
"//mediapipe/gpu:gpu_buffer",
"//mediapipe/gpu:gpu_shared_data_internal",
],
)

And I added to multi_hand_tracking/BUILD:

cc_binary(
name = "multi_hand_tracking_landmarks_gpu",
deps = [
"//mediapipe/examples/desktop:multi_hand_tracking_landmarks_gpu",
"//mediapipe/graphs/hand_tracking:multi_hand_mobile_calculators",
],
linkstatic=False,
)

My command to build:

bazel-1.0.0 build -c opt --copt -DMESA_EGL_NO_X11_HEADERS mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_landmarks_gpu

My command to run:

./bazel-bin/mediapipe/examples/desktop/multi_hand_tracking//multi_hand_tracking_landmarks_gpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_mobile.pbtxt

If you find any way to improve performances, please let me know!

Does this still work for the latest version? I followed your steps and recevied the following error:

ERROR: /home/tony/mediapipe/mediapipe/calculators/tflite/BUILD:401:1: C++ compilation of rule '//mediapipe/calculators/tflite:tflite_tensors_to_detections_calculator' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 118 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from /usr/include/EGL/eglplatform.h:128,
                 from /usr/include/EGL/egl.h:39,
                 from external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/portable_gl31.h:21,
                 from external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:27,
                 from ./mediapipe/util/tflite/config.h:39,
                 from mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:28:
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:63:9: error: expected unqualified-id before 'int'
   63 |   absl::Status Read(absl::Span<T> data) const;
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:67:9: error: expected unqualified-id before 'int'
   67 |   absl::Status Write(absl::Span<const T> data);
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:72:9: error: expected unqualified-id before 'int'
   72 |   absl::Status MappedRead(
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:78:9: error: expected unqualified-id before 'int'
   78 |   absl::Status MappedWrite(
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:81:9: error: expected unqualified-id before 'int'
   81 |   absl::Status MakeView(size_t offset, size_t bytes_size, GlBuffer* gl_buffer);
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:87:9: error: expected unqualified-id before 'int'
   87 |   absl::Status BindToIndex(uint32_t index) const;
      |         ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:116:7: error: expected unqualified-id before 'int'
  116 | absl::Status CopyBuffer(const GlBuffer& read_buffer,
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:119:7: error: expected unqualified-id before 'int'
  119 | absl::Status GetSSBOSize(GLuint id, int64_t* size_bytes);
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:127:7: error: expected unqualified-id before 'int'
  127 | absl::Status CreateReadWriteShaderStorageBuffer(uint32_t num_elements,
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:133:7: error: expected unqualified-id before 'int'
  133 | absl::Status CreateReadOnlyShaderStorageBuffer(absl::Span<const T> data,
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:138:7: error: expected unqualified-id before 'int'
  138 | absl::Status AppendFromBuffer(const GlBuffer& buffer, std::vector<T>* data) {
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:172:7: error: expected unqualified-id before 'int'
  172 | absl::Status CreatePersistentBuffer(size_t size, GlPersistentBuffer* gl_buffer);
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:248:7: error: expected unqualified-id before 'int'
  248 | absl::Status CreateReadWriteShaderStorageBuffer(uint32_t num_elements,
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:262:7: error: expected unqualified-id before 'int'
  262 | absl::Status CreateReadOnlyShaderStorageBuffer(absl::Span<const T> data,
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:275:7: error: expected unqualified-id before 'int'
  275 | absl::Status GlBuffer::Read(absl::Span<T> data) const {
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:288:7: error: expected unqualified-id before 'int'
  288 | absl::Status GlBuffer::Write(absl::Span<const T> data) {
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:299:7: error: expected unqualified-id before 'int'
  299 | absl::Status GlBuffer::MappedRead(
      |       ^~~~~~
external/org_tensorflow/tensorflow/lite/delegates/gpu/gl/gl_buffer.h:315:7: error: expected unqualified-id before 'int'
  315 | absl::Status GlBuffer::MappedWrite(
      |       ^~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:61:26: error: 'tflite::gpu::gl::CreateReadWriteShaderStorageBuffer' has not been declared
   61 | using ::tflite::gpu::gl::CreateReadWriteShaderStorageBuffer;
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./mediapipe/framework/port/status_macros.h:18,
                 from ./mediapipe/framework/packet.h:38,
                 from ./mediapipe/framework/calculator_state.h:30,
                 from ./mediapipe/framework/calculator_context.h:23,
                 from ./mediapipe/framework/calculator_base.h:22,
                 from ./mediapipe/framework/calculator_framework.h:54,
                 from mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:22:
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc: In lambda function:
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:407:20: error: 'CopyBuffer' was not declared in this scope; did you mean 'GpuBuffer'?
  407 |     RET_CHECK_CALL(CopyBuffer(input_tensors[0], gpu_data_->raw_boxes_buffer));
      |                    ^~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:407:5: note: in expansion of macro 'RET_CHECK_CALL'
  407 |     RET_CHECK_CALL(CopyBuffer(input_tensors[0], gpu_data_->raw_boxes_buffer));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:408:20: error: 'CopyBuffer' was not declared in this scope; did you mean 'GpuBuffer'?
  408 |     RET_CHECK_CALL(CopyBuffer(input_tensors[1], gpu_data_->raw_scores_buffer));
      |                    ^~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:408:5: note: in expansion of macro 'RET_CHECK_CALL'
  408 |     RET_CHECK_CALL(CopyBuffer(input_tensors[1], gpu_data_->raw_scores_buffer));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:54: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'Write'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |                                                      ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:9: note: in expansion of macro 'RET_CHECK_CALL'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |         ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:60: error: expected primary-expression before 'float'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |                                                            ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:9: note: in expansion of macro 'RET_CHECK_CALL'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |         ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:60: error: expected ')' before 'float'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |                                                            ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:9: note: in expansion of macro 'RET_CHECK_CALL'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |         ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:416:9: note: in expansion of macro 'RET_CHECK_CALL'
  416 |         RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.Write<float>(
      |         ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:421:13: error: 'CopyBuffer' was not declared in this scope; did you mean 'GpuBuffer'?
  421 |             CopyBuffer(input_tensors[2], gpu_data_->raw_anchors_buffer));
      |             ^~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:420:9: note: in expansion of macro 'RET_CHECK_CALL'
  420 |         RET_CHECK_CALL(
      |         ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:428:52: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'BindToIndex'
  428 |     RET_CHECK_CALL(gpu_data_->decoded_boxes_buffer.BindToIndex(0));
      |                                                    ^~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:428:5: note: in expansion of macro 'RET_CHECK_CALL'
  428 |     RET_CHECK_CALL(gpu_data_->decoded_boxes_buffer.BindToIndex(0));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:429:48: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'BindToIndex'
  429 |     RET_CHECK_CALL(gpu_data_->raw_boxes_buffer.BindToIndex(1));
      |                                                ^~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:429:5: note: in expansion of macro 'RET_CHECK_CALL'
  429 |     RET_CHECK_CALL(gpu_data_->raw_boxes_buffer.BindToIndex(1));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:430:50: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'BindToIndex'
  430 |     RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.BindToIndex(2));
      |                                                  ^~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:430:5: note: in expansion of macro 'RET_CHECK_CALL'
  430 |     RET_CHECK_CALL(gpu_data_->raw_anchors_buffer.BindToIndex(2));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:435:51: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'BindToIndex'
  435 |     RET_CHECK_CALL(gpu_data_->scored_boxes_buffer.BindToIndex(0));
      |                                                   ^~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:435:5: note: in expansion of macro 'RET_CHECK_CALL'
  435 |     RET_CHECK_CALL(gpu_data_->scored_boxes_buffer.BindToIndex(0));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:436:49: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'BindToIndex'
  436 |     RET_CHECK_CALL(gpu_data_->raw_scores_buffer.BindToIndex(1));
      |                                                 ^~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:436:5: note: in expansion of macro 'RET_CHECK_CALL'
  436 |     RET_CHECK_CALL(gpu_data_->raw_scores_buffer.BindToIndex(1));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:442:52: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'Read'
  442 |     RET_CHECK_CALL(gpu_data_->decoded_boxes_buffer.Read(absl::MakeSpan(boxes)));
      |                                                    ^~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:442:5: note: in expansion of macro 'RET_CHECK_CALL'
  442 |     RET_CHECK_CALL(gpu_data_->decoded_boxes_buffer.Read(absl::MakeSpan(boxes)));
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:444:51: error: 'mediapipe::GpuTensor' {aka 'class tflite::gpu::gl::GlBuffer'} has no member named 'Read'
  444 |     RET_CHECK_CALL(gpu_data_->scored_boxes_buffer.Read(
      |                                                   ^~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:444:5: note: in expansion of macro 'RET_CHECK_CALL'
  444 |     RET_CHECK_CALL(gpu_data_->scored_boxes_buffer.Read(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc: In lambda function:
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:20: error: 'CreateReadWriteShaderStorageBuffer' was not declared in this scope
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:5: note: in expansion of macro 'RET_CHECK_CALL'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:55: error: expected primary-expression before 'float'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:5: note: in expansion of macro 'RET_CHECK_CALL'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:55: error: expected ')' before 'float'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:5: note: in expansion of macro 'RET_CHECK_CALL'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:811:5: note: in expansion of macro 'RET_CHECK_CALL'
  811 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:20: error: 'CreateReadWriteShaderStorageBuffer' was not declared in this scope
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:5: note: in expansion of macro 'RET_CHECK_CALL'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:55: error: expected primary-expression before 'float'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:5: note: in expansion of macro 'RET_CHECK_CALL'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:55: error: expected ')' before 'float'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:5: note: in expansion of macro 'RET_CHECK_CALL'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:815:5: note: in expansion of macro 'RET_CHECK_CALL'
  815 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:20: error: 'CreateReadWriteShaderStorageBuffer' was not declared in this scope
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:5: note: in expansion of macro 'RET_CHECK_CALL'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:55: error: expected primary-expression before 'float'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:5: note: in expansion of macro 'RET_CHECK_CALL'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:55: error: expected ')' before 'float'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:5: note: in expansion of macro 'RET_CHECK_CALL'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:818:5: note: in expansion of macro 'RET_CHECK_CALL'
  818 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:20: error: 'CreateReadWriteShaderStorageBuffer' was not declared in this scope
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:5: note: in expansion of macro 'RET_CHECK_CALL'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:55: error: expected primary-expression before 'float'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:5: note: in expansion of macro 'RET_CHECK_CALL'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:55: error: expected ')' before 'float'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:5: note: in expansion of macro 'RET_CHECK_CALL'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:905:5: note: in expansion of macro 'RET_CHECK_CALL'
  905 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:20: error: 'CreateReadWriteShaderStorageBuffer' was not declared in this scope
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:5: note: in expansion of macro 'RET_CHECK_CALL'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:55: error: expected primary-expression before 'float'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:5: note: in expansion of macro 'RET_CHECK_CALL'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:55: error: expected ')' before 'float'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |                                                       ^~~~~
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:5: note: in expansion of macro 'RET_CHECK_CALL'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
./mediapipe/calculators/tflite/util.h:20:25: note: to match this '('
   20 |     const auto status = (call);                            \
      |                         ^
./mediapipe/framework/deps/status_macros.h:87:45: note: in definition of macro 'MP_RETURN_IF_ERROR'
   87 |           status_macro_internal_adaptor = {(expr), __FILE__, __LINE__}) { \
      |                                             ^~~~
mediapipe/calculators/tflite/tflite_tensors_to_detections_calculator.cc:909:5: note: in expansion of macro 'RET_CHECK_CALL'
  909 |     RET_CHECK_CALL(CreateReadWriteShaderStorageBuffer<float>(
      |     ^~~~~~~~~~~~~~
Target //mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_landmarks_gpu failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.945s, Critical Path: 4.77s
INFO: 5 processes: 5 linux-sandbox.
FAILED: Build did NOT complete successfully

[chillcloud-dev]

i recall having similar issues
here are the additional arguments i pass to compile it

-c opt --copt -DBOOST_ERROR_CODE_HEADER_ONLY --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 --cxxopt='-std=c++14'
i had sometimes weird errors and i figured it out that using gcc7.5 and g++8.4 works quite well for me

best luck