Mediapipe: How to get unnormalized key points of Hand Tracking?

[hejing-maker]

Dear :
Recently, I am learning Hand Tracking with GPU. I can get the coordinates of key points of gestures by modifying the landmarks_to_render_data_calculator.cc, but I find that the coordinates are normalized. Can I get the unnormalized coordinates,Looking forward to your reply

[bushixingan]

hello, when I learn Hand Tracking with Gpu,i got some questions .Firstly,the tulsi got an error, he told me,An error occurred during the fetch of repository 'local_config_git'...and then when I run the HandTrackingGpu demo ,an error occurred /mediapipe/mediapipe/examples/ios/handtrackinggpu/BUILD:43:1: no such package 'mediapipe/mediapipe/examples/ios': BUILD file not found in any of the following directories.
Could u help me to solve them?

[hejing-maker]

bazel build -c opt --config=ios_arm64 mediapipe/examples/ios/handtrackinggpu:HandTrackingGpuApp

------------------ 原始邮件 ------------------
发件人: "bushixingan"[e-mail];
发送时间: 2019年9月26日(星期四) 上午10:54
收件人: "google/mediapipe"[e-mail];
抄送: "776248249"[e-mail]; "Author"[e-mail];
主题: Re: [google/mediapipe] How to get unnormalized key points of HandTracking? (#134)

hello, when I learn Hand Tracking with Gpu,i got some questions .Firstly,the tulsi got an error, he told me,An error occurred during the fetch of repository 'local_config_git'...and then when I run the HandTrackingGpu demo ,an error occurred /mediapipe/mediapipe/examples/ios/handtrackinggpu/BUILD:43:1: no such package 'mediapipe/mediapipe/examples/ios': BUILD file not found in any of the following directories.
Could u help me to solve them?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[lcycoding]

If you know the original image width and height, maybe project the normalized coords back is the simplest way to get original (x,y)?

[hejing-maker]

The large size of the original picture should be obtained, but I don't know the scale of the picture, how can I solve it

------------------ 原始邮件 ------------------
发件人: "LinCheYu"[e-mail];
发送时间: 2019年9月26日(星期四) 中午11:40
收件人: "google/mediapipe"[e-mail];
抄送: "776248249"[e-mail]; "Author"[e-mail];
主题: Re: [google/mediapipe] How to get unnormalized key points of HandTracking? (#134)

If you know the original image width and height, maybe project the normalized coords back is the simplest way to get original (x,y)?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[bushixingan]

这个还是不行，可以加个qq聊一下吗

[hejing-maker]

776248249

---Original---
From: "bushixingan"<[email protected]>
Date: Thu, Sep 26, 2019 13:54 PM
To: "google/mediapipe"<[email protected]>;
Cc: "hejing-maker"<[email protected]>;"Author"<[email protected]>;
Subject: Re: [google/mediapipe] How to get unnormalized key points of Hand Tracking? (#134)

这个还是不行，可以加个qq聊一下吗

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[fanzhanggoogle]

The large size of the original picture should be obtained, but I don't know the scale of the picture, how can I solve it
…
------------------ 原始邮件 ------------------ 发件人: "LinCheYu"[e-mail]; 发送时间: 2019年9月26日(星期四) 中午11:40 收件人: "google/mediapipe"[e-mail]; 抄送: "776248249"[e-mail]; "Author"[e-mail]; 主题: Re: [google/mediapipe] How to get unnormalized key points of HandTracking? (#134) If you know the original image width and height, maybe project the normalized coords back is the simplest way to get original (x,y)? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

There's a ImagePropertiesCalculator that you can get the size of input. Then you can simply use this stream to calculate the image coordinates of the normalized landmarks. Does that solve your problem?

[hejing-maker]

The large size of the original picture should be obtained, but I don't know the scale of the picture, how can I solve it
…
------------------ 原始邮件 ------------------ 发件人: "LinCheYu"[email protected]; 发送时间: 2019年9月26日(星期四) 中午11:40 收件人: "google/mediapipe"[email protected]; 抄送: "776248249"[email protected]; "Author"[email protected]; 主题: Re: [google/mediapipe] How to get unnormalized key points of HandTracking? (#134) If you know the original image width and height, maybe project the normalized coords back is the simplest way to get original (x,y)? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

There's a ImagePropertiesCalculator that you can get the size of input. Then you can simply use this stream to calculate the image coordinates of the normalized landmarks. Does that solve your problem?

I found the width and height of the input in the Image_Properties_Calculator code as well as the width and height under the gpu, so how can I print it out? For example, I displayed the eighth key point of hand coordinate in landmarks_to_render_data_calculator. Cc with the following code:
const unsigned char lmIndex = 8;
auto* landmarks_text_render = render_data->add_render_annotations();
auto* landmark_text = landmarks_text_render->mutable_text();
std::string dispText = "LM:";
dispText.append(std::to_string(lmIndex));
landmark_text->set_normalized(true);
dispText.append((std::to_string(landmarks[lmIndex].x() + 0.02))+","+(std::to_string(landmarks[lmIndex].y())));
dispText.append(std::to_string(visualize_depth));
landmark_text->set_display_text(dispText);
landmark_text->set_font_height(0.03);
landmark_text->set_baseline(landmarks[lmIndex].y());
landmark_text->set_left(landmarks[lmIndex].x() + 0.02);

[crabbit0629]

I also have the same problem . How do I display the coordinate for each key points?

[AlexYiningLiu]

I am currently running the mediapipe_multi_hands_tracking_aar_example.
Inside the MainActivity class I am able to obtain the normalized x and y coordinates of the landmarks by using getX() and getY(0 on the received NormalizedLandmarks. However, echoing the question above, how can I obtain the coordinates of the original, unnormalized landmarks and use it in my MainActivity? If this is possible, what do these unnormalized coordinates represent? Are they pixels coordinates? Thank you.