Python-docs-samples: make_iap_request.py update seems to have broken the Launching Dataproc Jobs with Cloud Composer Qwiklab

[0yster]

In which file did you encounter the issue?

make_iap_request.py

The Qwiklab Launching Dataproc Jobs with Cloud Composer uses the make_iap_request.py file, which now no longer works. As a result the lab is currently offline.

Did you change the file? If so, how?

When a change was made by @arithmetic1728, sijunliu@ to this file, a Qwiklab that uses the file no longer works.
https://github.com/GoogleCloudPlatform/python-docs-samples/commit/5ef84ba176a1ab5a8259cbf1382fa76edbdf7842

Describe the issue

error:
(venv) student_00_da8d23ee74ab@cloudshell:~/professional-services/examples/cloud-composer-examples/composer_http_post_example (qwiklabs-gcp-00-68bd727eb07f)$ python dag_trigger.py \

--url="${AIRFLOW_URI}/api/experimental/dags/average-speed/dag_runs" \
--iapClientId=$CLIENT_ID \
--raw_path=gs://$PROJECT/cloud-composer-lab/raw-$EXPORT_TS/
Traceback (most recent call last):
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 289, in _call_meta
data_identity_endpoint
self._target_audience
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 172, in get
response,
google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=1087176421767-er001vugel6q9ottl7ihs8rp6963iv6o.apps.googl
eusercontent.com&format=full from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", )
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "dag_trigger.py", line 100, in
main()
File "dag_trigger.py", line 96, in main
data=json.dumps(payload))
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/make_iap_request.py", line 44, in make_iap_request
google_open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/oauth2/id_token.py", line 213, in fetch_id_token
credentials.refresh(request)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 312, in refresh
self.token, self.expiry = self._call_metadata_identity_endpoint(request)
File "/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 294, in _call_meta
data_identity_endpoint
six.raise_from(new_exc, caught_exc)
File "", line 3, in raise_from
google.auth.exceptions.RefreshError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=1087176421767-er001vugel6q9ottl7ihs8rp6963iv6o.apps.googleu
sercontent.com&format=full from the Google Compute Enginemetadata service. Status: 404 Response:\nb''", )

[arithmetic1728]

@0yster I think I figured out the problem. Could you help me verify if the fix works for you?

(1) uninstall google-auth

python -m pip uninstall google-auth

install google-auth with the proposed fix

python -m pip install git+https://github.com/googleapis/google-auth-library-python.git@66fa9436bd73bac3afca345a70fbf4a99e3cfcb2#egg=google-auth

(2) run python dag_trigger.py again and see if that fixes the problem?

[0yster]

That looks like the solution! The lab seems to be working correctly now!

Can you please let me know when the google-auth library is updated? The lab
always pulls from the library each time a student runs it, so once it's
ready, the lab is fixed.

Thank you so much for your quick response.

JoEllen Yannis

Technical Curriculum Developer, Self-paced labs

On Fri, May 8, 2020 at 4:41 PM arithmetic1728 [e-mail]
wrote:

@0yster https://github.com/0yster I think I figured out the problem.
Could you help me verify the fix?

(1) open the file
/home/student_00_da8d23ee74ab/professional-services/examples/cloud-composer-examples/composer_http_post_example/venv/lib/python3.7/site-packages/google/oauth2/id_token.py
(2) change line 215 to except (ImportError, exceptions.TransportError,
exceptions.RefreshError):
(3) run python dag_trigger.py again and see if that fixes the problem?

If it works I will fix it in google-auth library and do a new release,
then you can update your google-auth library. Thanks.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/python-docs-samples/issues/3712#issuecomment-626010981,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIT6OXJ4Z2WCC5HFWJCUFM3RQRU6HANCNFSM4M4N2HFQ
.

[arithmetic1728]

@0yster I will release google-auth version 1.14.3 asap (today or tomorrow) and let you know once it is released. Thank you for confirming the solution works!

[leahecole]

@arithmetic1728 thanks for the fix! Does this script need to be updated or is the google-auth update sufficient?

[arithmetic1728]

@leahecole I also need to update the google-auth version in requirements.txt. Could you take a look at https://github.com/GoogleCloudPlatform/python-docs-samples/pull/3726?

[leahecole]

merged!

[arithmetic1728]

@leahecole @busunkim96 Thank you Bu Sun and Leah!

@0yster The issue is solved (please use google-auth 1.14.3).