Google-api-python-client: Expose HttpError.reason when available

[achantavy]

Is there a better way to get the error reason than parsing HttpError.content from json string to object?

This is what I'm currently doing:

try:
    req = compute.zones().list(project=project_id, maxResults=max_results)
    res = req.execute()
    return res['items']
except HttpError as e:
    reason = json.loads(e.content)['error']['errors'][0]['reason']
    if reason == 'accessNotConfigured':
        <do the thing>

Ideally I'd like to avoid the call to json.loads().

[busunkim96]

Hi @achantavy, I believe you can get that through the error_details attribute on HttpError.

try:
    req = compute.zones().list(project=project_id, maxResults=max_results)
    res = req.execute()
    return res['items']
except HttpError as e:
    reason = e.error_details
    if reason == 'accessNotConfigured':
        <do the thing>

https://github.com/googleapis/google-api-python-client/blob/1d2e240a74d2bc0074dffbc57cf7d62b8146cb82/googleapiclient/errors.py#L34-L77

[achantavy]

Thanks for your reply!

error_details seems to be an empty string though. Here's a dump of the public members of the HttpError object that I get back; the reason field that I am interested in seems to be embedded in the content byte array:

exc.args = ({'vary': 'Origin, X-Origin', 'content-type': 'application/json; {xss-protection-stuff};}', 'transfer-encoding': 'chunked', 'status': '403', 'content-length': '986', '-content-encoding': 'gzip'}, b'{\n "error": {\n  "errors": [\n   {\n    "domain": "usageLimits",\n    "reason": "accessNotConfigured",\n    "message": "Access Not Configured. Compute Engine API has not been used in project {ProjectNumber} before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber} then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",\n    "extendedHelp": "https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber}"\n   }\n  ],\n  "code": 403,\n  "message": "Access Not Configured. Compute Engine API has not been used in project {ProjectNumber} before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber} then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry."\n }\n}\n')
exc.content = b'{\n "error": {\n  "errors": [\n   {\n    "domain": "usageLimits",\n    "reason": "accessNotConfigured",\n    "message": "Access Not Configured. Compute Engine API has not been used in project {ProjectNumber} before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber} then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",\n    "extendedHelp": "https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber}"\n   }\n  ],\n  "code": 403,\n  "message": "Access Not Configured. Compute Engine API has not been used in project {ProjectNumber} before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project={ProjectNumber} then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry."\n }\n}\n'
exc.error_details = ''
exc.resp = { '{xss-protection-stuff}' : '', 'transfer-encoding': 'chunked', 'status': '403', 'content-length': '986', '-content-encoding': 'gzip'}
exc.uri = 'https://www.googleapis.com/compute/v1/projects/{ProjectName}/zones/{ZoneName}/instances?alt=json'
exc.with_traceback = <built-in method with_traceback of HttpError object at 0x1234>

The approach in my original post takes the byte array exc.content, uses the json module to load it as a dict, and then gets the reason field.

It'd be more convenient if this field was exposed in the Python SDK as a member of HttpError so I could write something like

if exc.reason == 'accessNotConfigured':
    {do something}

[Ark-kun]

@busunkim96 I'm not sure this is fixed.

[busunkim96]

@Ark-kun Do you have an example?

[wonderchang]

I agree with @achantavy's idea mentioned in https://github.com/googleapis/google-api-python-client/issues/662#issuecomment-494187832

The approach in my original post takes the byte array exc.content, uses the json module to load it as a dict, and then gets the reason field.

For example, in my case, I don't want to repeat our self-implemented retrying mechanism when encountering the specific error, like "403 This file is too large to be exported". But the current API behaves poorly like below,

>>> e
<HttpError 403 when requesting https://www.googleapis.com/drive/v3/files/.../export?mimeType=application%2Fpdf&alt=media returned "This file is too large to be exported.">
>>>
>>> e.resp.reason
'Forbidden'

if I want to get the actual reason value, I need to parsing that error data (content) by myself

>>> e.content
b'{\n "error": {\n  "errors": [\n   {\n    "domain": "global",\n    "reason": "exportSizeLimitExceeded",\n    "message": "This file is too large to be exported."\n   }\n  ],\n  "code": 403,\n  "message": "This file is too large to be exported."\n }\n}\n'
>>>
>>> json.loads(e.content)['error']['errors'][0]['reason']
'exportSizeLimitExceeded'

Ideally, the property APIs of HttpError should have the same mapping fields of the data format defined in here. For the application, reason acted as a code name which is the best reference to determine the situation what you want to do in your code. So implementing the HttpError.reason is necessary.

I think this issue need to be reopened.

[busunkim96]

Hi,

Thanks for the additional context.

I'm not sure how consistent APIs are about defining errors. The YouTube Data API and Drive API seem to have a similar structure.

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "invalidParameter",
    "message": "Invalid string value: 'asdf'. Allowed values: [mostpopular]",
    "locationType": "parameter",
    "location": "chart"
   }
  ],
  "code": 400,
  "message": "Invalid string value: 'asdf'. Allowed values: [mostpopular]"
 }
}

```json
{
"error": {
"code": 400,
"errors": [
{
"domain": "global",
"location": "orderBy",
"locationType": "parameter",
"message": "Sorting is not supported for queries with fullText terms. Results are always in descending relevance order.",
"reason": "badRequest"
}
],
"message": "Sorting is not supported for queries with fullText terms. Results are always in descending relevance order."
}
}

The [Analytics API](https://developers.google.com/analytics/devguides/reporting/core/v4/errors) is different - the `status` seems to be the equivalent of `reason`.

```json
{
 "error": {
  "code": 403,
  "message": "User does not have sufficient permissions for this profile.",
  "status": "PERMISSION_DENIED"
 }
}

[wonderchang]

@busunkim96 thanks for your response.

The Analytics API is different ...

I would like to say the point you mentioned above is also important. The diversity of overall APIs should be taken into account in designing. Also the errors is an array structure which indicates there are possibly multiple error reasons. It's truly difficult to have a unified API to cover all the cases.

[busunkim96]

I spoke to some other folks today, and I don't think there's a error response structure we can count on across all the APIs Google publishes.

If you need a particular field in an error response, parsing it from the JSON is the best option.

[Ark-kun]

@Ark-kun Do you have an example?

I think I was creating a Cloud Function. I wanted an easier way to get the status and reason of HTTPError.