Botocore: Consistent documentation for the ClientError

[kevinburke]

The ClientError is now (I think?) the gateway to most AWS service exceptions. The documentation around it is not very clear.

Here, we're checking ['Error']['Code'] for the string 'InvalidInstanceID.NotFound':

from botocore.exceptions import ClientError

ec2 = session.get_client('ec2', 'us-west-2')
try:
    parsed = ec2.describe_instances(InstanceIds=['i-badid'])
except ClientError as e:
    logger.error("Received error: %s", e, exc_info=True)
    # Only worry about a specific service error code
    if e.response['Error']['Code'] == 'InvalidInstanceID.NotFound':
        raise

Here we're checking it for the string "404". Note int() will throw if any other error is present in that field.

import botocore
bucket = s3.Bucket('mybucket')
exists = True
try:
    s3.meta.client.head_bucket(Bucket='mybucket')
except botocore.exceptions.ClientError as e:
    # If a client error is thrown, then check that it was a 404 error.
    # If it was a 404 error, then the bucket does not exist.
    error_code = int(e.response['Error']['Code'])
    if error_code == 404:
        exists = False

• Can you provide more guidance on what the field in Error.Code looks like, and how this matches up with exceptions mentioned (but not linked ) in the documentation like LimitExceededException?
• Is Error.Code guaranteed to always exist for a ClientError, or do I need another level of try/catch error wrapping around that?
• What other fields exist on ClientError.response, besides Error ?

[jamesls]

• The error code will match the errors returned by the service, so the code in that example would be LimitExceededException.
• The error code will always exist. If it doesn't, it's a bug.
• The other top level key is ResponseMetadata, which can have a RequestId.

I'll get the docs updated with more info on this, I agree this should be documented.

[lenucksi]

👍

Also a clear documentation of every possible exception dict content at best linked from or included at the location where the said content may occur would be very helpful.

I would also find it very appreciable if you could provide proper, exception classes per service and exception instead of the dict approach. That would allow to use try in a more precise, pythonic way without catching the ClientError blank and then wading around in Exception details.

[crowdwave]

How do I work out what ClientError will be when a head_object is not found?

I can't find any documentation that tells me this.

thanks

[markddavidoff]

If anyone else ends up here looking for a list of error codes, the ClientError.response['Error']['Code'] error codes come from the aws sdk and can be found here

[kerma]

@crowdwave I'm years late, but I stumbled upon the same problem just now. Took me a while to figure it out, so leaving here for other poor souls.

S3 head_object is a tricky request, because no response body is returned. That means the error codes listed on AWS docs are not present in the S3 response. It seems botocore falls back to using HTTP status code parsing. The raised exception's response attribute is a dict looking something like this:

{'Error': {'Code': '412', 'Message': 'Precondition Failed'}

I have a piece of code which used to work fine:

try:
   resp = s3.head_object(Bucket=bucket, Key=key, IfMatch=etag)
except ClientError as e:
     if e.response['Error']['Code'] == 'PreconditionFailed':
         ...

So something definitely has changed at some point. Not sure it was S3 service itself or botocore.

It can be verified with awscli:

$ aws s3api head-object --bucket my-bucket --key missing.json
An error occurred (404) when calling the HeadObject operation: Not Found

$ aws s3api head-object --bucket my-bucket --key somekey.json --if-match not-matching-etag
An error occurred (412) when calling the HeadObject operation: Precondition Failed

$ aws s3api get-object --bucket my-bucket --key somekey.json --if-match not-matching-etag output.json
An error occurred (PreconditionFailed) when calling the GetObject operation: At least one of the 
pre-conditions you specified did not hold

[swetashre]

This documentation contains details about how to handle error with boto3
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/error-handling.html

If anyone has any concerns please reopen a new issue and we would be happy to help.