Boto3: botocore.exceptions.DataNotFoundError: Unable to load data for: ec2/2016-09-15/service-2

[asaed]

This issue started happening when we upgraded to boto3 1.4.1. When we try to create a ec2 resource from a boto3 session, we are getting this error
botocore.exceptions.DataNotFoundError: Unable to load data for: ec2/2016-09-15/service-2

The code we use to reproduce this issue is

import boto3
role_arn_to_assume = 'arn:aws:iam::xxxxxxxxxx:role/some-role-name'
sts_client = boto3.client('sts')
assumed_role = sts_client.assume_role(
            RoleArn=role_arn_to_assume,
            RoleSessionName='some-session-name'
 )
new_creds = assumed_role['Credentials']
session_args = {
            'aws_access_key_id': new_creds['AccessKeyId'],
            'aws_secret_access_key': new_creds['SecretAccessKey'],
            'aws_session_token': new_creds['SessionToken'],
}
session = boto3.session.Session(**session_args)
ec2_resource = session.resource('ec2')

When I run this code with 1.4.1, it gives the following error:

Traceback (most recent call last):
  File "/my/folder/workspace/my_script.py", line xxx, in <module>
    ec2_resource = session.resource('ec2')
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/boto3/session.py", line 389, in resource
    aws_session_token=aws_session_token, config=config)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/boto3/session.py", line 263, in client
    aws_session_token=aws_session_token, config=config)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/session.py", line 824, in create_client
    client_config=config, api_version=api_version)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/client.py", line 61, in create_client
    service_model = self._load_service_model(service_name, api_version)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/client.py", line 91, in _load_service_model
    api_version=api_version)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/loaders.py", line 123, in _wrapper
    data = func(self, *args, **kwargs)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/loaders.py", line 358, in load_service_model
    return self.load_data(full_path)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/loaders.py", line 123, in _wrapper
    data = func(self, *args, **kwargs)
  File "/my/folder/my-virtualenv/lib/python2.7/site-packages/botocore/loaders.py", line 382, in load_data
    raise DataNotFoundError(data_path=name)
botocore.exceptions.DataNotFoundError: Unable to load data for: ec2/2016-09-15/service-2

If I run it with previous release 1.4.0, it works fine.

[asaed]

Some other teammates found that this error even happens with simple code like

import boto3 
boto3.resource('ec2') 

the workaround they found was to rewrite that line to be

import boto3 
boto3.resource('ec2', region_name='us-east-1', api_version='2016-04-01') 

[gsmcwhirter]

We have this issue in #843. A quick fix for us was updating botocore to botocore>=1.4.58 (we were on 1.4.56)

[kyleknap]

Yeah this is a duplicate of the linked issue from the comment above. Closing this as a duplicate. Let's keep the conversation to the linked issue.

[monkut]

I had a similar DataNotFoundError with the kinesis client, where the erroring item was endpoints.

  File "/var/task/boto3/session.py", line 263, in client
  aws_session_token=aws_session_token, config=config)
  File "/var/task/botocore/session.py", line 875, in create_client
  endpoint_resolver = self._get_internal_component('endpoint_resolver')
  File "/var/task/botocore/session.py", line 744, in _get_internal_component
  return self._internal_components.get_component(name)
  File "/var/task/botocore/session.py", line 946, in get_component
  self._components[name] = factory()
  File "/var/task/botocore/session.py", line 191, in create_default_resolver
  endpoints = loader.load_data('endpoints')
  File "/var/task/botocore/loaders.py", line 132, in _wrapper
  data = func(self, *args, **kwargs)
  File "/var/task/botocore/loaders.py", line 424, in load_data
  raise DataNotFoundError(data_path=name)
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints

Adding api_version='2016-04-01' to the client instaniation did not help...

[monkut]

In my case I had a separate program that was removing the .json definition files needed by boto3 for client instantiation.

[rex-remind101]

@monkut just received the same error. find any solution?

[monkut]

Sorry my comment wasn't clear.

When I packaged my app, I was running an exclude pattern to remove unnecessary files. What I didn't realize at the time was that boto3 includes some .json definition files with the code, and removing these files (via my packaging script) caused the DataNotFound error.

[rex-remind101]

I see, thank you

[y2k-shubham]

Sorry my comment wasn't clear.

When I packaged my app, I was running an exclude pattern to remove unnecessary files. What I didn't realize at the time was that boto3 includes some .json definition files with the code, and removing these files (via my packaging script) caused the DataNotFound error.

@monkut It is a well-known fact that boto3 generates code on-the-fly (meta-programming) and these generated (temporary) files are stored in same directory where boto3 distribution resides. Because of this, they say, the possibility of bundling boto3 into a zip is completely ruled out.

I'm therefore confused as to why it worked for you in the first place. It maybe that the boto3 utilities that you invoked didn't require creation of any such temp files, but I'm not sure.

See #902