boto3 not return full list of resources

[leima965]

for example I have 150 asgs

when I run client.describe_auto_scaling_groups without any options
it will only return 100 asgs back.

so I have to run a loop with MaxRecords=100 and if there a NextToken filed then run it again with that token
I don't see this problem with AWS cli

[kyleknap]

You do not see this problem with the CLI because it will automatically do pagination for you if it is available. In boto3 through its client interface, you will have to use paginators. Here is a snippet on how to do this:

In [1]: import boto3

In [2]: client = boto3.client('autoscaling')

In [3]: paginator = client.get_paginator('describe_auto_scaling_groups')

In [4]: for page in paginator.paginate():
   ...:     print(page)
   ...:     
{u'AutoScalingGroups': [], 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '7f916f40-c455-11e5-b61b-c534c89980ac'}}

Then here is how the CLI collects all of the groups into one list:

In [5]: print(paginator.paginate().build_full_result())
{u'AutoScalingGroups': []}

Let me know if you have any more questions or comments

[kyleknap]

Closing. Please reopen if you have anymore questions

[AbrahamHussain]

Hello,

I have a similar problem but I am using the paginator. I am using the paginator to go download files from a large s3 and for some reason it prints out all the keys if I only do that, but it cuts off if I try to download. Any idea?