Aws-cli: --no-paginate seems to have the opposite behavior

[daveyarwood]

According to e.g. aws autoscaling describe-auto-scaling-instances help:

_describe-auto-scaling-instances is a paginated operation. Multiple API calls may be issued in order to retrieve the entire data set of results. You can disable pagination by providing the --no-paginate argument. When using --output text and the --query argument on a paginated response, the --query argument must extract data from the results of the following query expressions: AutoScalingInstances_

However, the behavior seems to be backwards:

$ aws autoscaling describe-auto-scaling-instances | wc -l
2020
$ aws autoscaling describe-auto-scaling-instances --no-paginate | wc -l
455

Including --no-paginate seems to produce less output, whereas I would expect it to produce more, i.e. all of the results.

This is on aws-cli version 1.11.47.

[dstufft]

The APIs in question are, as I understand it, always paginated. The --no-paginate option simply disables the CLIs mechanism to automatically retrieve the entire data set by loading every page of results behind the scenes.

[daveyarwood]

Another thing we noticed is that, when including --no-paginate, there is a NextToken field in the results (seemingly indicating that they are paginated), whereas without --no-paginate, there is no NextToken field.

[kyleknap]

So in general for AWS API's, they will return a next token to indicate if there are more results to return. If you call --no-paginate, it will return the response from a single API request, which will have a next token if not all of the resources are returned in that single response. The CLI by default (and possible) will make the subsequent API calls for you and since it continues paginating until there are no more resource left to list, it will not include a next token in the response displayed because it listed through all of the resources.

Let us know if this makes sense or if you have any more questions.

[daveyarwood]

I see -- so --no-paginate is for when you specifically only want to make one API call, which may or may not yield all of the results?

[kyleknap]

Yes exactly that.

[daveyarwood]

OK, that makes sense. Thank you for the explanation!

[yikaus]

Hi ,
I think it is still an issue from results perspective, as with —no-paginate option it actually paginate result and return with paginated page1.

With this option name people would think cli by default paginated the result and need this option to do opposite.

Should rename this option something like —paginate

[daveyarwood]

I do still think the option is confusingly named. I had to re-read this thread to recall how it works. It isn't as simple as "--no-paginate really means --paginate." The fact is that there is _always_ pagination at play because that's how the underlying AWS API works.

By default with the AWS CLI, you get the _illusion_ that there is no pagination, but really there are pages being fetched behind the scenes, and you get all the results at the end.

When you supply --no-paginate, the AWS CLI makes a single request to the API, returning a single page of results, and gives you the next token so that you can fetch more results. The --no-paginate flag actually disables auto-pagination.

As such, maybe it should be called --first-page-only or --no-autopaginate, or something like that that more accurately reflects its purpose.