Aws-sdk-js: Implicit `MaxItems` limit in Lambda#listFunctions

[lennym]

Calling Lambda#listFunctions with a MaxItems parameter greater than 50, or no value at all, will still only return the first 50 lambda functions. This is not documented at http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#listFunctions-property

Example:

const lambda = new AWS.Lambda({ region: 'eu-west-1' });
lambda.listFunctions({MaxItems: 100}, (err, data) => {
    console.log(data.NextMarker); // => is not null
    console.log(data.Functions.length); // => 50
});

Furthermore, it appears that the AWS management console is also using these methods (or similar) when creating lists of lambda functions, in particular in the SNS Create Subscription dialogues, which makes it impossible to create a subscription from a lambda function to an SNS topic though the AWS Web UI if that particular lambda is not one of your first 50 lambda functions.

Edit: tested with [email protected]

[lennym]

Note: this does not seem to be replicated in the cli:

$ aws lambda list-functions --max-items 100 | jq '.Functions | length'
51

[LiuJoyceC]

Hi @lennym
Thanks for bringing this to our attention. I will look into why this is happening, as the JavaScript SDK does not have any customizations for Lambda that would restrict MaxItems to a maximum of 50.

[LiuJoyceC]

Hi @lennym
The JavaScript SDK does not add any maximum limits on Lambda's MaxItems, so the Lambda service has defined that limit.

The CLI has a customization that overrides the maximum items imposed by service teams. It essentially uses Paginators under the hood to request more items until the user-specified maximum is reached. We can mark this as a feature request if you would like this to be implemented in the JavaScript SDK as well. If you would like to implement this yourself in your code, you can currently use the eachPage function on the Request object to do something like this:

var results = [];
var maxItems = 100;
var req = lambda.listFunctions();
req.eachPage(function(err, data) {
    if (err) {
        console.log(err);
        return false;
    } else {
        for (var i = 0; i < data.Functions.length; i++) {
            // returning false will stop `eachPage` from requesting more results
            if (results.length >= maxItems) return false;
            results.push(data.Functions[i]);
        }
    }
});

If you don't want the SDK to make multiple requests and would like to receive the specified maximum results in one request, then you can request that the service team raise the maximum limit on the AWS Forums: https://forums.aws.amazon.com/forum.jspa?forumID=186

Hope that helps!

[lennym]

@LiuJoyceC Thanks for your response.

To be honest, the behaviour of the JS SDK isn't really a problem for me, it's reasonably obvious when you see the NextMarker property what has happened, and while it would be good to be documented, it's not a blocker.

The real issue is the manifestation of the same limit in the Web UI for SNS subscriptions, which only show the first 50 available Lambda functions when attempting to add a subscription, with no indication at all that the list is truncated, and no means at all by which to add a subscription for a lambda not in that first set of 50.

I shall raise the same issue in the AWS Forums.

[LiuJoyceC]

@lennym

Thanks for raising the issue on the AWS Forums. Since this isn't a change you are looking for in the SDK, I will remove this as a feature request. Have a good day!

[shawnemullen]

If this isn't going to be "fixed" in the sdk, then I think that at a minimum, the documentation for the sdk should be updated. I've been trying for 2 hours to figure out why I wasn't getting all the functions, since the documentation says, "MaxItems - specifies the maximum number of AWS Lambda functions to return in response." It definitely implies that you can set the maximum number of functions to return, and since you can't, the documentation is wrong.

[leftclickben]

Please implement this feature, or at least update the documentation, as per @shawnemullen comment above, I too spent a lot of time trying to work out why this doesn't work as written in the docs.

It would be useful to be able to list the _latest_ versions rather than starting from the beginning with ancient versions nobody has cared about for a long time.

My use case is a simple script that finds the latest version and updates a specified alias to that version.

[ashwgupt]

Why isn't the API working as expected i.e. if I give a value of Max Item to be 300, I should receive a list of 300 functions back. Whats the point of having this param if one is always restricted to 50 max entries in one listFunctions call?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.