Amplify-cli: Cloudformation Parameter Limit AppSync

[pigmanbear]

Amplify CLI version: 0.1.13
OS: macOS 10.13.6

Any non-trivial AppSync application will have may have more than 60 parameters added to the nested api stack, which is beyond the limits of CloudFormation and causing amplify push to fail.

[mikeparisstuff]

This is a great point. My initial thought is that we can get around this by leveraging nested stacks more. If instead of outputting a single stack, the transformer output a stack per @model, for example, we could avoid violating the parameter count limit.

More generally, I think there is a need for a "Serializer" concept in the transform-core. Right now, the graphql-appsync-transformer handles taking the transformer context and writing it out in the correct format which seems wrong. I could see it working like this:

interface TransformerSerializer {
  // Returns the CloudFormation template as a string or buffer.
  serialize(ctx: TransformerContext): string | Buffer
}

// You could then optionally override the default serializer.
const transform = new GraphQLTransform({ transformers: [...], serializer: new CustomSerializer() })

A little off topic but with something like this it would be easier to swap out the logic that writes to one or multiple CF docs (or even terraform in the future) as we run into these constraints.

[pigmanbear]

A stack output for each @model is the direction I was exploring after running into the limit.

So, each model stack would be nested under the primary api stack, which is nested under the primary root? How would you handle the file creation?

As for the broader case, the "Serializer" makes sense, do you see this with just the default creation of per @model stacks unless a "CustomSerializer" is provided?

I was able to go through the source and forked/clone and will play around a bit.

Thanks for the tip on where in the source to start looking.

[mikeparisstuff]

Yeah I was thinking along the same lines. I'll do a little research as well and see if something seems to make more sense than a stack per @model. One thing to think about is that the schema currently has to be a single file (or single CF resource). Thus the primary api stack will have to contain the full schema and will have to deploy before the nested @model stacks which contain the resolvers, datasources, roles, tables etc.

[pigmanbear]

Yes, back in March, I created a deploy process for tearing down and building up resources using CloudFormation/AppSync, et al, etc. (essentially, a lot of what amplify is doing for me now). So, it was great to see amplify as 'infrastructure as code' with AppSync was a pain.

I ran into the 'order of operations' and parameters limit before. I handled with a combination of a CFTs and using Cloudformation through the Node SDK and nested stacks. Never loved the solution, but was working as I needed. I was also using the Serverless Framework, which had its own issues I was working around. We have been using terraform more recently, and just pure CFTs.

Amplify will solve a ton of pain points!

[mikeparisstuff]

I just sent a PR for a short term fix while we are working on the long term solution.

[kaustavghosh06]

Temporary fix pushed with latest npm publish -> 0.1.18

[babu-upcomer]

Has this been fixed for category auth too? I am on Amplify CLI 1.8.1 but still hit with this error,

UPDATE_FAILED               authboopamplifyrn AWS::CloudFormation::Stack Sun Jul 14 2019 03:05:50 GMT+0530 (India Standard Time) Template format error: Parameter count 70 is greater than max allowed 60

[meganwoods]

Hi,

I am on CLI 1.8.2 and after updating auth settings I am unable to push the changes.

"Template format error: Parameter count 61 is greater than max allowed 60"

The first time it pushes ok, any subsequent pushes fail with the above error.

This is the first time I have used amplify..

[kaustavghosh06]

@babu-upcomer and @meganwoods Could you’ll please share the CLI flow/selections from your terminal if you can when adding auth or alternatively share your ‘amplify/backend/auth//parameters.json’ file?

[haverchuck]

@babu-upcomer and @meganwoods: If you post your ‘amplify/backend/auth//parameters.json’ , we don't need the values from this json - just the keys.

[meganwoods]

Hi,

Thanks for responding, I have included the terminal output and the file

amplify/backend/auth//parameters.json
termAndProps.zip

Let me know if you need any more info!

MW

[haverchuck]

@meganwoods Thank you! We will dig into this.

[babu-upcomer]

@kaustavghosh06 I removed the resource and recreated. It didn't happen again. :|

[babu-upcomer]

@kaustavghosh06 I take back. Got this again,

UPDATE_FAILED               authcognito                       AWS::CloudFormation::Stack Fri Jul 19 2019 15:29:46 GMT+0530 (India Standard Time) Template format error: Parameter count 62 is greater than max allowed 60

I counted myself and there are 62 items in Parameters:
cognito-cloudformation-template.zip

"newCallbackURLs": [
        "http://localhost:19006/auth/signin/callback/"
    ],
    "newLogoutURLs": [
        "http://localhost:19006/auth/signout/callback/"
    ],

The above two were added in .json and the corresponding parameters added in .yml too. This is happening when we amplify update auth and even though skip making changes to the callback URLs.

TL;DR
Amplify adds unnecessary parameters even though we skip a lot of configuration without updating. I tried to add a lambda trigger to the existing auth and so had to walkthrough the whole configurations to add one by skipping every other steps. Shouldn't adding a lambda trigger be an explicit option in amplify update auth?

[SebSchwartz]

I got same error:
AWS::CloudFormation::Stack Mon Jul 29 2019 15:11:28 GMT+0200 (CEST) Template format error: Parameter count 78 is greater than max allowed 60

I created auth with a previous version of amplify cli (1.6 if i remember it right). Then updated it with lastest version to have cognito lambda trigger. I just did "amplify update auth" then added one lambda trigger and got this error...

As @babu-upcomer is saying, there is lots of unused params.

The big difference I saw in the update of the json is that the oAuthMetadata param is now splitted in lots of params... I was only one before containing a json, now it's 17 new params for that and it's still there... So maybe there is something to dig there?

Also, I didn't touch anything else of the config but "useDefault": "defaultSocial" is now: "useDefault": "manual"

[thibauld]

+1 this issue ultimately got us to abandon using Amplify :/

[SebSchwartz]

@haverchuck Isn't it a good idea to create a new issue for this as this one is in closed state? Or reopen this one?

[kaustavghosh06]

@thibauld Are you referring to the auth parameter limit issue? We merged the PR for the same out here - https://github.com/aws-amplify/amplify-cli/pull/1948 and will be part of our next npm release.

[SebSchwartz]

Great @kaustavghosh06 :)

[shlomizadok]

@thibauld Are you referring to the auth parameter limit issue? We merged the PR for the same out here - #1948 and will be part of our next npm release.

Thank you so much @kaustavghosh06 - any timeline for next npm release?

[shlomizadok]

Just updated to 1.11.0.
Ran amplify update auth again and then amplify push.
Works like charm.
Thanks ❤️