Amplify-cli: amplify push returning error like "Template may not exceed 460800 bytes in size". (related to file FunctionDirectiveStack.json)

[clinicalinkgithub]

Describe the bug
When I do amplify push I am getting error like "Template may not exceed 460800 bytes in size".
I am using a lot of lamba resolvers using the @function directive. (see screenshot)
==> The root cause is that the auto-generated file [FunctionDirectiveStack.json] is greater than 460800 bytes. ==> Cloudformation will not allow size greater than this.

Amplify CLI Version
my amplify version is 4.12.0

To Reproduce
create a large schema with a lot of lamba resolvers using the @function directive
==> this is related to the @function directive. (see attached screenshots)

==> similar issues still occur with @connection directive too
==> https://github.com/aws-amplify/amplify-cli/issues/1759 (logged as enhancment)
==> https://github.com/aws-amplify/amplify-cli/issues/2914 (logged as bug)
==> BOTH the @connection and the @function directive can reach the max cloudformation size limit (becuase they do not use the nested-stack logc ...that was fixed for the @model directive

Expected behavior
it should push with no cloudformation issues (similar to the nested stack for @model)
==> https://github.com/aws-amplify/amplify-cli/issues/185
==> you employed this nested-stack fix, but this only fixes @model
The @model has the nested stack fix. ==> the @function and @connection does not have the nested stack fix.

Screenshots

[kaustavghosh06]

@clinicalinkgithub Yes, we don't do do stack splitting for @functions and @connections directive yet since we didn't anticipate the functions and connections stacks to grow as big as the @model stack. Can you please send us your schema to amplify-cli@amazon. I'll put this in our backlog and prioritize with the team.

[alexdoan102]

@kaustavghosh06 has there been any movement on this? we need this fix urgently, because our minification workaround, does not work anymore. we have another issue now:

The FunctionDirectiveStack.json stack defines more than 200 resources. Cause: CloudFormation templates may contain at most 200 resources. How to fix: If the stack is a custom stack, break the stack up into multiple files in stacks/. If the stack was generated, you have hit a limit and can use the StackMapping argument in transform.conf.json to fine tune how resources are assigned to stacks.

we need the stack splitting applied to @functions, and we are currently dead in the water with this new limitation. we cannot continue our development on appysnc without a critical patch for this. please advise.

[parvaurea]

@alexdoan102 - Can you please help me with what minification workarounds you were using. Im also stuck with the same issue currently.

When i try to minify it manually then the size gets with in the limit. But on amplify push the cfn is regenerated and the size increases again. Which leads to failed deployment.

[alexdoan102]

@alexdoan102 - Can you please help me with what minification workarounds you were using. Im also stuck with the same issue currently.

When i try to minify it manually then the size gets with in the limit. But on amplify push the cfn is regenerated and the size increases again. Which leads to failed deployment.

@parvaurea if you use "amplify push", then any changes you make, will get overwritten by the "push" command. You need to use these commands:

1. amplify api gql-compile (this will compile, but not push)
2. minify-json amplify/backend/api/apiservices/build/stacks/FunctionDirectiveStack.json
3. amplify push --no-gql-override (this will push, but not recompile, so that it wont overwrite your changes.)

BTW, you have to install the "minify-json" npm package first.

this is only a temporary fix, sooner or later minification wont work anymore becuase you will hit the "200 resources" limit per file. (see my previous message here). There currently is no workaround for this.

[zakirbaytar]

1. amplify api gql-compile
2. minify-json amplify/backend/api/apiservices/build/stacks/FunctionDirectiveStack.json
3. amplify push --no-gql-override

This works perfectly for manual push. Is there any way I can minify function directive stack before auto builds?

My current build settings:

version: 1.0
backend:
  phases:
    build:
      commands:
        - amplifyPush --simple
frontend:
  phases:
    preBuild:
      commands:
        - yarn
    build:
      commands:
        - yarn build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

I couldn't figure out a way to implement temporary fix to auto builds, so I just left the helper script as it is. Any help is appreciated.

[blairtaylor]

@zakirbaytar We're looking for the same answer. We do dozens of CI builds per day so manual steps are not viable for us. Even if the manual push works, the CI build fails.

@nikhname I know this is not a simple issue. My issue was closed as a duplicate to this issue which has been o/s since Jan 24. Over 4 months. This issue is a show stopper. Could a minify of the schema not be added to the automated builds set up on the Amplify console?

[TheReactGuys]

+1

[TheReactGuys]

Any updates?

[blairtaylor]

Since we often do 30+ CI builds daily, manual workarounds are just not realistic. We are in the process of moving to the Serverless framework as we have to continue to add Lambda functions and this issue is a showstopper for us. I am interested in what others are doing when hitting this limitation when building larger applications. This issue has now been open 5 months, so doesn't look like there will be any solution soon.

[SwaySway]

Hello @blairtaylor
A --minify flag was added on amplify push as of this PR https://github.com/aws-amplify/amplify-cli/pull/3520. Please let us know if this works for you.

[blairtaylor]

@SwaySway thanks! Is this flag available in CLI release 4.21+? Is this documented anywhere?

[blazinaj]

Has anyone been able to use the --minify flag in conjunction with Amplify Console? --minify works great for us in the CLI, but can't figure out how to use it with the console

[blairtaylor]

@blazinaj We have the same question.

@SwaySway if we make this change in the amplify build for the console

amplifyPush --simple --minify

Will this minify the generated templates?

[blazinaj]

I tried putting that in the amplify console build file and it didn't seem to do anything. #742.

[kaustavghosh06]

@blairtaylor @blazinaj Please take a look at the recommendation in the Amplify Console repo out here - https://github.com/aws-amplify/amplify-console/issues/742#issuecomment-648480354

[blazinaj]

The above recommendation worked flawlessly for me, thank you.

[kaustavghosh06]

Hello, CloudFormation just announced a limit increase of the template size to more than double. Here's the launch announcement - https://aws.amazon.com/about-aws/whats-new/2020/10/aws-cloudformation-now-supports-increased-limits-on-five-service-quotas/ and if you're still running into template size limitations after this increase please let us know here or open up a new GitHub issue.