Amplify-cli: IAM policy size exceeded

[oliverandersencox]

I just created a fresh amplify project.

1. Amplify init
2. amplify add api
3. amplify push
4. amplify add api
5. amplify push / amplify push api < errors here

CREATE_FAILED AuthRolePolicy AWS::IAM::Policy Thu Nov 07 2019 20:31:40 GMT+0200 (South Africa Standard Time) Maximum policy size of 10240 bytes exceeded for role amplify-culture-app-dev-163322-authRole (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: e116276c-a03f-469a-b0f5-e0ac1fc43460)
CREATE_FAILED UnauthRolePolicy AWS::IAM::Policy Thu Nov 07 2019 20:31:39 GMT+0200 (South Africa Standard Time) Maximum policy size of 10240 bytes exceeded for role amplify-culture-app-dev-163322-unauthRole (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: 93255697-fe0d-4f4f-a085-69a63574d832)

I simply copied the schema from my working project and added it into the new one. I can't event see the policy amplify creates to see why it is too long.

Most models look similar to this:

`type City @model 
@key(name: "cityByName", fields: ["title"], queryField: "findCityByName")
  @auth(rules: [
    { allow: groups, groups: ["admin"] },
    { allow: groups, groups: ["users"], operations: [read] },
    { allow: public, provider: iam, operations: [read] }
  ])
@searchable {
  id: ID!
  title: String!
  description: String
  country: String
  continent: String
  location: Location 
  timezoneOffset: Float
  currency: String
  images: [String]
  createdAt: Float!
  updatedAt: Float!
}
`

any ideas as this is a blocker... again.

[kaustavghosh06]

@oliverandersencox Do you just have one AppSync API in your project and you're seeing the IAM policy limit getting exceeded?

[oliverandersencox]

yes, I just created the one api after a fresh init of a new amplify project

[kaustavghosh06]

Okay, i'm trying to reproduce this right now. Is the schema size correct or do you have more types?

[oliverandersencox]

my schema is very large, with over 30 types.

[kaustavghosh06]

@oliverandersencox Okay, if you have 30 types then I won't be able to reproduce it with the schema you've given above. I was able to push the template that you've given above.
But having said that , IAM service has a limit for 10240 bytes with the policy size which would increase in size as you add more IAM rules in the transformer model types.
You might want to open up a ticket with AWS support to check if you can lift this limit for your app. In the meanwhile I'm trying to get more information around it from our end.

IAM limitations - https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html

[oliverandersencox]

@kaustavghosh06 thanks for your help. Much appreciated.

[warrenmcquinn]

I'm having this same issue, and I've opened up a support forum discussion here.

On the CLI side of things, it would be beneficial if the authrole & unauthrole policies' lengths could be determined before amplify push fails to create the policies due to the length limit.

Another idea would be to generate & attach managed policies for the @models that use @auth({provider: iam}), instead of inline.

[houmark]

@kaustavghosh06 This seems to be an issue a lot of people are discovering, and AWS seems to be very silent about a solution or timeline. This is a duplicate of #2084 where more people are affected.

The solution seems to be that the CLI is generating and maintaining a managed policy just as @warrenmcquinn mentions. I don't understand why that seems to such a big issue for the CLI team to get to that and why they did not do it like that as part of the Multiauth feature. Other CLI features are already creating managed policies to the best of my knowledge so there should already be some code available to do it here also.

Having people try to raise limits either for their account or in IAM, in general, seems like a bad excuse for not solving this in Amplify when there, in fact, is a solution. Please correct me if a managed policy generated and attached by the CLI is not doable.

[kaustavghosh06]

@houmark I'd commented on this issue 24 hours back and our team has been looking into this for the last couple of days. We are definitely not trying to be silent about it. There are moving pieces here that need to be thought out in detail. The solution mentioned by @warrenmcquinn has limitations since there's a max of 10 IAM Managed policy that you can attach to any role. So if we have a managed policy per model as recommended by @warrenmcquinn - you'd have max limit of 10 models (which can have IAM auth on it), which is again a limitation.

Also, each of the managed policy also has a limit of 10,240 bytes, so we cannot group the policies into one role. We can strategize and enforce some bucketing mechanisms to fill in policies dynamically into the 10 IAM managed policies which we can attach to the auth and unauth roles, but still it would be easy for someone with a large number of models and field level authorizations with IAM to surpass this number.

In my response above, I'd mentioned that until we find out and release a solution as a potential workaround, was to raise a AWS support request for a increase in the IAM policy limits and in no ways was an excuse to not solve the problem. Our team tries really hard to address issues and concerns and sometimes issues are non-trivial like this one and it takes more time, thought, and testing

Also, for other categories we don't generate managed policies - and they're inline as well, so managed policies would be something new for the CLI. We didn't anticipate the policy size exceeding since we didn't have test cases which would exceed the IAM policy size limit.

[houmark]

Thank you!

In your last comment, you did not mention that this was being actively worked on, or solutions considered. On the other issue, nobody has commented in any way recently, so while you may feel from your end you are not being silent about it, I think you can tell by the amount of comments already raised by your customers, that people are eagerly awaiting any type of feedback.

The only potential solution mention, was to request a limit, which I am honestly not sure how to do and if that is even possible.

While I can understand test cases were not made to catch this, may I suggest that your team look into having a real-life web app with not just 1-2 models in order to be able to catch more of these neverending limitations on the AWS platform. Our app is in my opinion considered fairly small, so if we hit this, then I think 90% of your user base will sooner or later (if they use the multi auth features of course).

It seems to me that the Amplify team is trying to make the most granular, flexible and configurable setup, which I definitely appreciate, but you are doing this up against an existing platform full of limitations, without doing your proper research before, which then leads to these situations AFTER the World has started using them in production. Your competitors may have similar challenges, but they solve it quickly by quickly patching the core of the problem (by raising the limit globally) while you spend a huge amount of developer capacity to work around the limitation in the best way possible. This is not the first case of serious limitations.

We've had several of similar limitations before (which most do not have a solution at the time of this writing), and I am starting to doubt if it's good for us to continue building a critical platform on top of such a limited platform. If AWS wants to be in this game, then they need to get out of 2007 where they set limits like this. Frankly 10 Kilobyte in a modern platform is nothing, especially considering how explicit these policies and roles need to be due to the AWS format.

[houmark]

Also, for the record, so we do not spend time emailing someone that has no clue how to help, can you please provide the correct email we should email to attempt to get this limit raised until the CLI team can come up with a long term solution?

[kaustavghosh06]

@houmark Here's information as to how to can request AWS service support for cases like limit increases - https://docs.aws.amazon.com/awssupport/latest/user/getting-started.html

[warrenmcquinn]

@kaustavghosh06 Unfortunately, it appears that the quota for "Inline policies total length per role" is not adjustable:

https://console.aws.amazon.com/servicequotas/home?region=us-east-1#!/services/iam/quotas/L-D6222C68

However, the quota for "Managed policies per role" _is_ adjustable:

https://console.aws.amazon.com/servicequotas/home?region=us-east-1#!/services/iam/quotas/L-3C312957

[kaustavghosh06]

@warrenmcquinn Got it. Yes, we've in communication with the IAM service as well and with managed policy there is some flexibility around quotas and limits as you've mentioned.
We're planning on working towards moving the policies (at-least for AppSync due to its verbosity) from in-line to managed policy for this flexibility.

[oliverandersencox]

@kaustavghosh06 any ETA on this. I contacted support and was told they couldn't increase the policy size limit which puts the project on hold again

[undefobj]

@oliverandersencox sorry that support wasn't able to address the limit issue. We've got a team member looking into the implementation mentioned by @kaustavghosh06 above. I don't currently have an ETA but let me see if we can get crisper with some timelines in the next couple days. We appreciate your patience here.

[diego-palmeira]

Could you provide a workaraound for this issue? I just got it in the final dates of an important project for my company, and it's a blocking point for us. We are using Amplify and ApiGateway, Cognito and lambdas.
Do you have a concrete date for resolution?
Thank you,

[warrenmcquinn]

I've updated to the most recent Amplify CLI release (4.2.0) and when I attempt to amplify push an API update with new @model types, I'm still receiving the error:

UPDATE_FAILED               AuthRolePolicy                                        AWS::IAM::Policy            Mon Dec 02 2019 12:44:26 GMT-0600 (Central Standard Time) Maximum policy size of 10240 bytes exceeded for role redacted-authRole (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded...

@attilah @yuth Is there something that I need to do so that my existing API resource can utilize the new Managed Policy flow?

[yuth]

The fix is not released yet. It will be in the next release

[undefobj]

Wanted to follow up - this was released in the latest version of the CLI. Please do update and let us know how things are going.

[warrenmcquinn]

Works great for us. Thanks AWS Amplify CLI / FOSS team 👍

[nujhong]

I encountered this error after upgrading cli to 4.6.0

Resource Name: AuthRolePolicy01 (AWS::IAM::ManagedPolicy)
Event Type: create
Reason: Cannot exceed quota for PolicySize: 6144 (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: 3996646e-2109-4f1a-b088-5cbecf7f0c5d)

[attilah]

@nujhong could you please send us these to [email protected]:

• graphql schema
• amplify-meta.json
• built cloudformation file for the API: cloudformation-template.json

[attilah]

Reopening as it turned out that certain region names could cause problems with policy size calculations as the additional overhead used for calculation was small.

[lcx1204]

Hi @attilah @undefobj , I started a project with amplify-cli version 4.19 and then ran into the problem "maximum policy size of 10240 bytes exceeded". After reading through this thread, I upgraded amplify cli to version 4.20.0 today, but it is still giving out the same error when running "amplify push". Can you please advise how I should go about fixing it? thanks

[varunpsr]

Hello @attilah @undefobj, we're the facing the same issue when trying to amplify push our schema changes.
We have 51 types in our schema and 27 lambda functions as resolvers.
We added our 27th function today and that's what caused this issue for us.