Copilot-cli: AccessDenied sts:AssumeRole

[sirbully]

Hi, I'm new to AWS and I also don't have full access to our company's AWS account but I've been instructed to use copilot. I keep getting this error when I try to deploy:

AccessDenied: User: {user} is not authorized to perform: sts:AssumeRole on resource: {role}:{appname}-dev-EnvManagerRole

What permissions should I ask my boss to grant my user account?

[bvtujo]

Good morning @sirbully, when you're deploying with Copilot the credentials you use will need to have sts:AssumeRole permissions on the EnvManagerRoles for any environments you're authorized to deploy to.

Copilot's credential story is structured around a central administration account and environment accounts. The application account needs broad permissions, as it is responsible for creating environments, vpcs, pipelines, etc, but access can be restricted at the environment level. To deploy to any given environment as a developer, all you need is to be able to call sts:AssumeRole on the environment's EnvManagerRole. This role is created by Copilot and contains all the permissions needed to deploy and operate Copilot services, with no write-level permissions on any environment resources.

Does that help?

[sirbully]

Hello @bvtujo, Thanks so much for replying!

I solved the problem by editing the trusted entities under EnvManagerRole. It had something along the lines of arn:aws:iam::111122223333:root which I manually updated to arn:aws:iam::111122223333:user/<username>. Afterwards, when I run copilot svc deploy, I no longer had the access denied error.

Please let me know if I have made a catastrophic mistake by doing this. I feel like with copilot, I shouldn't be tinkering around the console at all.

[kohidave]

That's not catastrophic at all - just more limited role :) The root user should allow anyone in the account to assume the role - so that's a little odd. Out of curiosity, are you using Cloud9 or are you using your terminal?

[sirbully]

That's good to hear @kohidave! And yes, I do find it strange. I didn't experience this error when I was using copilot on my personal AWS account (where I had AdminAccess). Only in our company account (where I have very limited access). Also, I'm using my terminal for this.

On another note, I have another issue I need help with. This is probably due to my inexperience with AWS and less-than-stellar stackoverflow searching skills. I ran copilot app init in my repo twice. The first time, I had the same issue where I couldn't deploy it due to the AccessDenied error so I tried to copilot app delete but it gave me the same AccessDenied error. I haven't figured out the trusted entities workaround yet, so what I did was, I manually deleted the cloudformation stacks from the console and deleted the copilot directory too. Now, I think this one is truly catastrophic.

When I tried to run copilot app init for the second time, there was prompt asking me to use an existing app where I realized that it truly isn't fully deleted yet. I still had no luck deleting the app at that point so I went ahead and created a new app with a different name. Later, when I discovered the workaround, I still couldn't delete it because I already manually deleted the stack so it also deleted the first EnvManagerRole. That said, I feel like I should apologize for not using the CLI properly :(

Is there a way to fix that or is it going to be stuck forever? (which is understandable, it will be a constant reminder for me not to mess around with any automatically provisioned resources from the console)

[bvtujo]

@sirbully This is by no means unfixable, just annoying!

To do a fully clean deletion without using the app delete command, you need to delete the following things:

• All SSM parameters (in Systems Manager Parameter Store) prefixed with /copilot/...
• All cloudformation stacks containing the name of your app (stackset-myapp-infrastructure, myapp-infrastructure-roles, myapp-myenv, myapp-myenv-mysvc)
• All remaining myapp-myenv-EnvManagerRole and myapp-myenv-CFNExecutionRole IAM roles.

To delete stacks that were created with a given CFNExecutionRole (if that role no longer exists) you can (once you've deleted everything else) run copilot app init myapp again, then run copilot env init --name myenv. It may fail due to the stack already existing or being in an incorrect state, but the correct role will be recreated.

You can then manually delete your env CF stacks and the new /copilot/applications/myapp/environments/myenv parameter. Then when you run env init again, everything should be hunky dory.

Let me know if you have any questions! So sorry you're in this awkward state; it's quite easy to find yourself in a situation with CF that's difficult to resolve if you're not careful. We're trying to make things easier so that the path of least resistance is to use copilot exclusively for stack management but sometimes due to permissions issues these kinds of situations are hard to avoid.

[sirbully]

Thank you both, you've been very helpful! Looking forward to learning more of AWS (and properly using the copilot CLI) as well as the features you have planned for the future ✨

Before copilot, I actually tried to deploy our app manually from the AWS console, creating the required networking objects, the ecr, the cluster and everything that comes along with it. It was a good learning process to familiarize myself with the underlying infrastructure but I didn't want to have to do that every single time. Copilot is a heaven-sent CLI for me so I'm definitely eternally grateful! Please feel free to close this :)

Edit: huge shoutout to ContainersFromTheCouch team for leading me here as well

[bvtujo]

Closing this issue but feel free to reopen if anything else pops up. Thanks for giving Copilot a try!

[gautam-nutalapati]

Good morning @sirbully, when you're deploying with Copilot the credentials you use will need to have sts:AssumeRole permissions on the EnvManagerRoles for any environments you're authorized to deploy to.

Copilot's credential story is structured around a central administration account and environment accounts. The application account needs broad permissions, as it is responsible for creating environments, vpcs, pipelines, etc, but access can be restricted at the environment level. To deploy to any given environment as a developer, all you need is to be able to call sts:AssumeRole on the environment's EnvManagerRole. This role is created by Copilot and contains all the permissions needed to deploy and operate Copilot services, with no write-level permissions on any environment resources.

Does that help?

Hi, I am trying to see how to provide temporary access to copilot environment to users.
I tried assumeRole on EnvManagerRole mentioned here. Here's my full setup:
Copilot app deployed in Account "A".
Copilot environment deployed in Account "B".
After assuming EnvManagerRole in Account "B", I am not able to create new svc in environment in account "B".
copilot env ls or copilot svc init fail stating couldn't find an application named XXX in account B and region XXX.
copilot svc deploy in existing environment fail as well with select service: no services found
I think in addition to assumeRole operation, users need copilot application AWS account credentials as well right? Or is something I am doing is wrong?

[efekarakus]

Hi @gautam-nutalapati !

Your description is correct besides being able to assume the EnvManagerRole, users would need to be able to list/get SSM parameters starting with copilot/ stored in the application account.

Adding a ReadOnly SSM policy should hopefully unblock you!