Amplify-cli: custom cloudformation parameters for environments

[cyrfer]

How can I pass custom cloudformation parameters whose values change for different environments? I'd like to leverage Amplify Environments but I don't know what I need to do to get environment specific CF parameters passed to each CF stack. Does the Amplify CI/CD Console help?

I discovered if I create the "parameters.json" file in my Function stack folder, the values are passed to CF deploy. What happens when I define another environment? Does the "parameters.json" file get replaced?

Also, I only discovered "parameters.json" on a wild guess. Is there a bug in the code generator for Functions that this file is not created automatically?

* Which Category is your question related to? *

I want to give my Function stack custom Cloudformation parameters that are specific to the environment. For example, my API may have a dependency URI different in DEV and PROD environments.

* What AWS Services are you utilizing? *

AWS AppSync, Lambda, Cognito, Amplify CI/CD, etc

* Provide additional details e.g. code snippets *

E.g. Sample code, versions of Amplify you are using

npm list -g --depth=0 | grep -i amplify
@aws-amplify/[email protected]

[Ashish-Nanda]

Transferring to the CLI repo as the question seems related to CLI/console

[kaustavghosh06]

I discovered if I create the "parameters.json" file in my Function stack folder, the values are passed to CF deploy. What happens when I define another environment? Does the "parameters.json" file get replaced?

Also, I only discovered "parameters.json" on a wild guess. Is there a bug in the code generator for Functions that this file is not created automatically?

The parameters.json file contains parameter's which are passed to the Cloudformation template and is agnostic to environment changes and is not replaced per environment. For a simple "Hello world" Lambda function Cloudformation template, you don't need to pass any parameters to the Cloudformation template - that's why the file is not generated.

* Which Category is your question related to? *

I want to give my Function stack custom Cloudformation parameters that are specific to the environment. For example, my API may have a dependency URI different in DEV and PROD environments.

For environment specific parameters, I would recommend injecting the values in the amplify/team-provider-info.json file. The format of the file per environmnet would be like the the following:

{
    "dev": {
        "awscloudformation": {
            "AuthRoleName": "xxxxxxxxxxx",
            "UnauthRoleArn": "xxxxxxxxxxx",
            "AuthRoleArn": " "xxxxxxxxxxx"",
            "Region": "xxxxxxxxxxx",
            "DeploymentBucketName":  "xxxxxxxxxxx",
            "UnauthRoleName":  "xxxxxxxxxxx"
            "StackName": "xxxxxxxxxxx",
            "StackId":  "xxxxxxxxxxx"
        },
        "categories": {
            "function": {
                "<function-resource-name>": {
                           <parameter-name>: <value>
                   }
            }
        }
    },
    "prod": {
        "awscloudformation": {
            "AuthRoleName": "xxxxxxxxxxx",
            "UnauthRoleArn": "xxxxxxxxxxx",
            "AuthRoleArn": " "xxxxxxxxxxx"",
            "Region": "xxxxxxxxxxx",
            "DeploymentBucketName":  "xxxxxxxxxxx",
            "UnauthRoleName":  "xxxxxxxxxxx"
            "StackName": "xxxxxxxxxxx",
            "StackId":  "xxxxxxxxxxx"
        },
        "categories": {
            "function": {
                "<function-resource-name>": {
                           <parameter-name>: <value>
                   }
            }
        }
    }
}

The parameters referred as <parameter-name> are auto-injected into the Cloudformation template and are environment specific.

@cyrfer Please let me know if you have any more questions around this.

[kaustavghosh06]

Closing based on the last response. Please comment on this thread if you're still stuck and we'll re-open the issue.

[cyrfer]

@kaustavghosh06 this works great when i run amplify push from my local machine.

When I commit to codecommit and git push the amplify CICD build breaks saying my custom parameters were not provided.

EDIT:

I suspect the build spec (console makes it look like its using amplify.yml which is not checked into my repo) is not checking out the environment. My local system already has the amplify environment checked out, so maybe that's why I can deploy my backend from local using amplify push.

How can I verify the environment is actually "checked out" during the build process of Amplify CICD Console?

EDIT 2:

I tried this in another project and was able to do a little better, but I still have an issue when deploying from my local machine. In that project, I used the team file to successfully pass a resource value that is unique to each stack. Unfortunately I did not know the value when deploying the PROD stack, so I used a placeholder value "*" when deploying PROD. After the deploy succeeded, I determined the right value, plugged it into the team file in the PROD parameter section, and now when I run amplify push with my PROD env checked out locally, amplify push shows "no changes detected".

I am comfortable with Cloudformation but the deployment of amplify is still a mystery to me and others I am trying to evangelize it to.

[cyrfer]

This is linking the CICD question, which is related to the local CLI problems in EDIT 1
https://forums.aws.amazon.com/thread.jspa?threadID=318114

[kaustavghosh06]

@cyrfer If you made changes to your team-provider.info file and want to do a push - you can install the latest version of the CLI - 4.16.1 and run amplify push --force - this would pickup your changes from the team-provider-info.json file and populate it into the Cloudformation tempalte.

Are you still stuck with the Amplify console build issue? Could you share the error stack from the Amplify console build step?

[kaustavghosh06]

Closing this issue. If you're still stuck please comment on this thread.

[cyrfer]

@kaustavghosh06 yes, still stuck in amplify console. The build console output before the error was not that useful, but the downloadable logs were more useful.

Here is the Cloudformation event showing the real errors, obviously from the custom parameters that are still not being passed from the team file (attached):

Parameters: [InternalBucketName, MongoCollectionJobs, MongoUrl, MongoCollectionUsers, MongoDb, MongoUrlSecret, stateMachineArn] must have values

[cyrfer]

I want this tool to work for me but it really drives me crazy at times.

I wish I could get some insights at when the backend-config or team provider files are used in deployment. Today I found a critical feature that is barely documented, but I cant get it to work either.

@kaustavghosh06 you ask me to reply but you don't open the ticket. What should a developer do to get support with the CLI workflow since it is not supported by AWS Support?

[jaga810]

How about using SSM Parameter Store to store and get CFn parameters.
e.g. SSM Parameter Store Key: cognito-federation-access-id-${env name}

You can access amplify env name in CFn template.
e.g. Parameters in auth category template

Parameters:
  env:
    Type: String

[wizage]

We found a semi work around for passing new environment variables to the functions:

Modifying the team-provider-info.json we were able to add a new section under function like this:

"dev": {
        "awscloudformation": {
           . . .
        },
        "categories": {
            "function": {
                "demowbd58adad4": {
                    "url": "dev.example.com
                }
            }
        }
    }

We then took went to the CloudFormation template for the function and added it to the parameters at the top and was able to !Ref it later in the file for environment variables. This seemed to work just fine when using amplify push.

Now the bug we found is around this:
Next we need to create a new env for prod. So we run amplify env add and fill in the defaults for the new env.

Now we will update the team-provider-info.json with our environment variables that we added like above:

"prod": {
        "awscloudformation": {
           . . .
        },
        "categories": {
            "function": {
                "demowbd58adad4": {
                    "url": "prod.example.com
                }
            }
        }
    }

We decide not to run push and check out our dev environment again, the team-provider-info.json file deletes any variables put in the categories that was not synced. When checkout any env it will remove all the variables from all env even if they are pushed.

Other ways of it causing issues include:

• Running amplify init --env will also remove this variables (This could be the cause of aws-amplify/amplify-console#206 )

Related tickets:

• aws-amplify/amplify-console#459

[cyrfer]

@jaga810 please, continue. How can I leverage that for specific configurations in each environment?

[jaga810]

1. Store specific configurations for each environment in Parameter Store using env name in key.
   

2. Refer configuration in Parameter Store in your CFn template using !Sub and resolve:ssm

```template.yml
Parameters:
env:
Default: production
Type: String

Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub '{{resolve:ssm:/your-project/${env}/s3-bucket-name:2}}'
```

If you store credentials, you should use resolve:ssm-secure instead of resolve:ssm

[cyrfer]

@jaga810 that's really cool. I did not know about resolve. I'll look for opportunities to apply it.

Unfortunately, I should have mentioned, some of the parameters I want to pass are resource ARNs that are created in other stacks of Amplify. So for an extra environment, there would be new ARNs that and I would not want to go around updating SSM Parameters.

Luckily, for this part of the problem, I discovered and noted a solution over here. The file parameters.json supports a syntax gem that helps a ton. Anyone know where this is documented?

{
  "stateMachineArn": {
    "Fn::GetAtt": [
        "stateMachineaudio",
        "Outputs.StateMachineArn"
    ]
  }
}

[cyrfer]

I was able to use the CLI (version 4.19.0, with bug fix #4161) to deploy a 2nd environment including specifications for environment-specific resources and environment variables.

I did run into a small problem that I wrote a separate ticket for.
https://github.com/aws-amplify/amplify-cli/issues/4335

Now I have only a couple concerns remaining about using Amplify for multiple environments,

1. I have not yet (but will soon) tested deploying into a different account or region.
2. Sometimes I need to deploy a piece of my application (async workflows) into multiple regions in order to avoid S3 egress charges, and I don't think I can deploy only a piece of an Amplify application.

[kaustavghosh06]

@cyrfer Glad that you were able to deploy your environments.
I responded to your issue #4335

Regarding multiple environments, you will be able to deploy your distinct environments in multiple regions or different accounts. The key out here is to have the right AWS profiles set and have that associated with the corresponding environments that you create (you make that association whenever you create a new env). Please feel free to comment on this thread if you have more questions. Closing this issue.

[cyrfer]

@kaustavghosh06 using the profiles makes sense for multiple accounts and regions.

For point 2, I think I need a subset of the app deployed, not the full app. For example, I need a State Machine in multiple regions, but my API in only one region.

[cyrfer]

@kaustavghosh06 this issue is not resolved. Custom parameters per environment are still a challenge. After seeing success passing parameters using team...json file, I see them overwritten when I:

1. git clone MYREPO
2. amplify init

The clone has my parameters, but running amplify init overwrites them. I must discard the change amplify init causes for my parameters to work.

Only the parameters for the function resource categories were affected. The hosting and api categories were not overwritten.

$ amplify -v
4.19.0

[kaustavghosh06]

@cyrfer If you're sharing your project with your team or cloning it on a different machine it is recommended to remove the team-provider-info.json file from your .gitignore file - that way you'll not loose your team-provider-info.json file.
We've documented this out here as well - https://docs.amplify.aws/cli/teams/shared#sharing-projects-outside-the-team

[cyrfer]

@kaustavghosh06 I did. Like I said, some categories were not affected.

To be clear,

The clone has my parameters

and

running amplify init overwrites them

I don't understand why anyone would add the team...json to .gitignore. It's required for passing environment specific parameters.

[kaustavghosh06]

@cyrfer The overwriting of these parameters was fixed in #4161 (and is a part of the latest version of the CLI) and you'd previously acknowledged of that the fix worked for you. Is that still an issue?

This is the reason why anyone would add the team-provier-info.json file to the .gitignore file - https://docs.amplify.aws/cli/teams/shared#sharing-projects-outside-the-team

[cyrfer]

and you'd previously acknowledged of that the fix worked for you. Is that still an issue?

I acknowledged that CLI 4.19.0 allowed me to successfully deploy one environment with env-specific parameters, specified in the team.json file, per working with @wizage offline.

Deploying to one environment is no longer an issue.

It would be cruel to the other team if you remove that file. I can imagine omitting things, but removing means they would need to reverse engineer what keys need to be in the team...json file for each environment.

@kaustavghosh06

[cyrfer]

@kaustavghosh06 please keep this ticket open until this is resolved.
https://github.com/aws-amplify/amplify-cli/issues/3240#issuecomment-634109040

[kaustavghosh06]

Okay - so the request out here is to store the team-provider-info.json file in the cloud as well
to share across team-members since there is hesitation that git isn't the right way to share this file within the team? Let me know if that's the request out here? I'll change the GitHub issue title since this is a different request altogether than the original bug filed initially which we fixed.

[cyrfer]

Not even close. :(

[kaustavghosh06]

@cyrfer Sorry, could you please provide more details? The request is not clear based on your last comment.

[cyrfer]

I don't know how to say it any clearer than I did here.
https://github.com/aws-amplify/amplify-cli/issues/3240#issuecomment-634109040

[Mentioum]

Okay - so the request out here is to store the team-provider-info.json file in the cloud as well

This is very close to what Terraform does with its .tfstate files. Big fan of it tbh along with state locking when deploying using DynamoDB. docs here

That aside. Having read through this thread I do think that the docs could benefit from an end to end guide on how to add custom variables to Lambda functions from beginning to end. If this already exists I'd really appreciate a link :pray:

[cyrfer]

@Mentioum

Here is the current guide on using Custom CF, which provides enough info that one can use it for custom environment variables in Lambda Functions.
https://docs.amplify.aws/cli/usage/customcf

To be clear, I am not asking for help with custom environment variables for a Lambda. I'm comfortable customizing the CF templates generated by the Amplify CLI, and passing CF template parameters as Function environment variables.

[Mentioum]

@Mentioum

Here is the current guide on using Custom CF, which provides enough info that one can use it for custom environment variables in Lambda Functions.
https://docs.amplify.aws/cli/usage/customcf

To be clear, I am not asking for help with custom environment variables for a Lambda. I'm comfortable customizing the CF templates generated by the Amplify CLI, and passing CF template parameters as Function environment variables.

Thanks for the link. Yep I understand what you're looking for re multienv and I'll want that too eventually. At the moment I'm just adding All envs to the CF template and then using them conditionally based on the ENV environment variable.

Unideal and obviously not as secure as I'd like in the longer run. But ok... for now.

Looking forward to multienv env support 💪, until then it looks looks like KMS is the best option... bit of a pain.

[BennettSmith]

Storing environment specific properties in the team-provider-info.json file breaks down when using AWS Console PR previews. Deployments fail in CF with “variable must contain a value” errors because the synthesized environment names for each preview are not in the team-provider-info.json file.

Is there any workable solution to that issue?