Amplify-console: Amplify Continuous Deployment Problem: Monorepo settings and can't find aws-exports.js

[geoseong]

Hi.

I'm working monorepo project.

• github repo: kendra-button
  
  
  
  • I've tried with temp branch and feature-frontend-api branch.
  
  

I've tried so many ways to deploy via Hosting with Amplify Console (Managed hosting with custom domains, Continuous deployment).

the major problem for me is,

• amplify's build image can't pull aws-export.js file when my amplify project is placed in other folder
• even when amplify project is placed in root, sometimes can't find aws-exports.js(this is other project's case)
• I followed monorepo configuration guide, amplify yml doesn't recognize what I've configured

Please let me know why this situation happens and how to deploy it without errors.
I wasted about more than 3 days resolving this problem.

here is my situations below.

1st attempt: normal

Project architecture

• tried way 1:
  
  
• tried way 2:
  
  

/amplify.yml in my repository

version: 0.1
frontend:
  phases:
    build:
      commands:
        - cd frontend/kendra-button-front
        - yarn
        - yarn build
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: frontend/kendra-button-front/src/out
    files:
      - '**/*'
  cache:
    paths: []

Errors in "Build" step in Amplify Console

2st attempt: try to pull amplify's environment files manually

Project architecture

• same architecture as 1st attempt

/amplify.yml in my repository

• I'd add Environment Variables in Amplify Console
• the reason why I put the long amplify pull command is
  
  
  
  • try to pull amplify's environment files manually
  
  
  • especially to get aws-exports.js!
  
  
  • It's working in my macos with this long amplify pull command
  
  

version: 0.1
frontend:
  phases:
    # preBuild:
    #   commands:
    build:
      commands:
        - AMPLIFY="{\
          \"projectName\":\"$PROJECT_NAME\",\
          \"appId\":\"$AWS_APP_ID\",\
          \"envName\":\"$ENV_NAME\",\
          \"defaultEditor\":\"code\"\
          }"
        - AWSCLOUDFORMATIONCONFIG="{
          \"configLevel\":\"project\",
          \"useProfile\":false,
          \"profileName\":\"default\",
          \"accessKeyId\":\"$ACCESS_KEY_ID\",
          \"secretAccessKey\":\"$SECRET_ACCESS_KEY\",
          \"region\":\"us-west-2\"
          }"
        - PROVIDERS={
          \"awscloudformation\":$AWSCLOUDFORMATIONCONFIG
          }
        - amplify pull --amplify $AMPLIFY --providers $PROVIDERS --yes
        - find . -name "aws-exports.js"
        - cd frontend/kendra-button-front
        - yarn
        - yarn build
  artifacts:
    # IMPORTANT - Please verify your build output directory
    baseDirectory: frontend/kendra-button-front/src/out
    files:
      - '**/*'
  cache:
    paths: []

Errors in "Build" step in Amplify Console

• I tried more than 20 times to solve this problem
  

3st attempt: "Monorepo settings"

refer to this document

Project architecture

/amplify.yml in my repository

version: 1
applications:
  - appRoot: frontend/kendra-button-front
    frontend:
      phases:
        build:
          - find . -name "aws-exports.js"
          - yarn
          - yarn build
      artifacts:
        files:
          - '**/*'
        baseDirectory: frontend/kendra-button-front/src/out

Errors in "Build" step in Amplify Console

[rocketmaniac]

Thanks, @geoseong! your debug line, - find . -name "aws-exports.js", helped me to figure out the path i needed in my code, i.e.:

import config from '../aws-exports.js'

Your input, plus removing aws-exports.js from .gitignore helped to get my app up and running again in the short-term. I'm pretty sure committing aws-exports.js is an anti-pattern, but it keeps me moving forward on my prototype for now, until I can figure out a better solution!

[geoseong]

@swaminator I also asked it to Discord to @brene , he said you are an expert on that. please help to get the solution how it works and how to solve the problem 😓

[geoseong]

@rocketmaniac like you said, committing aws-exports.js is anti-pattern and I think there is no reason to use amplify build with that way.

so I will use amplify-cli-action with S3 distribution until the response...

[rocketmaniac]

@geoseong did amplify-cli-action work for you? If so, can you please share a sanitized version of your flow?

[geoseong]

@rocketmaniac I was planning to deploy to s3 bucket in amplify-cli-action and configure CloudFront and custom domain manually.
but I got a better solution to make Continuous Deployment in Amplify Console work.

first, make the shell script to execute amplify pull

(be careful with environment naming start with AWS. is the AWS's system environment)

I got a hint from #496 . thanks @vchaddha

#!/usr/bin/env bash

AMPLIFY="'{\
\"projectName\":\"${PROJECT_NAME}\",\
\"appId\":\"${AWS_APP_ID}\",\
\"envName\":\"${ENV_NAME}\",\
\"defaultEditor\":\"code\"\
}'"
AWSCLOUDFORMATIONCONFIG="{\
\"configLevel\":\"project\",\
\"useProfile\":false,\
\"profileName\":\"default\",\
\"accessKeyId\":\"${ACCESS_KEY_ID}\",\
\"secretAccessKey\":\"${SECRET_ACCESS_KEY}\",\
\"region\":\"${REGION}\"\
}"
PROVIDERS="'{\
\"awscloudformation\":${AWSCLOUDFORMATIONCONFIG}\
}'"

cmd="amplify pull --amplify ${AMPLIFY} --providers ${PROVIDERS} --yes"
echo $cmd
eval $cmd

and add to execute that sh file command lines in amplify.yml.

for me, like this below.

version: 0.1
frontend:
  phases:
    preBuild:
      commands:
        - cd frontend/kendra-button-front
        - chmod +x amplify-pull.sh
        - ./amplify-pull.sh
    build:
      commands:
        - find . -name "aws-exports.js"
        - yarn
        - yarn build
  artifacts:
    baseDirectory: frontend/kendra-button-front/src/out
    files:
      - '**/*'
  cache:
    paths: []

[dbhagen]

Ok, so this is my current solve for it. It relies on on pre-post git hooks (using Yorkie: https://github.com/yyx990803/yorkie)
First, setup an EXPORT_KEY environment variable (I have it in my .zshrc file on my mac, export EXPORT_KEY=<some random key>). Hang on to this variable, you'll need it later.

In your package.json file, add these scripts:

{
  "scripts": {
    "unlock-amplify": "echo $EXPORT_KEY | gpg --batch -d --passphrase-fd 0 --yes aws-exports.js.gpg > aws-exports.js",
  },
  "gitHooks": {
    "post-commit": "[[ ! -z \"${EXPORT_KEY}\" ]] && [ -f .commit ] && rm -f .commit aws-exports.js.gpg && echo $EXPORT_KEY | gpg --batch --yes --passphrase-fd 0 -c aws-exports.js && git add aws-exports.js.gpg && git commit --amend -C HEAD --no-verify",
    "pre-commit": "touch .commit",
    "post-merge": "[[ ! -z \"${EXPORT_KEY\" ]] && echo \"$EXPORT_KEY\" | gpg --batch -d --passphrase-fd 0 --yes aws-exports.js.gpg > aws-exports.js"
  }
}

What you'll see is that it will gpg encrypt your aws-exports.js during a commit as aws-exports.js.gpg. This gets committed to your repo.

The latest build image seemed to be able to fire the post-merge script above on checking out the code for building, so you shouldn't need this next step, but I used to do this.
In your Amplify.yml file, add this to your prebuild:

frontend:
  phases:
    preBuild:
      commands:
        - yarn unlock-amplify
or if you use NPM
        - npm run unlock-amplify

This allows you to decrypt the aws-exports.js.gpg back into a aws-exports.js file needed for building.

[rocketmaniac]

@geoseong , @dbhagen thanks for your solutions!

I ended up going with the amplify pull approach so as not to introduce any more dependancies to the build, and it works great.

It turns out the approach is somewhat documented here:

https://docs.amplify.aws/cli/usage/headless#amplify-pull-parameters

but @geoseong i like your doc better as it illustrates the use of the env vars ;-)

Since the amplify sample projects all make use of aws-exports in the frontend code, it would save everyone a lot of time and frustration if the docs or build templates were updated to cover the requisite preBuild step for amplify pull so that the sample code can actually build properly OOTB.

[andrewbtp]

For anybody using @geoseong 's approach or script, I had to switch profile to true to make it work.

[geoseong]

@andrewbtp do you mean \"useProfile\":true in AWSCLOUDFORMATIONCONFIG in shell script? if it is right, I'll take a note

[andrewbtp]

@geoseong That's it!

[kldeb]

Does anyone know the parameter to only generate the aws-exports file and not add the amplify directory? I'm testing locally since I can't figure out what it isn't working in the amplify console and it just deployed a stale backend on me, not sure why.

[geoseong]

I found another solution using backend and changing the version to 1 in amplify.yml. and create customized amplifyPush shell script. here is issue that I commented

need to wait until public solution comes...

[kldeb]

@geoseong thanks, that's quite the workaround. I added aws-exports.js to the repo for now. were you able to get a an aws-exports.js file generated using your amplify pull script? it didn't work for me.

[geoseong]

@kldeb yes. I was able to get aws-exports.js file using amplify pull cli. it's strange, need to know more specific that you'd tried 😅

[kldeb]

@geoseong here's my amplify.yml file:

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm install
        - chmod +x amplify-pull.sh
        - ./amplify-pull.sh
    build:
      commands:
        - find . -name "aws-exports.js"
        - npm run build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

and here's my amplify-pull.sh file:

#!/bin/bash
set -e
IFS='|'

AMPLIFY="{\
\"projectName\":\"my-project-name\",\
\"appId\":\"$APP_ID\",\
\"envName\":\"$USER_BRANCH\",\
\"defaultEditor\":\"code\"\
}"

REACTCONFIG="{\
\"SourceDir\":\"src\",\
\"DistributionDir\":\"build\",\
\"BuildCommand\":\"npm run-script build\",\
\"StartCommand\":\"npm run-script start\"\
}"

AWSCLOUDFORMATIONCONFIG="{\
\"configLevel\":\"project\",\
\"useProfile\":false,\
\"profileName\":\"default\",\
\"accessKeyId\":\"${ACCESS_KEY_ID}\",\
\"secretAccessKey\":\"${SECRET_ACCESS_KEY}\",\
\"region\":\"us-east-1\"\
}"

FRONTEND="{\
\"frontend\":\"javascript\",\
\"framework\":\"react\",\
\"config\":$REACTCONFIG\
}"

PROVIDERS="{\
\"awscloudformation\":$AWSCLOUDFORMATIONCONFIG\
}"

cmd="amplify pull --amplify ${AMPLIFY} --frontend ${FRONTEND} --providers ${PROVIDERS} --yes"
echo $cmd
eval $cmd

[geoseong]

@kldeb did you test the shell script of your amplify-pull.sh in your local?

and remove the frontend parameter and try again.

[kldeb]

@geoseong yes, it worked locally but it added the amplify directory and files to the project. I don't see the option to turn that off in the headless docs.

I'll try removing the frontend parameter.

[kennandavison]

Clean way to do this is to set your Amplify project-config.json destination path to your mono repo's common folder and import from there.

[litwicki]

@geoseong are you still having any issues or can we close this issue?