Copilot-cli: CodeBuild is failing to build services with the following output

[bryceandress]

`

[Container] 2020/11/13 21:23:39 Running command export COLOR="false"

807 |  
808 | [Container] 2020/11/13 21:23:39 Running command svcs=$(./copilot-linux svc ls --local --json \| jq '.services[].name' \| sed 's/"//g')
809 |  
810 | [Container] 2020/11/13 21:23:39 Running command envs=$(./copilot-linux env ls --json \| jq '.environments[].name' \| sed 's/"//g')
811 |  
812 | [Container] 2020/11/13 21:23:39 Running command tag=$(sed 's/:/-/g' <<<"$CODEBUILD_BUILD_ID")
813 |  
814 | [Container] 2020/11/13 21:23:39 Running command for env in $envs; do
815 | for svc in $svcs; do
816 | ./copilot-linux svc package -n $svc -e $env --output-dir './infrastructure' --tag $tag;
817 | done;
818 | done;
819 |  
820 |  
821 | [Container] 2020/11/13 21:23:40 Running command ls -lah ./infrastructure
822 | total 48K
823 | drwxr-xr-x 2 root root 4.0K Nov 13 21:23 .
824 | drwxr-xr-x 5 root root 4.0K Nov 13 21:23 ..
825 | -rw-r--r-- 1 root root 731 Nov 13 21:23 api-test.params.json
826 | -rw-r--r-- 1 root root 15K Nov 13 21:23 api.stack.yml
827 | -rw-r--r-- 1 root root 733 Nov 13 21:23 front-test.params.json
828 | -rw-r--r-- 1 root root 15K Nov 13 21:23 front.stack.yml
829 |  
830 | [Container] 2020/11/13 21:23:40 Running command for svc in $svcs; do
831 | ADDONSFILE=./infrastructure/$svc.addons.stack.yml
832 | if [ -f "$ADDONSFILE" ]; then
833 | tmp=$(mktemp)
834 | timestamp=$(date +%s)
835 | aws s3 cp "$ADDONSFILE" "s3://stackset-essessment-infr-pipelinebuiltartifactbuc-ymibfjy3xogl/manual/$timestamp/$svc.addons.stack.yml";
836 | jq --arg a "https://stackset-essessment-infr-pipelinebuiltartifactbuc-ymibfjy3xogl.s3-us-east-1.amazonaws.com/manual/$timestamp/$svc.addons.stack.yml" '.Parameters.AddonsTemplateURL = $a' ./infrastructure/$svc-test.params.json > "$tmp" && mv "$tmp" ./infrastructure/$svc-test.params.json
837 | fi
838 | done;
839 |  
840 |  
841 | [Container] 2020/11/13 21:23:40 Running command for svc in $svcs; do
842 | manifest=$(cat $CODEBUILD_SRC_DIR/copilot/$svc/manifest.yml \| ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))')
843 | base_dockerfile=$(echo $manifest \| jq '.image.build')
844 | build_dockerfile=$(echo $manifest\| jq 'if .image.build?.dockerfile? then .image.build.dockerfile else "" end' \| sed 's/"//g')
845 | build_context=$(echo $manifest\| jq 'if .image.build?.context? then .image.build.context else "" end' \| sed 's/"//g')
846 | dockerfile_args=$(echo $manifest \| jq 'if .image.build?.args? then .image.build.args else "" end \| to_entries?')
847 | df_rel_path=$( echo $base_dockerfile \| sed 's/"//g')
848 | if [ -n "$build_dockerfile" ]; then
849 | df_rel_path=$build_dockerfile
850 | fi
851 | df_path=$df_rel_path
852 | df_dir_path=$(dirname "$df_path")
853 | if [ -n "$build_context" ]; then
854 | df_dir_path=$build_context
855 | fi
856 | build_args=
857 | if [ -n "$dockerfile_args" ]; then
858 | for arg in $(echo $dockerfile_args \| jq -r '.[] \| "(.key)=(.value)"'); do
859 | build_args="$build_args--build-arg $arg "
860 | done
861 | fi
862 | echo "Service: $svc"
863 | echo "Relative Dockerfile path: $df_rel_path"
864 | echo "Docker build context: $df_dir_path"
865 | echo "Docker build args: $build_args"
866 | echo "Running command: docker build -t $svc:$tag $build_args-f $df_path $df_dir_path";
867 | docker build -t $svc:$tag $build_args-f $df_path $df_dir_path;
868 | image_id=$(docker images -q $svc:$tag);
869 | for env in $envs; do
870 | repo=$(cat $CODEBUILD_SRC_DIR/infrastructure/$svc-$env.params.json \| jq '.Parameters.ContainerImage' \| sed 's/"//g');
871 | region=$(echo $repo \| cut -d'.' -f4);
872 | $(aws ecr get-login --no-include-email --region $region);
873 | docker tag $image_id $repo;
874 | docker push $repo;
875 | done;
876 | done;
877 |  
878 | Service: api
879 | Relative Dockerfile path: polygraph-api/Dockerfile.aws
880 | Docker build context: polygraph-api
881 | Docker build args:
882 | Running command: docker build -t api:pipeline-essessment-Presage-Security-polygraph-api-BuildProject-f4217010-79a8-4da9-a0a5-3ee6f569e8d1 -f polygraph-api/Dockerfile.aws polygraph-api
883 | unable to prepare context: path "polygraph-api" not found
884 | WARNING! Using --password via the CLI is insecure. Use --password-stdin.
885 | WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
886 | Configure a credential helper to remove this warning. See
887 | https://docs.docker.com/engine/reference/commandline/login/#credentials-store
888 |  
889 | Login Succeeded
890 | "docker tag" requires exactly 2 arguments.
891 | See 'docker tag --help'.
892 |  
893 | Usage: docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
894 |  
895 | Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
896 | The push refers to repository [994342159206.dkr.ecr.us-east-1.amazonaws.com/essessment/api]
897 | An image does not exist locally with the tag: 994342159206.dkr.ecr.us-east-1.amazonaws.com/essessment/api
898 | Service: front
899 | Relative Dockerfile path: polygraph-front/Dockerfile
900 | Docker build context: polygraph-front
901 | Docker build args:
902 | Running command: docker build -t front:pipeline-essessment-Presage-Security-polygraph-api-BuildProject-f4217010-79a8-4da9-a0a5-3ee6f569e8d1 -f polygraph-front/Dockerfile polygraph-front
903 | unable to prepare context: path "polygraph-front" not found
904 | WARNING! Using --password via the CLI is insecure. Use --password-stdin.
905 | WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
906 | Configure a credential helper to remove this warning. See
907 | https://docs.docker.com/engine/reference/commandline/login/#credentials-store
908 |  
909 | Login Succeeded
910 | "docker tag" requires exactly 2 arguments.
911 | See 'docker tag --help'.
912 |  
913 | Usage: docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
914 |  
915 | Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
916 | The push refers to repository [994342159206.dkr.ecr.us-east-1.amazonaws.com/essessment/front]
917 | An image does not exist locally with the tag: 994342159206.dkr.ecr.us-east-1.amazonaws.com/essessment/front
918 |  
919 | [Container] 2020/11/13 21:23:43 Command did not exit successfully for svc in $svcs; do
920 | manifest=$(cat $CODEBUILD_SRC_DIR/copilot/$svc/manifest.yml \| ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))')
921 | base_dockerfile=$(echo $manifest \| jq '.image.build')
922 | build_dockerfile=$(echo $manifest\| jq 'if .image.build?.dockerfile? then .image.build.dockerfile else "" end' \| sed 's/"//g')
923 | build_context=$(echo $manifest\| jq 'if .image.build?.context? then .image.build.context else "" end' \| sed 's/"//g')
924 | dockerfile_args=$(echo $manifest \| jq 'if .image.build?.args? then .image.build.args else "" end \| to_entries?')
925 | df_rel_path=$( echo $base_dockerfile \| sed 's/"//g')
926 | if [ -n "$build_dockerfile" ]; then
927 | df_rel_path=$build_dockerfile
928 | fi
929 | df_path=$df_rel_path
930 | df_dir_path=$(dirname "$df_path")
931 | if [ -n "$build_context" ]; then
932 | df_dir_path=$build_context
933 | fi
934 | build_args=
935 | if [ -n "$dockerfile_args" ]; then
936 | for arg in $(echo $dockerfile_args \| jq -r '.[] \| "(.key)=(.value)"'); do
937 | build_args="$build_args--build-arg $arg "
938 | done
939 | fi
940 | echo "Service: $svc"
941 | echo "Relative Dockerfile path: $df_rel_path"
942 | echo "Docker build context: $df_dir_path"
943 | echo "Docker build args: $build_args"
944 | echo "Running command: docker build -t $svc:$tag $build_args-f $df_path $df_dir_path";
945 | docker build -t $svc:$tag $build_args-f $df_path $df_dir_path;
946 | image_id=$(docker images -q $svc:$tag);
947 | for env in $envs; do
948 | repo=$(cat $CODEBUILD_SRC_DIR/infrastructure/$svc-$env.params.json \| jq '.Parameters.ContainerImage' \| sed 's/"//g');
949 | region=$(echo $repo \| cut -d'.' -f4);
950 | $(aws ecr get-login --no-include-email --region $region);
951 | docker tag $image_id $repo;
952 | docker push $repo;
953 | done;
954 | done;
955 | exit status 1
956 | [Container] 2020/11/13 21:23:43 Phase complete: POST_BUILD State: FAILED
957 | [Container] 2020/11/13 21:23:43 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: for svc in $svcs; do
958 | manifest=$(cat $CODEBUILD_SRC_DIR/copilot/$svc/manifest.yml \| ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))')
959 | base_dockerfile=$(echo $manifest \| jq '.image.build')
960 | build_dockerfile=$(echo $manifest\| jq 'if .image.build?.dockerfile? then .image.build.dockerfile else "" end' \| sed 's/"//g')
961 | build_context=$(echo $manifest\| jq 'if .image.build?.context? then .image.build.context else "" end' \| sed 's/"//g')
962 | dockerfile_args=$(echo $manifest \| jq 'if .image.build?.args? then .image.build.args else "" end \| to_entries?')
963 | df_rel_path=$( echo $base_dockerfile \| sed 's/"//g')
964 | if [ -n "$build_dockerfile" ]; then
965 | df_rel_path=$build_dockerfile
966 | fi
967 | df_path=$df_rel_path
968 | df_dir_path=$(dirname "$df_path")
969 | if [ -n "$build_context" ]; then
970 | df_dir_path=$build_context
971 | fi
972 | build_args=
973 | if [ -n "$dockerfile_args" ]; then
974 | for arg in $(echo $dockerfile_args \| jq -r '.[] \| "(.key)=(.value)"'); do
975 | build_args="$build_args--build-arg $arg "
976 | done
977 | fi
978 | echo "Service: $svc"
979 | echo "Relative Dockerfile path: $df_rel_path"
980 | echo "Docker build context: $df_dir_path"
981 | echo "Docker build args: $build_args"
982 | echo "Running command: docker build -t $svc:$tag $build_args-f $df_path $df_dir_path";
983 | docker build -t $svc:$tag $build_args-f $df_path $df_dir_path;
984 | image_id=$(docker images -q $svc:$tag);
985 | for env in $envs; do
986 | repo=$(cat $CODEBUILD_SRC_DIR/infrastructure/$svc-$env.params.json \| jq '.Parameters.ContainerImage' \| sed 's/"//g');
987 | region=$(echo $repo \| cut -d'.' -f4);
988 | $(aws ecr get-login --no-include-email --region $region);
989 | docker tag $image_id $repo;
990 | docker push $repo;
991 | done;
992 | done;
993 | . Reason: exit status 1
994 | [Container] 2020/11/13 21:23:43 Expanding base directory path: .
995 | [Container] 2020/11/13 21:23:43 Assembling file list
996 | [Container] 2020/11/13 21:23:43 Expanding .
997 | [Container] 2020/11/13 21:23:43 Expanding file paths for base directory .
998 | [Container] 2020/11/13 21:23:43 Assembling file list
999 | [Container] 2020/11/13 21:23:43 Expanding infrastructure/*
1000 | [Container] 2020/11/13 21:23:43 Found 4 file(s)
1001 |  
`

Looks like the error is occuring at line 957. Has anyone seen this before?

[efekarakus]

Heya @bryceandress !

Which version of Copilot are you using?

One hypothesis on the failure: it's possible that you upgraded on a newer version of Copilot but the buildspec was generated using an older version of the CLI. Would you mind deleting the buildspec.yml, then running the following again to regenerate the buildspec:

copilot pipeline init               # regenerates the buildspec
git add copilot/buildspec.yml       # commit and push!
git commit -m "add new buildspec"
git push         

It's possible that we pushed a bug fix in between that might solve the failure.

[bryceandress]

That appears to have taken care of it. I think an extra svc snuck into my copilot directory causing the problem not an update of the tool. Regardless the pipeline init took care of it.