Aws-codebuild-docker-images: Support for secondary-artifacts in codebuild_build.sh

[PierreKiwi]

Hello,
I started playing with local CodeBuild builds recently and it looks like the instruction secondary-artifacts is not honored.

buildspec.yml

version: 0.2

phases:
  build:
    commands:
      - cd $CODEBUILD_SRC_DIR/dir1
      - touch file1
      - cd $CODEBUILD_SRC_DIR/dir2
      - touch file2

artifacts:
  secondary-artifacts:
    artifact1:
      base-directory: $CODEBUILD_SRC_DIR/dir1
      files:
        - file1
    artifact2:
      base-directory: $CODEBUILD_SRC_DIR/dir2
      files:
        - file2

command
./codebuild_build.sh -i aws/codebuild/python:3.6.5 -a codebuild_build

output

...
agent_1  | [Container] 2018/10/11 20:41:57 Phase context status code: CLIENT_ERROR Message: no definition for primary artifacts in buildspec
agent_1  | [Container] 2018/10/11 20:41:57 Runtime error (*errors.errorString: no definition for primary artifacts in buildspec)

If I add a files section to artifacts, CodeBuild does the zipping of whatever it is specified but secondary artifacts are still not created.

[awszhen]

Hi! This is resolved in the latest version of the local agent. SHA: c2dacbc1091a274d587be349075dcec64e5890a58232ed7329c55bd972608a81

[vibhuyadav]

It is probably unrelated to local agent but I am having the same issue on AWS Codebuild when I only specify secondary artifacts.

[subinataws]

@vibhuyadav - could you please post that error message and the buildspec (remove any sensitive info) on CodeBuild's AWS forums (https://forums.aws.amazon.com/forum.jspa?forumID=230)? Oncalls will take a look.

[awszhen]

Hi @vibhuyadav , this is the expected behavior. Primary artifacts location is required in local agent. And you can not specify secondary artifacts unless you specify primary artifacts.

[vibhuyadav]

Hi @vibhuyadav , this is the expected behavior. Primary artifacts location is required in local agent. And you can not specify secondary artifacts unless you specify primary artifacts.

@awszhen But this is AWS Codebuild and not local agent. I should post this in AWS forums including the buildspec. Thanks @subinataws

[shduke]

@vibhuyadav did you find a resolution to this? Or do you have a link to your forum post?

[dev-dean-xu]

Each artifact identifier under secondary-artifacts block in buildspec.yml must match an artifact defined in the secondaryArtifacts attribute of CodeBuild project.

[vibhuyadav]

@shduke I did not. For the sake of it I tried again using localbuild and I still see it.
Buildspec -

  post_build:
    commands:
      - echo Pre-Build Phase


artifacts:
  secondary-artifacts:
    bin:
      files:
        - deploy/**
      name: bin
      discard-paths: no

Error -

agent_1  | [Container] 2019/07/22 20:19:19 Phase complete: UPLOAD_ARTIFACTS State: FAILED
agent_1  | [Container] 2019/07/22 20:19:19 Phase context status code: CLIENT_ERROR Message: no definition for primary artifacts in buildspec
agent_1  | [Container] 2019/07/22 20:19:19 Runtime error (*clienterr.PhaseContextError: no definition for primary artifacts in buildspec)

So basically from the documentation https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
This won't work -

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  secondary-artifacts:
    artifact1:
      files:
        - directory/file
      name: secondary-artifact-name-1        
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

[solsglasses]

Bump, this is still a problem. Please re-open. codebuild_build.sh is not matching CodeBuild in regards to secondary-artifacts config

@vibhuyadav post above mine explains it best. When copy/pasting from AWS's own documentation, codebuild_build.sh produces a different result

[edalongeville]

This is still happening. Could this issue be reopened please ?

[ioMatrix]

This is due to a bad example in the documentation. This will NOT work:

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  secondary-artifacts:
    artifact1:
      files:
        - directory/file
      name: secondary-artifact-name-1
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

The documentation, obscurely, points it out: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

artifacts/files
Required sequence. Represents the locations that contain the build output artifacts in the build environment. Contains a sequence of scalars, with each scalar representing a separate location where CodeBuild can find build output artifacts, relative to the original build location or, if set, the base directory.

So this is an edited version that does work:

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  files:
    - directory/*
  secondary-artifacts:
    artifact1:
      files:
        - directory/file
      name: secondary-artifact-name-1
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

[datsabk]

This doesn't work for me either. Here's my artifact part of the build spec.

artifacts:
  files:
    - '**/*'
  secondary-artifacts: 
    artifact1:
      files: 
        - '**/*'
      base-directory: 'deploy'
      name: 'BuildArtifact'
    artifact2: 
      files: 
        - '**/*'
      base-directory: 'rocketml_orchestration/orc_deploy'
      name: 'OrcArtifact'

Error:

2716 | [Container] 2020/08/29 14:47:27 Phase context status code: Message:
2717 | [Container] 2020/08/29 14:47:27 Preparing to copy secondary artifacts BuildArtifact
2718 | [Container] 2020/08/29 14:47:27 Phase complete: UPLOAD_ARTIFACTS State: FAILED
2719 | [Container] 2020/08/29 14:47:27 Phase context status code: CLIENT_ERROR Message: no definition for secondary artifact BuildArtifact in buildspec

[gsi-joseespinosa]

This is due to a bad example in the documentation. This will NOT work:

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  secondary-artifacts:
    artifact1:
      files:
        - directory/file
      name: secondary-artifact-name-1
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

The documentation, obscurely, points it out: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

artifacts/files
Required sequence. Represents the locations that contain the build output artifacts in the build environment. Contains a sequence of scalars, with each scalar representing a separate location where CodeBuild can find build output artifacts, relative to the original build location or, if set, the base directory.

So this is an edited version that does work:

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  files:
    - directory/*
  secondary-artifacts:
    artifact1:
      files:
        - directory/file
      name: secondary-artifact-name-1
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

Exactly with this script update build step works. Documentation must to update as well adding missing commands.

[subinataws]

@ioMatrix @gsi-joseespinosa - Thanks for pointing out the issue with the documentation. We'll get that fixed

[awszhen]

The documentation in https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html for secondary artifacts has already been updated. Mark this issue as resolved.

[subinataws]

@gsi-joseespinosa - The buildspec shown in https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html already had the correct syntax. Where did you see the buildspec that had directory/* called out instead of **/*?

version: 0.2

phases:
build:
  commands:
    - echo Building...
artifacts:
  files:
    - '**/*'
  secondary-artifacts:
    artifact1:
      files:
        - directory/file1
      name: secondary-artifact-name-1
    artifact2:
      files:
        - directory/file2
      name: secondary-artifact-name-2

[ioMatrix]

When I looked at it in Jun it was not correct. I figured it out by reading
the spec and hence my response a few months back. I'm happy it's updated.

On Thu., Sep. 24, 2020, 5:03 p.m. Subin Mathew [e-mail]
wrote:

@gsi-joseespinosa https://github.com/gsi-joseespinosa - The buildspec
shown in
https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
already had the correct syntax. Where did you see the buildspec that had
directory/* called out instead of */?

version: 0.2

phases:
build:
commands:

• echo Building...
  artifacts:
  files:
• '*/'
  secondary-artifacts:
  artifact1:
  files:
• directory/file1
  name: secondary-artifact-name-1
  artifact2:
  files:
• directory/file2
  name: secondary-artifact-name-2

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-codebuild-docker-images/issues/128#issuecomment-698647724,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEFQYZE5S3XYQUY3ZVIF2M3SHPM5HANCNFSM4F3BLBQQ
.