Aws-cdk: [pipelines] generate CodeBuild reports within @aws-cdk/cdk-pipelines

[nbaillie]

Description

To be able to easily use @aws-cdk/assert within @aws-cdk/cdk-pipelines. and use the reports function of CodeBuild.

Use Case

Running tests on CDK using @aws-cdk/assert to increase the ability to Fail Fast and implement TDD.

Proposed Solution

I have considered 2 options so far: ( there may be other options, and there may well be some other/better way to do this )

#

Option1: add a testAction that adds another stage to codepipeline using another codebuild project :

const pipeline = new CdkPipeline(this, 'Pipeline', {
      pipelineName: 'PipelineWithTests',
      sourceAction: ...
      synthAction: ...
      testAction: simpleTestAction(...)
    });

Where the new testAction includes support for reports in the buildSpec.

Option2: Enable adding reports to the existing SimpleSynthAction(and npm yarn variants) buildSpec:
adding the needed changes to the build step commands "yarn build && yarn test".
and adding along the lines of the bellow to the build spec perhaps with a testEnable Flag in the SynthAction.

reports:
  jest_reports:
    files:
      - <report filename>
    file-format: JUNITXML
    base-directory: <test report directory>

Other

I understand that this can be done by creating a codeBuild on its own and adding to the pipeline, but that breaks the elegance and simplicity of using cdk-pipelines which dose a great job at simplifying the codepipeline/buid experience.

I would expect that jest tests would be ran by yarn or npm as described in the @aws-cdk/assert docs.
Reference: test-report-jest

---

This is a :rocket: Feature Request

[BrianFarnhill]

Looking at this, I think it could be made pretty simple by separating this out in to two different changes:

1. Add support for a testCommand attribute to go along with the ones like install, build etc.
2. Add an ability to specify report outputs

the report outputs could look like this:

    const synthAction = pipelines.SimpleSynthAction.standardNpmSynth({
        buildCommand: 'npm run build',
        testCommand: 'npm run test', //This would be new and doesn't exist today
        testReports: [{
            name: "GroupName" | "GroupArn", // standard syntax for the buildspec, if its not the ARN then the group gets created based on the name
            reportType: "Test" | "CodeCoverage",
            files: string[],
            fileFormat: "JUNITXML" | "OTHER_TYPES" | etc...,
            baseDirectory: string,
        }]
    });

This way the only real change we are making is that the testReports attribute feeds in to the generation of the buildspec, and you can still run your test command however you need to based on the type of project you have. This should mean we don't hit issues around different languages and ways of running tests (since you just use the testCommand to trigger the execution). The only thing I can think of here is that we would need to look at making sure that if you specify the testReports property I'm talkng about adding here, that we automatically add IAM permissions to put stuff in those groups in to the IAM policy for the build as well.

That's my thoughts on it at least - I'll keep giving it some thought though, and keen to hear other opinions.

[nbaillie]

@BrianFarnhill
if you drop back to SimpleSynthAction from SimpleSynthAction.standardNpmSynth it already has the testCommand available. So would just be to do the second part as you have described above.

synthAction: new SimpleSynthAction({
        sourceArtifact,
        cloudAssemblyArtifact,
        installCommand: "yarn install --frozen-lockfile",
        buildCommand: `yarn build`,
        testCommands: [`yarn test unit`],
        synthCommand: `npx cdk synth`
}),

would become

synthAction: new SimpleSynthAction({
        sourceArtifact,
        cloudAssemblyArtifact,
        installCommand: "yarn install --frozen-lockfile",
        buildCommand: `yarn build`,
        testCommands: [`yarn test unit`],
        synthCommand: `npx cdk synth`,
        testReports: [{
            name: "GroupName" | "GroupArn",
            reportType: "Test" | "CodeCoverage",
            files: string[],
            fileFormat: "JUNITXML" | "OTHER_TYPES" | etc...,
            baseDirectory: string,
        }]
}),