Copilot-cli: Expose optional Docker layer caching and custom local caching in pipelines

[bvtujo]

Codebuild has a native local Docker layer cache mode which lets builds take advantage of pulls from previous builds and can help alleviate the rate limits customers run into. We should enable it either by default or through an optional field in pipeline.yml so customers don't have to constantly work around the dockerhub rate limits.

See discussion in #1869 for more context.

[mikelhamer]

@bvtujo if one were to want to work on this, where would be a good place to start?

[bvtujo]

@mikelhamer Thanks so much for your interest! A quick crash course in defining a new manifest field (it can be a little involved, especially when there are complex interactions which require validation or messaging):

• Manifest fields are defined in manifest/pipeline.go.
• Data structures for pipeline Cloudformation generation are defined in deploy/pipeline.go.
• Pipelines are created by cli/pipeline_update.go.
• CFN template is defined in templates/cicd/pipeline_cfn.yml. This directory is also where the default pipeline manifest and buildspec are defined.

If you wanted to work on this, I'd recommend adding a new field to the manifest to handle local cache behaviors, which is converted at deploy time into the CFN template. The rough flow is:

copilot pipeline update
read manifest
unmarshal manifest to `manifest.go` data structure
convert manifest values to deploy.CreatePipelineInput
render CF template
deploy CF template

Other gotchas might be in validation. For example, if buildspec.yml specifies custom local caching but it's not enabled in pipeline.yml. We should validate for incompatible scenarios like this when running copilot pipeline update.

Does that help?

[bvtujo]

In the meantime, you can enable local caching for your build project in the console by following the instructions here.

[mikelhamer]

A million thanks @bvtujo!

[bvtujo]

I think a great first step for this would be to enable Docker layer caching by default in all our Codebuild projects. There may be some validation issues since Docker layer caching is only supported in linux environments, and we've offered the ability to specify the Codebuild image. If customers specify a windows image (for some reason) we should not use Docker caching.

This greatly simplifies the scope of the PR, as it's just modifying the template conditionally based on the image and doesn't even need to touch the manifest package or pipeline update.