Aws-cdk: [CDK Pipelines] - Bundling Assets Using Docker Container

[srethira]

I’m trying out new CDK Pipelines in developer preview. I have a question on how to bundle assets that are using docker container during the build process. Currently during “synth_action”, getting following error:

Reproduction Steps

pipelines.CdkPipeline(self,
                       "Pipeline",
                       pipeline_name="Test",
                       cloud_assembly_artifact=cloud_assembly_artifact,
                       source_action=codepipeline_actions.CodeCommitSourceAction(
                               action_name="CodeCommit_Source",
                               repository=code,
                               branch="cdkpipeline",
                              output=source_artifact),
                       synth_action=pipelines.SimpleSynthAction.standard_npm_synth()
                               source_artifact=source_artifact,
                               cloud_assembly_artifact=cloud_assembly_artifact,
                               build_command="npx cdk synth")

Error Log

stderr: docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.   
See 'docker run --help'. 
at AssetStaging.bundle (/tmp/jsii-kernel-O8ZKbG/node_modules/@aws-cdk/core/lib/asset-staging.js:121:19) 
at new AssetStaging (/tmp/jsii-kernel-O8ZKbG/node_modules/@aws-cdk/core/lib/asset-staging.js:38:35)
 at new Asset (/tmp/jsii-kernel-O8ZKbG/node_modules/@aws-cdk/aws-s3-assets/lib/asset.js:21:25)

Environment

• CDK CLI Version: 1.51.0
• Module Version: 1.51.0
• Node.js Version: v14.5.0
• OS: OSX Mojave
• Language (Version): Python (3.7.3)

---

This is :bug: Bug Report

[srethira]

Is there any workaround to run privileged CodeBuild container using these command ?

nohup /usr/local/bin/dockerd --host=unix:///var/run/docker.sock --host=tcp://127.0.0.1:2375 --storage-driver=overlay2& - timeout 15 sh -c "until docker info; do echo .; sleep 1; done"

Since CodeBuild is created dynamically during synth action.

[justin8]

You should be able to add this in the synth definition to get a privileged container:

environment: {privileged: true}

[srethira]

@justin8 argh! I overlooked this property. I leave this ticket open since @eladb mentoined that this feature could be automated.

[eladb]

Since bundling is a framework feature, We should automatically identify if docker is required for asset bundling and configure your synth environment accordingly.

[eikeon]

@justin8, @srethira, Adding environment: {privileged: true} to SimpleSynthAction.standardNpmSynthdidn't seem to help. Was that the wrong place to add it? /me continues to dig

[eikeon]

Ah, I have to add it and deploy before depending on it ;) -- working now.

[rix0rrr]

Since bundling is a framework feature, We should automatically identify if docker is required for asset bundling and configure your synth environment accordingly.

I don't quite get why we would need docker during the synth step. iirc we should only need it during asset packaging.

Are you using the aws-lambda-nodejs package by any chance?

[justin8]

I'm seeing it during the synth when using the aws-lambda-python package. It appears the nodejs one works in a very similar fashion.

[eladb]

Asset bundling uses docker during synthesis.

[blastedcode]

I too was having the same issue when deploying a Ruby Lambda function that required extra Gems and the CDK code was written in Python. It uses what @justin8 its just an example if people need it.

synth_action=pipelines.SimpleSynthAction(
    source_artifact=source_artifact,
    cloud_assembly_artifact=cloud_assembly_artifact,
    install_command='npm install -g aws-cdk && pip install -r requirements.txt',
    synth_command='cdk synth',
    environment={
        'privileged': True
     }
)

The Lambda deployment code is as follows:

aws_lambda.Function(self, 'EmailHandler',
    code=_lambda.Code.from_asset(path.join(os.getcwd(), 'lambda_resources'),
    bundling={
         'image': _lambda.Runtime.RUBY_2_7.bundling_docker_image,
         'command': ['bash', '-c', 'bundle install  && cp -au . /asset-output']
         }),
    runtime=_lambda.Runtime.RUBY_2_7,
    handler='app.lambda_handler'
)

You do have to do 1 deployment synth run with only the privileged code included first otherwise it always fails