Aws-cdk: asset staging: do not recurse into the output directory

[jeshan]

:bug: Bug Report

What is the problem?

I have hit a corner case with the code related to copy directories. It's going in recursively until Node can no longer handle such long file names.

https://github.com/aws/aws-cdk/blob/16eb65867b79798779bf98a0316b1a05915720f1/packages/%40aws-cdk/assets/lib/fs/copy.ts#L7-L18

The offending snippet is when I had a docker asset declared as follows:

        new DockerImageAsset(this, 'app-image-asset', {
            directory: '.',
            repositoryName: 'my-repo/app',
        });

Running cdk deploy showed me this error:

Error: ENAMETOOLONG: name too long, mkdir 'cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.7e4b3cc320bdf668daa4c0d065bb1a9841162c955cb702f7a01d7d3d7e88dbfe/cdk.out/asset.03918990104d8973acd2ad0abe022844192fcce618cf74f7734a7077f4111d1f'
    at Object.mkdirSync (fs.js:750:3)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:39:16)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)
    at copyDirectory (/mnt/sdb/ideaProjects/my-project/node_modules/@aws-cdk/assets/lib/fs/copy.js:40:13)

_(Declaring the directory as its full path did not work neither.)_
This is quite a serious issue for me, personally.

A possible solution is to exclude cdk.out. I think it's reasonable to assume that cdk.out name is reserved for this project.

As a temporary workaround for me, I've edited the code so that it looks like the following:

    const exclude = options.exclude || ['cdk.out'];

Another possible solution is to avoid recursion if we've already seen the directory in question.

Apparently, it's an issue where the docker build context is the same path as the cdk project.
That said, I'm not sure if the above is enough to reproduce the issue. When I comment the code to create the docker asset, the issue disappears (probably more of a symptom rather than the cause). I've checked if symlinks are the issue (apparently, it's not)

Environment

• CDK CLI Version: 1.6.1
• OS: all
• Language: all

[rhboyd]

This usually happens when you try to build an image from within a directory you want included in the image as well. It creates a recursive path that never ends.

[joezhou888]

I had the same problem.

The only way to make it work is:
Project
-App Directory
-Env Directory

[jeshan]

Thanks Joe.
@rhboyd , we could make the exclude directory list an option in cdk.json. What do you think?

[eladb]

This is a bug. It makes sense that when we copy assets into the output directory, we won't descend into the same directory as we copy.

[askulkarni2]

Saw this issue for a similar use case while using the aws-s3-assets module. I was trying to zip up my cdk project into s3 and use it to bootstrap CodeCommit. Excluding cdk.out seems to work.

const asset = new Asset(this, 'CodeAsset', {
      path: path.join(__dirname, '../'),
      exclude: [
        'node_modules',
        '.git',
        'cdk.out'
      ]
    });

[BinaryShrub]

I added cdk* to .dockerignore to get it working:

// Create Fargate Service
const fargateService = new ecs_patterns.NetworkLoadBalancedFargateService(
  this,
  'sample-app',
  {
    cluster,
    taskImageOptions: {
      image: ecs.ContainerImage.fromAsset('./') // blows up without cdk.out in .dockerignore
    }
  }
)

[victorL91]

I bumped into this issue again. It may be stupid but I solved it by making sure that the version of aws-cdk on my machine was the same as on the package.json. I was not bootstrapping it the correct way.