Aws-cdk: Unconfigurable /.cdk path

[DerkSchooltink]

I'm trying to deploy my CDK deployment from a Jenkins node, which is running on an EC2 instance. I provide the deployment with a user ID and default region. CDK then tries to resolve my account using STS lookup and store it in some sort of cache. Apparently it tries to create a directory for this (~/.cdk), which is not allowed in that path. The path is the root of the Jenkins node. Obviously this shouldn't be attempted to be created in the root but instead in something like /var/lib/jenkins/.

I tried setting the CDK_HOME to another location but that doesn't seem to affect the location for creating the .cdk directory.

Reproduction Steps

1. Run Jenkins in an EC2 instance
2. Provide defaults for CDK home, account ID and region
3. Run cdk deploy -v

Error Log

CDK toolkit version: 1.9.0 (build 30f158a)

Command line arguments: { _: [ 'synth' ],

  'ignore-errors': false,

  ignoreErrors: false,

  json: false,

  j: false,

  verbose: true,

  v: true,

  ec2creds: undefined,

  i: undefined,

  'version-reporting': undefined,

  versionReporting: undefined,

  'path-metadata': true,

  pathMetadata: true,

  'asset-metadata': true,

  assetMetadata: true,

  staging: true,

  defaultAccount: <account id redacted>,

  defaultRegion: 'eu-central-1',

  home: '/var/lib/jenkins/',

  'role-arn': undefined,

  r: undefined,

  roleArn: undefined,

  '$0': '/usr/local/bin/cdk' }

Determining whether we're on an EC2 instance.

cdk.json: {

  "app": "python3 app.py"

}

Looks like EC2 instance.

merged settings: { versionReporting: true,

  pathMetadata: true,

  output: 'cdk.out',

  app: 'python3 app.py',

  context: {},

  tags: [],

  assetMetadata: true,

  toolkitBucket: {},

  staging: true }

Unable to determine AWS region from environment or AWS configuration (profile: "default")

Setting "CDK_DEFAULT_REGION" environment variable to undefined

Resolving default credentials

Looking up default account ID from STS

Unable to determine AWS region from environment or AWS configuration (profile: "default")

Default account ID: <account id redacted>

Unable to determine the default AWS account (did you configure "aws configure"?): { [Error: EACCES: permission denied, mkdir '/.cdk'] errno: -13, code: 'EACCES', syscall: 'mkdir', path: '/.cdk' }

Setting "CDK_DEFAULT_ACCOUNT" environment variable to undefined

context: { 'aws:cdk:enable-path-metadata': true,

  'aws:cdk:enable-asset-metadata': true }

outdir: cdk.out

env: { CDK_DEFAULT_REGION: undefined,

  CDK_DEFAULT_ACCOUNT: undefined,

  CDK_CONTEXT_JSON:

   '{"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',

  CDK_OUTDIR: 'cdk.out' }

Some context information is missing. Fetching...

Need to perform AWS calls for account <account id redacted>, but no credentials found. Tried: default credentials.

Error: Need to perform AWS calls for account <account id redacted>, but no credentials found. Tried: default credentials.

    at CredentialsCache.getCredentials (/usr/local/lib/node_modules/aws-cdk/lib/api/util/sdk.ts:261:11)

    at process._tickCallback (internal/process/next_tick.js:68:7)

Environment

• CLI Version :1.9.0 (build 30f158a)

Proposed solution

It seems like the CDK_HOME environment variable is honoured for creating the /.cdk directory. Ideally this would be configurable, either through aforementioned environment variable or in some other way. But perhaps I'm misunderstanding the CDK_HOME environment variable, this is just a suggestion of course!

This is :bug: Bug Report

[nmussy]

Could you send the output of cdk deploy --debug?

[DerkSchooltink]

Could you send the output of cdk deploy --debug?

The output of running cdk deploy -v is already attached to the issue, using --debug doesn't do anything and doesn't appear in the cdk deploy --help. What exactly are you after?

[nmussy]

I was hoping to see those messages: https://github.com/aws/aws-cdk/blob/dcba28d0fe545e8c65edf14a4ea85c86648f2e19/packages/aws-cdk/lib/api/util/sdk.ts#L245

[DerkSchooltink]

I was hoping to see those messages:

https://github.com/aws/aws-cdk/blob/dcba28d0fe545e8c65edf14a4ea85c86648f2e19/packages/aws-cdk/lib/api/util/sdk.ts#L245

Ah, it seems like it might be failing on previous steps. I'm running cdk synth -v && cdk diff -v && cdk deploy -v with a docker run. The synth seems to trigger this error. Running cdk deploy -v alone results into this:

CDK toolkit version: 1.9.0 (build 30f158a)

Command line arguments: { _: [ 'deploy' ],

  'ignore-errors': false,

  ignoreErrors: false,

  json: false,

  j: false,

  verbose: true,

  v: true,

  ec2creds: undefined,

  i: undefined,

  'version-reporting': undefined,

  versionReporting: undefined,

  'path-metadata': true,

  pathMetadata: true,

  'asset-metadata': true,

  assetMetadata: true,

  staging: true,

  ci: false,

  defaultAccount: <account id redacted>,

  defaultRegion: 'eu-central-1',

  home: '/var/lib/jenkins',

  'role-arn': undefined,

  r: undefined,

  roleArn: undefined,

  'build-exclude': [],

  E: [],

  buildExclude: [],

  '$0': '/usr/local/bin/cdk' }

Determining whether we're on an EC2 instance.

cdk.json: {

  "app": "python3 app.py"

}

Looks like EC2 instance.

merged settings: { versionReporting: true,

  pathMetadata: true,

  output: 'cdk.out',

  app: 'python3 app.py',

  context: {},

  tags: [],

  assetMetadata: true,

  toolkitBucket: {},

  staging: true }

Unable to determine AWS region from environment or AWS configuration (profile: "default")

Setting "CDK_DEFAULT_REGION" environment variable to undefined

Resolving default credentials

Looking up default account ID from STS

Unable to determine AWS region from environment or AWS configuration (profile: "default")

Default account ID: <account id redacted>

Unable to determine the default AWS account (did you configure "aws configure"?): { [Error: EACCES: permission denied, mkdir '/.cdk'] errno: -13, code: 'EACCES', syscall: 'mkdir', path: '/.cdk' }

Setting "CDK_DEFAULT_ACCOUNT" environment variable to undefined

context: { 'aws:cdk:enable-path-metadata': true,

  'aws:cdk:enable-asset-metadata': true }

outdir: cdk.out

env: { CDK_DEFAULT_REGION: undefined,

  CDK_DEFAULT_ACCOUNT: undefined,

  CDK_CONTEXT_JSON:

   '{"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true}',

  CDK_OUTDIR: 'cdk.out' }

Some context information is missing. Fetching...

Need to perform AWS calls for account <account id redacted>, but no credentials found. Tried: default credentials.

Error: Need to perform AWS calls for account <account id redacted>, but no credentials found. Tried: default credentials.

    at CredentialsCache.getCredentials (/usr/local/lib/node_modules/aws-cdk/lib/api/util/sdk.ts:261:11)

    at process._tickCallback (internal/process/next_tick.js:68:7)

I don't see your specified message though. It just seems to try to resolve the credentials through some internal STS call.

[nmussy]

I think I found the problematic line: https://github.com/aws/aws-cdk/blob/467d855577272db4a1ce1b88fcc097e09b14a8ad/packages/aws-cdk/lib/api/util/account-cache.ts#L24

I'll make a PR a little later today

[cristianrat]

Also hitting this issue (deploying from Jenkins pipeline, agent is a self build cdk image with all my requirements)

Solution (if I can call it that, not ideal) is to create that dir and give it 777 permissions.
Still want a proper fix, don't want that kind of stuff in my Dockerfile(s) :)