Aws-cdk: Fluent bit (Firelens) config does not allow local file path

[computerwizjared]

Reproduction Steps

const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDefinition', {
  cpu: 1024,
  memoryLimitMiB: 2048
});
taskDefinition.addFirelensLogRouter('FluentBitLogRouter', {
  image: ecs.ContainerImage.fromRegistry('amazon/aws-for-fluent-bit:latest'),
  essential: true,
  firelensConfig: {
    type: ecs.FirelensLogRouterType.FLUENTBIT,
    options: {
      enableECSLogMetadata: true,
      configFileType: ecs.FirelensConfigFileType.FILE,
      configFileValue: '/fluent-bit/configs/parse-json.conf'
    }
  }
});

Error Log

 11/15 | 6:15:58 AM | CREATE_FAILED        | AWS::IAM::Policy                          | TaskDefinition/ExecutionRole/DefaultPolicy (TaskDefinitionExecutionRoleDefaultPolicy1F3406F5) Resource /fluent-bit/configs/parse-json.conf must be in ARN format or "*". (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: 6b7a29dd-17db-48f8-bf7e-2c1927b9f94d)
    new Policy (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-iam/lib/policyp.ts:114:22)
    \_ Role.addToPolicy (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-iam/lib/role.ts:329:28)
    \_ Function.addToPrincipal (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-iam/lib/grant.ts:147:61)
    \_ Function.addToPrincipalOrResource (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-iam/lib/grant.ts:117:26)
    \_ Import.grant (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-ecr/lib/repository.ts:227:22)
    \_ Import.grantPull (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-ecr/lib/repository.ts:240:22)
    \_ EcrImage.bind (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-ecs/lib/images/ecr.ts:29:21)
    \_ new ContainerDefinition (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-ecs/lib/container-definition.ts:361:36)
    \_ FargateTaskDefinition.addContainer (/opt/atlassian/pipelines/agent/build/cdk/node_modules/@aws-cdk/aws-ecs/lib/base/task-definition.ts:366:12)
    \_ new HeliosStack (/opt/atlassian/pipelines/agent/build/cdk/lib/helios-stack.ts:51:44)
    \_ Object.<anonymous> (/opt/atlassian/pipelines/agent/build/cdk/bin/helios.ts:11:1)
    \_ Module._compile (internal/modules/cjs/loader.js:955:30)
    \_ Module.m._compile (/opt/atlassian/pipelines/agent/build/cdk/node_modules/ts-node/src/index.ts:814:23)
    \_ Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    \_ Object.require.extensions.<computed> [as .ts] (/opt/atlassian/pipelines/agent/build/cdk/node_modules/ts-node/src/index.ts:817:12)
    \_ Module.load (internal/modules/cjs/loader.js:811:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:723:14)
    \_ Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
    \_ main (/opt/atlassian/pipelines/agent/build/cdk/node_modules/ts-node/src/bin.ts:226:14)
    \_ Object.<anonymous> (/opt/atlassian/pipelines/agent/build/cdk/node_modules/ts-node/src/bin.ts:485:3)
    \_ Module._compile (internal/modules/cjs/loader.js:955:30)
    \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    \_ Module.load (internal/modules/cjs/loader.js:811:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:723:14)
    \_ Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
    \_ /usr/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

Environment

• CLI Version : 1.22.0
• Framework Version: 1.22.0
• OS : Alpine Linux
• Language : TypeScript

Other

Trying to use Fluent bit and Firelens to pipe logs to Datadog, and getting this error. I think CDK is trying to parse the path as an S3 ARN for some reason, but this is actually a local path within the Firelens container. This usage is described here: https://github.com/aws-samples/amazon-ecs-firelens-examples/tree/master/examples/fluent-bit/parse-json

---

This is :bug: Bug Report

[efekarakus]

This PR should fix the issue: https://github.com/aws/aws-cdk/pull/6322

[rcollette]

Because I couldn't find a direct way to configure json-parse.conf via a logDriver configuration, and my search took me through this path, I think this could help others.

https://stackoverflow.com/questions/64299664/how-to-configure-aws-cdk-applicationloadbalancedfargateservice-to-log-parsed-jso