Aws-cdk: Subscription Filter not added to cloudformation stack

[fedeBollo]

:question: Subscription Filter Object not create relative CloudFormation resources

The Question

Using SubscriptionFilter object, the relative resource are not created in cloudformation stack. Relative error are not showed.
Synth and deploy command doesn't return any error.

-->

Environment

• **CDK CLI Version:1.13.1
• **Module Version: aws-logs
• **OS: MacOSX
• **Language: Javascipt

Other information

Here is the code:

let cloudWatchGroup = cl_logs.LogGroup.fromLogGroupArn(this, 'lambda-log-group', res);
let filterPattern = cl_logs.FilterPattern.allEvents();
const subscriptionFilter = new cl_logs.SubscriptionFilter(this, 'elastic-search-subscription',{ destination: lambdaDestination, filterPattern: filterPattern, logGroup: cloudWatchGroup, });
`

[nmussy]

Hey @fedeBollo,

Trying to read your reproduction stack was a little hard, so here is what I came up with, tell me if I'm too far off your use-case:

import {FilterPattern, LogGroup, SubscriptionFilter} from '@aws-cdk/aws-logs';
import {Code, Function as LambdaFn, Runtime} from '@aws-cdk/aws-lambda';
import {LambdaDestination} from '@aws-cdk/aws-logs-destinations';
import {Construct, Stack, StackProps} from '@aws-cdk/core';
// ...
const cloudWatchGroup = LogGroup.fromLogGroupArn(
  this,
  'lambda-log-group',
  'arn:aws:logs:us-east-1:123456789012:log-group:my-log-group',
);

const filterPattern = FilterPattern.allEvents();

const lambda = new LambdaFn(this, 'function', {
  runtime: Runtime.NODEJS_8_10,
  handler: 'hello.handler',
  code: Code.fromInline('() => {}'),
});

new SubscriptionFilter(this, 'elastic-search-subscription', {
  destination: new LambdaDestination(lambda),
  filterPattern: filterPattern,
  logGroup: cloudWatchGroup,
});

When I run cdk synth, I get the following:

// ...
  elasticsearchsubscription333E993D:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      DestinationArn:
        Fn::GetAtt:
          - functionF19B1A04
          - Arn
      FilterPattern: ""
      LogGroupName: test
    Metadata:
      aws:cdk:path: TestStack10/elastic-search-subscription/Resource

Is this not what you are getting?

EDIT: I've edited my example to use a valid log group ARN, without it the logGroupName property wasn't set.

[fedeBollo]

Thanks for the reply.

Actually editing my code, I get the errore showed here:

/app/www/infocert/iqp_cloudwatch_event/node_modules/@aws-cdk/core/lib/construct.js:344
for (const reference of node._references) {
^

TypeError: node._references is not iterable

This error is show if I use "fromFunctionArn" method and also if I create a new LambdaFunction Object as you made in your examples.

Maybe is something depending on jsii implementation for Javascript?

Thanks for the help, really appreciate.

[nmussy]

No problem @fedeBollo,

I'm fairly certain the JSII TypeScript runtime is the same as the JavaScript one, except for the exported d.ts types.

Could you send a complete and minimal reproduction of your error?

[fedeBollo]

Hello @nmussy, sorry for the late answer.
Here is the complete trace error.

{project_dir}/node_modules/@aws-cdk/core/lib/construct.js:344
            for (const reference of node._references) {
                                         ^

TypeError: node._references is not iterable
    at recurse ({project_dir}/node_modules/@aws-cdk/core/lib/construct.js:344:42)
    at recurse ({project_dir}/node_modules/@aws-cdk/core/lib/construct.js:348:17)
    at ConstructNode.get references [as references] ({project_dir}/node_modules/@aws-cdk/core/lib/construct.js:351:9)
    at IqpCloudwatchEventStack.prepare ({project_dir}/node_modules/@aws-cdk/core/lib/stack.js:349:37)
    at Function.prepare ({project_dir}/node_modules/@aws-cdk/core/lib/construct.js:89:27)
    at Function.synth ({project_dir}/node_modules/@aws-cdk/core/lib/construct.js:52:14)
    at App.synth ({project_dir}/node_modules/@aws-cdk/core/lib/app.js:71:52)
    at Object.<anonymous> ({project_dir}/bin/iqp_cloudwatch_event.js:15:5)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)

Here you can find the complete CDK code, with all the method inside the constructor:

``javascript let lambdaArn = "lambda-arn"; let lambdaElasticSearch = lambda.Function.fromFunctionArn(this, 'lambda-elasticsearch', lambdaArn); let cloudWatchGroup = cl_logs.LogGroup.fromLogGroupArn(this, 'log-group', "log-group-arn"); let filterPattern = cl_logs.FilterPattern.allEvents(); const subscriptionFilter = new cl_logs.SubscriptionFilter(this, 'elastic-search-subscription',{ destination: new cl_logs_destination.LambdaDestination(lambdaElasticSearch), filterPattern: filterPattern, logGroup: cloudWatchGroup, }); new cdk.CfnOutput(this, 'cloudwatch-subscription', { value: cloudWatchGroup.logGroupName });

[nmussy]

Alright, I've been able to reproduce the issue, it does seem to be JS-related. Starting from scratch:

$ cdk init --language javascript
$ npm i @aws-cdk/aws-lambda @aws-cdk/aws-logs @aws-cdk/aws-logs-destinations

const cdk = require('@aws-cdk/core');
const lambda = require('@aws-cdk/aws-lambda');
const logs = require('@aws-cdk/aws-logs');
const logs_destinations = require('@aws-cdk/aws-logs-destinations');


class DeployJsStack extends cdk.Stack {
  /**
   *
   * @param {cdk.Construct} scope
   * @param {string} id
   * @param {cdk.StackProps=} props
   */
  constructor(scope, id, props) {
    super(scope, id, props);

    const lambdaElasticSearch = lambda.Function.fromFunctionArn(
        this,
        'lambda-elasticsearch',
        'arn:b:c:d:e:f',
    );
    const cloudWatchGroup = logs.LogGroup.fromLogGroupArn(
        this,
        'log-group',
        'arn:aws:logs:us-east-1:123456789012:log-group:my-log-group',
    );
    const filterPattern = logs.FilterPattern.allEvents();
    new logs.SubscriptionFilter(
        this,
        'elastic-search-subscription',
        {
          destination: new logs_destinations.LambdaDestination(lambdaElasticSearch),
          filterPattern: filterPattern,
          logGroup: cloudWatchGroup,
        },
    );

    new cdk.CfnOutput(this, 'cloudwatch-subscription', {
      value: cloudWatchGroup.logGroupName,
    });
  }
}

module.exports = { DeployJsStack }

$ npx cdk synth

node_modules\@aws-cdk\core\lib\construct.ts:423
      for (const reference of node._references) {
                                   ^
TypeError: node._references is not iterable
    at recurse (node_modules\@aws-cdk\core\lib\construct.ts:423:36)
    at recurse (node_modules\@aws-cdk\core\lib\construct.ts:428:9)
    at ConstructNode.get references [as references] (node_modules\@aws-cdk\core\lib\construct.ts:432:5)
    at DeployJsStack.prepare (node_modules\@aws-cdk\core\lib\stack.ts:487:33)
    at Function.prepare (node_modules\@aws-cdk\core\lib\construct.ts:84:28)
    at Function.synth (node_modules\@aws-cdk\core\lib\construct.ts:41:10)
    at App.synth (node_modules\@aws-cdk\core\lib\app.ts:128:36)
    at process.App.process.once (node_modules\@aws-cdk\core\lib\app.ts:111:45)
    at Object.onceWrapper (events.js:281:20)
    at process.emit (events.js:193:13)

Really not sure what's going on there

[fedeBollo]

Thanks @nmussy. If you know a workaround also, please let me know, or I just wait for the fix.

Appreciate your help.

[nmussy]

Switching to TypeScript seems like it would solve your issue.

If you don't feel like learning it, you could add allowJs: true to your tsconfig.json to keep your existing JS file, but the issue might still be there? I haven't tested it.

[fedeBollo]

Hello @nmussy, your assumption are correct: switching to TypeScript with the same JS code and the allowJs field, solve the issue. Seems a non-related code error, but something with JS kernel or interpreter.
Here is the template:

Resources:
  elasticsearchsubscription333E993D:
    Type: AWS::Logs::SubscriptionFilter
    Properties:
      DestinationArn: arn:aws:lambda:eu-west-1:956251722582:function:LogsToElasticsearch_svts-monitoring
      FilterPattern: ""
      LogGroupName: /aws/lambda/IqpCloudwatchEventStack-iqpcloudwatchrulelambdaDDB-50WCW1RC9OOM:*
    Metadata:
      aws:cdk:path: SubscriptionTypescriptStack/elastic-search-subscription/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.11.0,@aws-cdk/assets=1.13.1,@aws-cdk/aws-cloudwatch=1.13.1,@aws-cdk/aws-ec2=1.13.1,@aws-cdk/aws-events=1.13.1,@aws-cdk/aws-iam=1.13.1,@aws-cdk/aws-kms=1.13.1,@aws-cdk/aws-lambda=1.13.1,@aws-cdk/aws-logs=1.13.1,@aws-cdk/aws-logs-destinations=1.13.1,@aws-cdk/aws-s3=1.13.1,@aws-cdk/aws-s3-assets=1.13.1,@aws-cdk/aws-sqs=1.13.1,@aws-cdk/aws-ssm=1.13.1,@aws-cdk/core=1.13.1,@aws-cdk/cx-api=1.13.1,@aws-cdk/region-info=1.13.1,jsii-runtime=node.js/v10.15.1
    Condition: CDKMetadataAvailable
Outputs:
  cloudwatchsubscription:
    Value: /aws/lambda/IqpCloudwatchEventStack-iqpcloudwatchrulelambdaDDB-50WCW1RC9OOM:*
Conditions:
  CDKMetadataAvailable:
    Fn::Or:
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-northeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-1
          - Fn::Equals:
              - Ref: AWS::Region
              - ap-southeast-2
          - Fn::Equals:
              - Ref: AWS::Region
              - ca-central-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - cn-northwest-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-central-1
      - Fn::Or:
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-north-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-2
          - Fn::Equals:
              - Ref: AWS::Region
              - eu-west-3
          - Fn::Equals:
              - Ref: AWS::Region
              - me-south-1
          - Fn::Equals:
              - Ref: AWS::Region
              - sa-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-east-2
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-1
          - Fn::Equals:
              - Ref: AWS::Region
              - us-west-2

I will check through the code and if I find something I will let you know.
Thanks again.

[nmussy]

Alright, I've found the issue. The template is being initialized with outdated aws-cdk and @aws-cdk/core packages if your cdk CLI is outdated. Upgrading them solves the issue.

$ cdk --version
0.36.0 (build 6d38487)

$ cdk init --language javascript
Applying project template app for javascript
Executing npm install...
npm notice created a lockfile as package-lock.json. You should commit this file.

$ npm outdated
Package        Current   Wanted   Latest  Location
@aws-cdk/core   0.36.2   0.36.2   1.13.1  deploy
@types/node    8.10.45  8.10.45  12.11.1  deploy
aws-cdk         0.36.2   0.36.2   1.13.1  deploy

$ npm i aws-cdk@latest @aws-cdk/core@latest
+ [email protected]
+ @aws-cdk/[email protected]

$ npm outdated
Package      Current   Wanted   Latest  Location
@types/node  8.10.45  8.10.45  12.11.1  deploy

I'm used to using local packages with npx, which is why mine is so old. Yours is probably more recent, but doesn't have the recently merged #2821, which removed _references from ConstructNode.

To upgrade your global CDK CLI version:

npm i -g aws-cdk@latest

Not sure what we should do about this issue.

[fedeBollo]

Thansk @nmussy, this solve issue! I have an "auto-upgrade" script that periodically check if some of my packages is outdated and automatically upgrade it, maybe something is not working there...
This complete a really complicated stack I'm actually developing, really appreciate your help.

Thanks again!

[nmussy]

No problem. 👍