Aws-cdk: [aws-stepfunctions-tasks]: How can we select a whole json attribute while putting data in dynamodb using step function

[devendra-kumar1]

:question: General Issue

How can we select a whole json attribute while putting data in dynamodb using step function

The Question

Environment

• CDK CLI Version: 1.52.0
• Module Version: 1.52.0
• Node.js Version: 12
• OS: Windows 10
• Language (Version): TypeScript (3.9.0)

Other information

I have a requirement where i have to persist a whole json attribute in dynamodb using step function.
Below is the code i am using for putting data.

*JSON Attribute *

{
"mgmtPolicy": {
        "name": "DOC",
        "encryptionPolicy": "DEFAULT",
        "retentionPolicy": "DEFAULT",
        "archivalPolicy": "DEFAULT",
        "basePath": "doc",
        "path": "POLICY-DOC/",
        "createdDate": 1579866334000,
        "modifiedDate": 1579866334000
    }
}

to assign in item attribute of DynamoPutItem construct i am putting like this

DynamoAttributeValue.fromString(JsonPath.stringAt('$.mgmtPolicy'))
it is throwing error while executing
could not be used to start the Task: [The value for the field 'S' must be a STRING]"
}

2 Issue 2
When i am trying to set item as number using below code
DynamoAttributeValue.fromNumber(JsonPath.numberAt('$.mgmtPolicy.createdDate'))
In step function definition it is getting set as default value
-1.8881545897087976e+289

Getting below error

Number overflow. Attempting to store a number with magnitude larger than supported range.

[michaelwiles]

So I can tell you how to do the second one:

DynamoAttributeValue.numberFromString(JsonPath.stringAt('$.mgmtPolicy.createdDate'))

Will work - but dynamo only accepts number rendered as strings (it does not accept numbers as numbers). This is a dynamo issue and has nothing to do with step functions or cdk - so as it stands your number: 1579866334000 it will reject (dynamo will reject). That has to be in quotes: "1579866334000" (I share your pain on this).

As for your first question... I don't think that's possible though I'd really like it to be possible. It is something that I am going to need as well...

[shivlaks]

@devendra-kumar1 for your second issue, you can use the solution mentioned by @michaelwiles

As for your first issue, you're trying to allocate an object into a string and it's not expected to work, what's the string value that you'd like to put into DynamoDB? / how does this work directly in Step Functions outside of the CDK. Can an object be allocated to a string?

[michaelwiles]

@shivlaks I suspect the requirement is that a dynamo db column is loaded with the string:

"mgmtPolicy": {
"name": "DOC",
"encryptionPolicy": "DEFAULT",
"retentionPolicy": "DEFAULT",
"archivalPolicy": "DEFAULT",
"basePath": "doc",
"path": "POLICY-DOC/",
"createdDate": 1579866334000,
"modifiedDate": 1579866334000
}

But as you say, I don't think it's possible with step functions...

[devendra-kumar1]

@michaelwiles @shivlaks Thanks for your reply.
yes as an alternative i introduced a lambda which will give me the data in required format to put in dynamodb using step function task.
for 2nd point i stringyfy the object using JSON.stringyfy at lambda and it worked.

Actually i was looking for a function which does stringyfy out of the box and return the result in JsonPath.

[shivlaks]

@devendra-kumar1 gotcha, since the CDK is only generating the state machine definition, we would need to produce syntax that is valid in Amazon States Language (ASL) for taking Json path from state input and stringifying it.

I don't believe this can be done in Step Functions today, but I could be wrong. If it can, we should turn this into a feature request.

[michaelwiles]

@devendra-kumar1 If you want you could use an EvaluateExpression step... with this you can run arbitrary nodejs code.

The secret sauce is that in the background a lambda will be deployed and hooked up (and you won't have to).

python code:

chain = EvaluateExpression(self, 'AddTImeToTtl', expression='JSON.stringify($.mgmtPolicy)',
            input_path='$',)

@shivlaks the principle that I've come to is that anything that requires more than just referencing is not possible in a step function. So even string concatenation can't be done. So something like this would not be possible either.

[devendra-kumar1]

@michaelwiles Thanks for your input. Will give a try on the code snippet you shared.

@shivlaks Thanks for your clarification. In future this feature will be integrated with step function then it would be a great help. For now i am sorted with help of lambda to transform my input to dynamo db put item task.
We can close this issue or kept it as feature request. It would be final call from you guyz. @michaelwiles @shivlaks

[shivlaks]

@devendra-kumar1 perfect I'll close out this issue in that case. I'll also engage the Step Functions team to ensure the feature request is tracked. I believe it won't be significant effort from the cdk side of things if/when it's supported in Step Functions.

[michaelwiles]

@devendra-kumar1 looks like this is now supported...

[devendra-kumar1]

Thanks @michaelwiles for update 👍

[michaelwiles]

Though I'm not 100% sure you'd be able to use it yet as the existing cdk
mechanisms may not let you.

On Wed, 19 Aug 2020, 17:41 devendra-kumar1, [e-mail]
wrote:

Thanks @michaelwiles https://github.com/michaelwiles for update 👍

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-cdk/issues/9181#issuecomment-676503422, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAODX5DZSVXUMSNVH4J7ZKDSBPXBPANCNFSM4PDNDNCQ
.