Aws-sdk-java-v2: DynamoDB Local

[chrylis]

I am writing a new module that uses DynamoDB, and I would like to use the v2 SDK. The only manageable way to run functional tests on my module is to use the DynamoDBLocal implementation to make sure that round-trip reads and writes behave as expected.

However, I'm not able to find any documentation that mentions local support for the v2 SDK. The DynamoDBEmbedded.create() call in DynamoDBLocal 1.11 returns the v1 AmazonDynamoDB interface, and there don't appear to be adapters to the v2 API. I also haven't been able to find a version 2.x of the DynamoDBLocal dependency.

Is a v2 version of DynamoDBLocal available? If not, is it on the roadmap, and is there a way (maybe using a loopback IP connection) to use the v1 embedded provider with the v2 client APIs?

[chrylis]

(Note: The migration documentation has a link for configuring "side-by-side" v1 and v2, but the link is broken.)

[zoewangg]

We don't own DynamoDBLocal and I will reach out to the service team regarding the migration plan for v2. Marking this as a feature request.

Thank you for reporting the broken link! We will make sure it gets fixed.

[chrylis]

As a followup, while it's a bit slow and requires some hackery to assign dynamic ports and wait until it's ready, I have been able to launch captive Local instances in a test setup and then point a v2 SDK client at them.

[wsargent]

@chrylis Can you share in a project or blog post? I'm running into the same issue.

[wsargent]

Is the recommended option to use https://hub.docker.com/r/amazon/dynamodb-local/ as per https://aws.amazon.com/about-aws/whats-new/2018/08/use-amazon-dynamodb-local-more-easily-with-the-new-docker-image/ ?

[humanzz]

I've just learnt that the existing DynamoDBLocal can be used with v2 clients - just not using DynamoDBEmbedded but using it in server mode.

See https://github.com/aws/aws-sdk-java-v2/blob/93269d4c0416d0f72e086774265847d6af0d54ec/services-custom/dynamodb-enhanced/src/test/java/software/amazon/awssdk/extensions/dynamodb/mappingclient/functionaltests/LocalDynamoDb.java for more an example on using it in server mode.

[LME10]

@humanzz have you got it working connecting v2 clients with existing DynamoDBLocal?
I ran into several issues. One of most common issue I faced was Class Not found exception for model "CancellationReason"
If you got working, could you share link for reference?

[chrylis]

@humanzz I was able to do that back when, but it's very slow compared to an in-memory instance.

[shapan1]

Did anyone find any resolution on this?
@humanzz @chrylis @zoewangg

[humanzz]

@shapan1 I ended up using it in server mode like above. It's working just fine in our unit tests.

[shapan1]

@shapan1 I ended up using it in server mode like above. It's working just fine in our unit tests.

@humanzz Could you please provide more info or reference to your code on this? because I cannot find DynamoDBProxyServer or ServerRunner as this is no longer available in aws sdk 2.

[shapan1]

@shapan1 I ended up using it in server mode like above. It's working just fine in our unit tests.

@humanzz Could you please provide more info or reference to your code on this? because I cannot find DynamoDBProxyServer or ServerRunner as this is no longer available in aws sdk 2.

@humanzz
NVM, found out I had a IntelliJ issue where it was not able to pull the 'com.amazonaws:DynamoDBLocal:1.15.0' dependency. Deleting .idea file and reimporting the project worked.

[zoewangg]

I think DynamoDB Local has added Java SDK v2 support in the latest version.

DynamoDBEmbedded.create().dynamoDbClient();
DynamoDBEmbedded.create().dynamoDbAsyncClient();

[humanzz]

@shapan1 apologies for not getting back to you earlier. Let me know if you need further help.

@zoewangg I don't think that version has been published to Maven. The latest I can see is 1.15.x https://mvnrepository.com/artifact/com.amazonaws/DynamoDBLocal. I am guessing you're referring to 1.16.x?

[zoewangg]

@humanzz Yeah, 1.16.0 is actually available now :) https://github.com/aws/aws-sdk-java-v2/pull/2452

[spanierm]

@zoewangg Thanks for the information about the release of 1.16.0. We were successfully able to start an embedded DynamoDB local instance in a JUnit test. However, once we now face some compiler warning:

w: Some JAR files in the classpath have the Kotlin Runtime library bundled into them. This may cause difficult to debug problems if there's a different version of the Kotlin Runtime library in the classpath. Consider removing these libraries from the classpath
w: /home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar: Library has Kotlin runtime bundled into it

Moreover, all integration tests fail due to conflicting Kotlin standard library files, see the following error details:

***************************
APPLICATION FAILED TO START
***************************
Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
    kotlin.reflect.jvm.internal.impl.descriptors.Visibilities.<clinit>(Visibilities.kt:56)
The following method did not exist:
    'java.util.Map kotlin.collections.MapsKt.createMapBuilder()'
The method's class, kotlin.collections.MapsKt, is available from the following locations:
    jar:file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar!/kotlin/collections/MapsKt.class
    jar:file:/home/markus/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.5.0/29dae2501ca094416d15af0e21470cb634780444/kotlin-stdlib-1.5.0.jar!/kotlin/collections/MapsKt.class
The class hierarchy was loaded from the following locations:
    kotlin.collections.MapsKt: file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar
    kotlin.collections.MapsKt___MapsKt: file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar
    kotlin.collections.MapsKt__MapsKt: file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar
    kotlin.collections.MapsKt__MapsJVMKt: file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar
    kotlin.collections.MapsKt__MapWithDefaultKt: file:/home/markus/.gradle/caches/modules-2/files-2.1/com.amazonaws/DynamoDBLocal/1.16.0/f705aa07cee4eb8199e70a393c5bd8173e7290ac/DynamoDBLocal-1.16.0.jar
Action:
Correct the classpath of your application so that it contains a single, compatible version of kotlin.collections.MapsKt

Can you help us with the issue or guide us towrads a collegue that can provide more information?

We are using Kotlin 1.5, AWS SDK 2, Spring Boot, Corretto 11 as JRE, Gradle. Moreover, it seems like the DynamoDB local jar contains indeed a copy of the Kotlin standard library classes/files. Is it possible to remove those from the bundle?

[zoewangg]

Hi @spanierm I've reached out to the team that maintains DynamoDB Local and will follow up if there's any update. Could you create a new issue to track this since this issue is to track the 2.x support?

[zoewangg]

Closing this issue. Feel free to open new issues if you have further questions.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.