Azure-sdk-for-java: [BUG] Different serialization from findBy vs. @Query

[JV-TMCZ]

Describe the bug
Query result could not be serialized to POJO when query is defined by @Query annotation.
While the very same data during the very same method call are successfully serialized by findBy* methods.

Problem could be somehow related to orderBy or java.time.Instant (?).
I am unable to anyhow processed query which is working from data explorer.
SELECT top 1 * FROM c where c.roomId = '7abc533e-3e4b-47d6-84e0-XXXXXXXXXXXX' order by c._ts desc
result:
[
{
"id": "1bb6c8c4-445d-4827-b5d3-XXXXXXXXXXXX",
"roomId": "7abc533e-3e4b-47d6-84e0-XXXXXXXXXXXX",
"occupied": true,
"timestamp": 1610025258.019,
"_rid": "XXX",
"_self": "XXX",
"_etag": "XXX",
"_attachments": "attachments/",
"_ts": 1610025259
}
]

Exception or Stack Trace
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.azure.spring.data.cosmos.exception.CosmosAccessException: Failed to find items; nested exception is java.lang.IllegalStateException: Failed to get POJO.] with root cause

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of java.time.Instant (no Creators, like default constructor, exist): no int/Int-argument constructor/factory method to deserialize from Number value (1610024969)
at [Source: UNKNOWN; line: -1, column: -1] (through reference chain: cz.tmobile.roof.db.entity.Occupancy["_ts"])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1615) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1077) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromInt(ValueInstantiator.java:262) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromInt(StdValueInstantiator.java:356) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromNumber(BeanDeserializerBase.java:1359) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:178) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:166) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:257) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.module.afterburner.deser.SuperSonicBeanDeserializer.deserialize(SuperSonicBeanDeserializer.java:155) ~[jackson-module-afterburner-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4495) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2728) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:3181) ~[jackson-databind-2.11.2.jar:2.11.2]
at com.azure.cosmos.implementation.JsonSerializable.toObject(JsonSerializable.java:651) ~[azure-cosmos-4.9.0.jar:na]
at com.azure.cosmos.models.ModelBridgeInternal.toObjectFromJsonSerializable(ModelBridgeInternal.java:504) ~[azure-cosmos-4.9.0.jar:na]
at com.azure.cosmos.CosmosAsyncContainer.lambda$prepareFeedResponse$12(CosmosAsyncContainer.java:494) ~[azure-cosmos-4.9.0.jar:na]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_231]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[na:1.8.0_231]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[na:1.8.0_231]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[na:1.8.0_231]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[na:1.8.0_231]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_231]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[na:1.8.0_231]
at com.azure.cosmos.CosmosAsyncContainer.prepareFeedResponse(CosmosAsyncContainer.java:496) ~[azure-cosmos-4.9.0.jar:na]
at com.azure.cosmos.CosmosAsyncContainer.lambda$queryItemsInternalFunc$9(CosmosAsyncContainer.java:471) ~[azure-cosmos-4.9.0.jar:na]
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:100) ~[reactor-core-3.3.10.RELEASE.jar:3.3.10.RELEASE]
...

To Reproduce
Any @Query defined query ...

Code Snippet

@Repository
public interface OccupancyRepository extends CosmosRepository<Occupancy, String> {

    // ERROR at runtime serialization
    @Query(value = "SELECT top 1 * FROM c where c.roomId = @roomId order by c._ts desc")
    List<Occupancy> findLatestByRoomId(@Param("roomId") String roomId);

    // ERROR at runtime serialization
    @Query(value = "SELECT * FROM c where c.roomId = @roomId")
    List<Occupancy> findX(String roomId);

    // ERROR at runtime serialization
    @Query(value = "SELECT * FROM c")
    List<Occupancy> findZ();

    // OK
    List<Occupancy> findAllByRoomId(String roomId);

    // OK
    List<Occupancy> findByRoomIdOrderByRoomIdDesc(String roomId);

    // Error at startup: No property orderBy found for type String! Traversed path: Occupancy.roomId.
    List<Occupancy> findByRoomIdOrderBy_tsDesc(String roomId);

    // OK
    List<Occupancy> findByRoomIdOrderByRoomId(String roomId);

    // OK
    Optional<Occupancy> findFirstByRoomId(String roomId);

    // Error at startup: No property orderBy found for type String! Traversed path: Occupancy.roomId.
    Optional<Occupancy> findFirstByRoomIdOrderBy_tsDesc(String roomId);
}

@Data
@Container
public class Occupancy {
    @Id
    private String id;
    private String roomId;
    private Boolean occupied;
    /** CreatedOn timestamp*/
    private Instant timestamp;
    /** CosmoDB system timestamp: Last update */
    private Instant _ts;
}

Setup (please complete the following information):

• OS: win10
• IDE : IntelliJ
• Version of the Library used
  
  org.springframework.boot
  spring-boot-starter-parent
  2.3.4.RELEASE
  
  
  
  
  org.springframework.cloud
  spring-cloud-function-adapter-azure

    </dependency>
    <!-- Provides HTTP servlet container for local run -->
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-function-web</artifactId>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-spring-data-cosmos</artifactId>
        <version>3.2.0</version>
    </dependency>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-function-dependencies</artifactId>
            <version>3.1.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Additional context
Any idea how to get latest item from container? Ideally by internal _ts or custom timestamp field?
Optional findLatestByFieldOrderBy_ts(String param);

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @kushagraThapar, @anfeldma-ms

[kushagraThapar]

@mbhaskar - please take a look at this issue!

[JV-TMCZ]

I just tried it in simple spring boot app (without spring cloud), based on sample from azure-sdk-for-javasdkspringazure-spring-boot-samples
I just add private Instant field to User entity and CosmosRepository with @Query ... same bug.

Simplify repro project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.7.RELEASE</version> <!-- {x-version-update;org.springframework.boot:spring-boot-starter-parent;external_dependency} -->
  </parent>

  <groupId>com.azure.spring</groupId>
  <artifactId>azure-spring-boot-sample-cosmos</artifactId>
  <version>1.0.0</version>
  <packaging>jar</packaging>

  <name>Azure Spring Boot Starter Sample - Cosmos DB</name>
  <description>Sample project for Azure Document DB Starter</description>
  <url>https://github.com/Azure/azure-sdk-for-java</url>

  <dependencies>
    <dependency>
      <groupId>com.azure.spring</groupId>
      <artifactId>azure-spring-boot-starter-cosmos</artifactId>
      <version>3.0.0</version> <!-- {x-version-update;com.azure.spring:azure-spring-boot-starter-cosmos;current} -->
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
  </dependencies>

</project>

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
package com.azure.spring.sample.cosmos;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.util.Assert;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.time.Instant;
import java.util.List;
import java.util.Optional;

@SpringBootApplication
public class CosmosSampleApplication2 implements CommandLineRunner {

    private static final Logger LOGGER = LoggerFactory.getLogger(CosmosSampleApplication2.class);

    @Autowired
    private UserRepository2 repository;

    public static void main(String[] args) {
        SpringApplication.run(CosmosSampleApplication2.class, args);
    }

    public void run(String... var1) {
        User testUser = new User("testId", "testFirstName", "testLastName", "test address line one", null, Instant.now());

        // Save the User class to Azure Cosmos DB database.
        User saved = repository.save(testUser);
        LOGGER.info("saved: " + saved);

        // OK
        User findByName = repository.findByFirstName("testFirstName");
        LOGGER.info("findByName: " + findByName);

        // OK
        Optional<User> findById = repository.findById(testUser.getId());
        LOGGER.info("findById: " + findById);

        // ERROR Failed to get POJO.
//        List<User> findByFirstNameQuery = repository.findByFirstNameQuery(testUser.getFirstName());
//        LOGGER.info("findByFirstNameQuery: " + findByFirstNameQuery);

        // ERROR Failed to get POJO.
        User findOneByFirstNameQuery = repository.findOneByFirstNameQuery(testUser.getFirstName());
        LOGGER.info("findOneByFirstNameQuery: " + findOneByFirstNameQuery);
    }
}



md5-c356449318fc3cca92863c031162ae3d



// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

package com.azure.spring.sample.cosmos;

import com.azure.spring.data.cosmos.core.mapping.Container;
import com.azure.spring.data.cosmos.core.mapping.PartitionKey;
import org.springframework.data.annotation.Id;

import java.time.Instant;

@Container(containerName = "mycollection")
public class User {
    @Id
    private String id;
    private String firstName;
    @PartitionKey
    private String lastName;
    private String address;
    private Instant timestamp;
    private Instant _ts;

    public User() {

    }

    public User(String id, String firstName, String lastName, String address) {
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.address = address;
    }

    public User(String id, String firstName, String lastName, String address, Instant _ts, Instant timestamp) {
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.address = address;
        this._ts = _ts;
        this.timestamp = timestamp;
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }

    public String getAddress() {
        return address;
    }

    public void setAddress(String address) {
        this.address = address;
    }

    public Instant getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(Instant timestamp) {
        this.timestamp = timestamp;
    }

    public Instant get_ts() {
        return _ts;
    }

    public void set_ts(Instant _ts) {
        this._ts = _ts;
    }

    @Override
    public String toString() {
        return "User{" +
            "id='" + id + '\'' +
            ", firstName='" + firstName + '\'' +
            ", lastName='" + lastName + '\'' +
            ", address='" + address + '\'' +
            ", timestamp=" + timestamp +
            ", _ts=" + _ts +
            '}';
    }
}



md5-c356449318fc3cca92863c031162ae3d



// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

package com.azure.spring.sample.cosmos;

import com.azure.spring.data.cosmos.repository.CosmosRepository;
import com.azure.spring.data.cosmos.repository.Query;
import com.azure.spring.data.cosmos.repository.ReactiveCosmosRepository;
import org.springframework.stereotype.Repository;
import reactor.core.publisher.Flux;

import java.util.List;
import java.util.Optional;

@Repository
public interface UserRepository2 extends CosmosRepository<User, String> {

    User findByFirstName(String firstName);

    @Query(value = "SELECT * FROM c where c.firstName = @firstName")
    List<User> findByFirstNameQuery(String firstName);

    @Query(value = "SELECT * FROM c where c.firstName = @firstName")
    User findOneByFirstNameQuery(String firstName);

}

[JV-TMCZ]

BTW: I am very confused from so many similar maven dependencies... Could you please point me to explanation? Thanks. (these are concurent libraries? or one team (MS/spring) pass it to other? or is it the same but repackaged / renamed? Very confusing. I found several "official" samples and each using somehow different dependencies )-:

(directly or through spring dependencies)
azure-sdk-for-javasdkspringazure-spring-boot-samples
hello-spring-function-azure

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-spring-data-cosmos</artifactId>
            <version>3.2.0</version>
        </dependency>

(Baeldung)

        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>spring-data-cosmosdb</artifactId>
            <version>${cosmodb.version}</version>
        </dependency>       

com.azure:azure-cosmos-java-sql-api-samples:1.0.0.RELEASE

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-cosmos</artifactId>
            <version>4.8.0</version>
        </dependency>

[kushagraThapar]

@JV-TMCZ - to provide some details on your last question.
There are 3 different SDKs that are supported by Cosmos DB Java team.

1. azure-cosmos SDK - which is the native Java SDK that can be used to access Cosmos DB: https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-java-v4 Maven link - https://mvnrepository.com/artifact/com.azure/azure-cosmos
2. azure-spring-data-cosmos SDK - which is Spring Data Cosmos SDK - uses spring-data framework - https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-sdk-java-spring-v3?tabs=explore This SDK is based on azure-cosmos SDK and leverages it to talk to Cosmos DB. Maven link - https://mvnrepository.com/artifact/com.azure/azure-spring-data-cosmos
3. azure-spring-boot-starter-cosmos SDK - which is Spring Boot Cosmos DB Starter - uses spring boot framework , https://docs.microsoft.com/en-us/java/api/overview/azure/spring-boot-starter-cosmos-readme?view=azure-java-stable This SDK leverages azure-spring-data-cosmos SDK to use spring data framework to talk to Cosmos DB. Maven link - https://mvnrepository.com/artifact/com.azure.spring/azure-spring-boot-starter-cosmos

Rest of the SDKs that you see are just older versions of these, which we do not recommend to be used anymore like these:

<dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>spring-data-cosmosdb</artifactId>
            <version>${cosmodb.version}</version>
        </dependency>

and this:

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-cosmosdb-spring-boot-starter</artifactId>
    <version>2.3.3</version>
</dependency>

[mbhaskar]

@JV-TMCZ Thanks for reporting the issue. Annotated queries are currently using the Objectmapper from the underlying cosmos sdk which is not configured to handle Jdk8 time modules. We will fix this in an upcoming release. As a mitigation for the timebeing, you can use JsonNode as the deserializing object instead of User and deserialize the object to type in your code later. Something like

    @Query(value = "SELECT * FROM c where c.firstName = @firstName")
    List<JsonNode> findByFirstNameQuery(String firstName);

Will update the issue once fix is available.

[kushagraThapar]

Fix has been merged in and will be released this week.