Azure-cosmos-dotnet-v3: Objects not being deserialized from Query
Describe the bug
When returning a collection from a query, the items in the collection are default/unpopulated objects.
To Reproduce
```c#
public async Task
{
QueryDefinition qd = await PersonSearchQueryDefinitionParser.BuildSearchQuery(search);
List<Person> ret = new List<Person>();
FeedIterator<Person> feed = Container.GetItemQueryIterator<Person>(qd);
while (feed.HasMoreResults)
{
var curr = await feed.ReadNextAsync();
ret.AddRange(curr);
}
return ret;
}
_ret_ has the expected number of objects in its collection, but they are default. The custom CosmosSerializer is called only once for _PartitionedQueryExecutionInfo_, but not for either of the returned objects.
Serializer:
```c#
public class StorageSerializer : CosmosSerializer
{
private static readonly JsonSerializer Serializer = JsonSerializer.Create(new JsonSerializerSettings {
Converters = new List<JsonConverter>() { new ContactConverter() },
ContractResolver = new StorageContractResolver()
});
private static readonly Encoding DefaultEncoding = new UTF8Encoding(false, true);
public override T FromStream<T>(Stream stream)
{
using (stream)
{
if (typeof(Stream).IsAssignableFrom(typeof(T)))
{
return (T)(object)(stream);
}
using (StreamReader sr = new StreamReader(stream))
using (JsonTextReader r = new JsonTextReader(sr))
{
return Serializer.Deserialize<T>(r);
}
}
}
public override Stream ToStream<T>(T input)
{
MemoryStream ret = new MemoryStream();
using (StreamWriter sw = new StreamWriter(ret, encoding: DefaultEncoding, bufferSize: 1024, leaveOpen: true))
using (JsonTextWriter jw = new JsonTextWriter(sw))
{
jw.Formatting = Formatting.Indented;
Serializer.Serialize(jw, input);
jw.Flush();
sw.Flush();
}
ret.Position = 0;
return ret;
}
}
Expected behavior
_ret_ should be populated with two deserialized objects.
Actual behavior
It appears that deserialization doesn't happen. Because the Serializer being used is custom and based on Json.NET, the objects aren't decorated with non-Json.NET attributes.
When debugging, _curr_.CosmosElements is populated with two objects. The custom CosmosSerializer is not called for the two elements.
Environment summary
SDK Version: Microsoft.Azure.Cosmos v3.0.0
OS Version (e.g. Windows, Linux, MacOSX): Windows 10
davidxcrowe
All 13 comments
Query uses a lazy deserialization. It only deserializes once the values are read. @davidxcrowe can you share the types for the repo?
j82w
on 23 Jul 2019
@j82w
```c#
[JsonObject(MemberSerialization.OptIn, ItemNullValueHandling = NullValueHandling.Ignore)]
[CADSObject("Person", "/api/persons")]
public class Person : CADSObject
{
public Person() : base("Person")
{ }
[JsonProperty("personNumber")]
public virtual int PersonNumber { get; set; }
[Required]
[JsonProperty("familyName")]
public string FamilyName { get; set; }
[Required]
[JsonProperty("givenName")]
public string GivenName { get; set; }
[Required]
[JsonProperty("title")]
public string Title { get; set; }
[JsonProperty("initials")]
public string Initials { get; set; }
[Required]
[JsonProperty("birthDate")]
public DateTime DateOfBirth { get; set; }
[JsonProperty("contacts")]
public IList<Contact> Contacts { get; set; }
[JsonProperty("relatedPersons")]
public IList<Association> RelatedPersons { get; set; }
[JsonProperty("associations")]
public IList<Association> Associations { get; set; }
}
```
davidxcrowe
on 23 Jul 2019
And the parent object is:
```c#
[JsonObject(MemberSerialization.OptIn)]
public abstract class CADSObject
{
protected CADSObject(string type)
{
Type = type;
}
[JsonProperty("@context", Order = 1)]
public IEnumerable<string> Context { get; set; }
[Required]
[JsonProperty("type", Order = 2)]
public virtual string Type { get; private set; }
[JsonProperty("id", Order = 3)]
public Guid ID { get; set; }
[JsonProperty("uri", Order = 4, NullValueHandling = NullValueHandling.Ignore)]
[CADSUri("Type", "ID")]
public string Uri { get; set; } = "uri";
[JsonProperty("createdAt", Order = 5)]
public virtual DateTime CreatedAt { get; set; }
[JsonProperty("lastModifiedAt", Order = 6)]
public virtual DateTime LastModifiedAt { get; set; }
[JsonProperty("createdBy", Order = 7)]
public virtual Guid CreatedBy { get; set; }
[JsonProperty("lastModifiedBy", Order = 8)]
public virtual Guid LastModifiedBy { get; set; }
[JsonProperty("services", Order = 9)]
public IEnumerable<CADSService> Services { get; set; }
}
```
Services and URI aren’t in the database (they’re dynamically added by the API).
davidxcrowe
on 23 Jul 2019
@davidxcrowe does ReadItemAsync work correctly for you? I'm trying to figure out if this is query specific.
j82w
on 23 Jul 2019
Yes, ReadItemAsync works fine.
The query is:
SELECT DISTINCT p FROM p WHERE CONTAINS(p.familyName, @familyName)
_@familyName_ is being passed into the parameters (basically, there's a tokeniser that takes a straight input and makes assumptions about the content and then builds an SQL query out of it).
The output from the failed version is:
[
{
type:"Person",
id:"00000000-0000-0000-0000-000000000000",
services:[
{
operation:"List",
verb:"Get",
type:"Note",
serviceUri:"https://localhost:44366/api/notes/{Type}/{ID}"
}
],
birthDate:"0001-01-01T00:00:00",
lastModifiedBy:"00000000-0000-0000-0000-000000000000",
personNumber:0,
createdBy:"00000000-0000-0000-0000-000000000000",
createdAt:"0001-01-01T00:00:00",
uri:"https://localhost:44366/api/persons/00000000-0000-0000-0000-000000000000",
lastModifiedAt:"0001-01-01T00:00:00"
},
{
type:"Person",
id:"00000000-0000-0000-0000-000000000000",
services:-[
{
operation:"List",
verb:"Get",
type:"Note",
serviceUri:"https://localhost:44366/api/notes/{Type}/{ID}"
}
],
birthDate:"0001-01-01T00:00:00",
lastModifiedBy:"00000000-0000-0000-0000-000000000000",
personNumber:0,
createdBy:"00000000-0000-0000-0000-000000000000",
createdAt:"0001-01-01T00:00:00",
uri:"https://localhost:44366/api/persons/00000000-0000-0000-0000-000000000000",
lastModifiedAt:"0001-01-01T00:00:00"
}
]
Output from a straight read is:
{
id:"a5137b0d-ea03-4255-93b6-48dc46d86e53",
associations:-[
{
id:"fc10ba6a-e340-48d1-b681-340bc862b0fe",
relationship:"owner",
type:"Application",
uri:"https://localhost:44366/api/application//fc10ba6a-e340-48d1-b681-340bc862b0fe"
},
{
id:"e2eddac0-2a1a-40ae-b42b-583c5044cd7e",
relationship:"owner",
type:"Enrolment",
uri:"https://localhost:44366/api/enrolment//e2eddac0-2a1a-40ae-b42b-583c5044cd7e"
}
],
relatedPersons:[
{
id:"41fb7c34-10b4-4e76-862f-4a5624b7a852",
relationship:"Partner",
type:"Person",
uri:"https://localhost:44366/api/persons/41fb7c34-10b4-4e76-862f-4a5624b7a852"
}
],
initials:"J",
birthDate:"1985-01-01T00:00:00",
uri:"https://localhost:44366/api/persons/a5137b0d-ea03-4255-93b6-48dc46d86e53",
lastModifiedAt:"0001-01-01T00:00:00",
givenName:"Jane",
title:"Miss",
type:"Person",
createdBy:"00000000-0000-0000-0000-000000000000",
contacts:-[
{
id:"0c6ccacf-a593-429c-b026-9fe7d15df8dc",
state:"Active",
type:"PostalAddress",
name:"Correspondence"
},
{
addressCountry:"UK",
streetAddress1:"123 Here Street",
addressLocality:"Thereton",
id:"01a6662d-fc1b-4398-92ce-5f7ab7e8a1f2",
postalCode:"SE10 8RD",
addressRegion:"Everywhereshire",
type:"PostalAddress",
name:"Permanent",
state:"Active"
},
{
emailAddress:"[email protected]",
id:"0c661b46-7f42-4b37-869e-babf50bce093",
state:"Active",
type:"Email",
name:"Personal"
},
{
emailAddress:"test@domain",
id:"c8383a82-ba24-4ce9-b821-9b675dd821df",
state:"Active",
type:"Email",
name:"Insititutional"
},
{
id:"b53ce308-917a-4b31-b9d9-a508bc604d0c",
state:"Active",
type:"Telephone",
telephoneNumber:"07777 987654",
name:"Mobile"
},
{
id:"1dcba4df-a543-441f-b5b5-2925c9f3be6b",
state:"Active",
type:"Telephone",
telephoneNumber:"01234 567890",
name:"Home"
},
{
addressCountry:"UK",
streetAddress1:"999 The Street",
addressLocality:"The Town",
id:"2f56e94a-7643-4f56-98df-17bb334fb81a",
postalCode:"AB12 3CD",
addressRegion:"The County",
type:"PostalAddress",
name:"Correspondence",
state:"Active"
}
],
personNumber:1,
services:[
{
operation:"List",
verb:"Get",
type:"Note",
serviceUri:"https://localhost:44366/api/notes/{Type}/{ID}"
}
],
createdAt:"0001-01-01T00:00:00",
lastModifiedBy:"00000000-0000-0000-0000-000000000000",
familyName:"Example"
}
@davidxcrowe it might be possible you are hitting this issue
j82w
on 23 Jul 2019
It's possible, but the problem isn't with the Cosmos query (because I can see the response internally on the FeedResponse which has the expected content), but on deserializing the FeedResponse content.
davidxcrowe
on 25 Jul 2019
Does it work if you serialize and deserialize it just using the custom StorageSerializer?
Stream stream = StorageSerializer.ToStream(...)
var response = StorageSerializer.FromStream(stream)
j82w
on 25 Jul 2019
If I remove my mods (which add in customer Converter classes to handle abstract types), it doesn't work at all.
If I leave them in, I just get default objects for the List method.
davidxcrowe
on 30 Jul 2019
You can see there is a test here that verifies the serializer is getting called.
j82w
on 30 Jul 2019
I added a sample using the custom serializer. Everything appears to be working. Is there any way you can provide a sample that has a repo?
j82w
on 30 Jul 2019
I can probably come up with something tomorrow.
D
On Tue, 30 Jul 2019 at 19:43, j82w notifications@github.com wrote:
I added a sample
https://github.com/Azure/azure-cosmos-dotnet-v3/pull/611 using the
custom serializer. Everything appears to be working. Is there any way you
can provide a sample that has a repo?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/azure-cosmos-dotnet-v3/issues/583?email_source=notifications&email_token=ABE75EPBUFJ45JHYV5LIDATQCCDUHA5CNFSM4IGC7Z42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3E5OUY#issuecomment-516544339,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABE75EMS6IUXRUD7A6CBVADQCCDUHANCNFSM4IGC7Z4Q
.
davidxcrowe
on 30 Jul 2019
I'm closing this issue since the tests are passing. If you can provide a repo please reopen the issue.
j82w
on 15 Aug 2019
Related issues
j82w
·
4Comments
wahyuen
·
4Comments
Jubast
·
6Comments
ealsur
·
6Comments
nulltoken
·
3Comments