Azure-cosmos-dotnet-v2: BulkImportAsync does not respect JsonSerializerSettings

Created on 21 Sep 2018  路  7Comments  路  Source: Azure/azure-cosmos-dotnet-v2

Description:
When using BulkImportAsync, the JsonSerializerSettings passed into DocumentClient is not respected.

Expected:
Objects added to the database with BulkImportAsync should have the same TypeNameHandling as the JsonSerializerSettings in the DocumentClient.

Actual:
No types are ever written in the JSON of the document.

Simplified Repro:

interface IInterface {
    Property { get; set; }
}

class ImplementedInterface : IInterface {
    Property { get; set; }
}

class Document {
    IInterface DocumentProperty { get; set; }
}

var documentClient = new DocumentClient(serializerSettings: new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
var bulkExecutor = new BulkExecutor(documentClient);
await bulkExecutor.BulkImportAsync(new List<Document> { new Document { DocumentProperty = new ImplementedInterface() } });

The Document JSON will not have any types.

My Current Workaround:

await bulkExecutor.BulkImportAsync(documents.Select(document => JsonConvert.SerializeObject(document, BulkImportJsonSerializerSettings)));

Serializing every object before passing it to BulkImportAsync solves the issue of typing, but adds an increase in memory, which builds up when trying to import 100,000s of documents.

All 7 comments

@cgelon Which version of the BulkExecutor library are you using?

I've used both 1.0.2 and 1.1.0, and both of them had this issue.
I see that the package got updated today to 1.1.1; I will try that and report back.

@cgelon Please let me know if it repros with 1.1.1 so we can investigate.

It still repros with 1.1.1.

It still repros with 2.1.0-preview as well.

Still an issue in 2.3.0-preview2.

Here is my workaround:

public static async Task<BulkImportResponse> BulkImportAsync(this IBulkExecutor bulkExecutor, JsonSerializerSettings serializerSettings, IEnumerable<object> documents, bool enableUpsert, CancellationToken cancellationToken )
{
    var serializer = JsonSerializer.Create(serializerSettings);
    var serializedDocuments = documents.Select(x => JObject.FromObject(x, serializer));
    var importResponse = await bulkExecutor.BulkImportAsync(serializedDocuments, enableUpsert, cancellationToken: cancellationToken);
    return importResponse;
}

It looks like internally the libary checks first to see if the objects passed in are of type JObject and if so will save them without trying to do serialization.

This appears to be fixed on the latest version stream.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

christian-vorhemus picture christian-vorhemus  路  5Comments

ermontgo picture ermontgo  路  5Comments

Frankwayne picture Frankwayne  路  3Comments

kspearrin picture kspearrin  路  4Comments

Lihan-Chen picture Lihan-Chen  路  6Comments