Elasticsearch-net: Why SerializableData doesn't use SourceSerializer ?

[jAwAno]

Hi ! I'm trying to upgrade NEST from 5.x to 6.x . and I know from NEST 6.x , we have to explicitly define JSON serializer if we want to use some of others of NEST defaults.

but even if we define and set SourceSerializer explicitly , some of places use only RequestResponseSerializer.
for example , when we using LowLevel API , we post json as SerializableData like this

 return await ElasticClient.LowLevel.IndexPutAsync<IndexResponse>(
                index, DefaultType, id, PostData.Serializable(document),
               new IndexRequestParameters {OpType = OpType.Create , Routing = document.Parent} , cancellationToken);

and SerializableData seems only using RequestResponseSerializer for serialize, it causes some problem on my code.

I already found workaround . that is just stop to use LowLevel API or serialize object as string and use PostData.String(), but I would like to know...

• any other workaround or primary way to use SourceSerializer when calling LowLevel API
• is this behavior by design ? or bug ?
• and some of other LowLevel API that for searching (e.g. LowLevel.GetAsync) are seems use SourceSerializer , why such an asymmetric behavior between update and search

I look forward to hearing from you. thank you .

---

• NEST/Elasticsearch.Net 6.4.0
• NEST.JsonNetSerializer 6.4.0
• newtonsoft.json 11.0.2

[russcam]

This is not a bug, but it's not so obvious why it happens, so I'll explain. In summary, if your request is changed to

```c#
return await ElasticClient.LowLevel.IndexPutAsync(
index, DefaultType, id, PostData.Serializable(new IndexRequest(document)),
new IndexRequestParameters {OpType = OpType.Create , Routing = document.Parent} , cancellationToken);

where `TDocument` is the type of `document`, it will work as expected.

#### Explanation

The low level client, `IElasticLowLevelClient`, only knows about a single `IElasticsearchSerializer`. It has its own `IElasticsearchSerializer` when used in isolation e.g.

```c#
var lowLevelClient = new ElasticLowLevelClient();

will use LowLevelRequestResponseSerializer:

https://github.com/elastic/elasticsearch-net/blob/6ee869fcc14ddd51388c47c9c558cd5efcefc21e/src/Elasticsearch.Net/Serialization/LowLevelRequestResponseSerializer.cs#L10

This serializer will be used to serialize and deserialize everything for the client.

The low level client used by the high level client and exposed by the .LowLevel property on IElasticClient still only knows about a single IElasticsearchSerializer, but when instantiated inside of the high level client, the serializer used by the low level client is the DefaultHighLevelSerializer:

https://github.com/elastic/elasticsearch-net/blob/6ee869fcc14ddd51388c47c9c558cd5efcefc21e/src/Nest/CommonAbstractions/SerializationBehavior/DefaultHighLevelSerializer.cs#L11

which is exposed as the RequestResponseSerializer on the high level client (as well as the SourceSerializer by default - more on this in a bit). This is the IElasticsearchSerializer that the high level client (by delegating to the low level client) will use to serialize and deserialize everything.

Internally, when the high level client calls into the low level client's DoRequest/DoRequestAsync methods, the type passed to serialize is one that implements IRequest

https://github.com/elastic/elasticsearch-net/blob/fdb193323f969cd5c5e74a3a2c6b9e9f148d3324/src/Nest/ElasticClient.cs#L121-L150

The RequestResponseSerializer knows how to serialize NEST types, and delegate to SourceSerializer when a type or property is one that should be serialized by an IElasticsearchSerializer that can be specified by the user, such as for the type that represents a document in the case of an index request. By default, the SourceSerializer is the same as RequestResponseSerializer, but users can configure their own IElasticsearchSerializer such as the JsonNetSerializer provided in the Nest.JsonNetSerializer nuget package.

Going back to serialization delegation, or "donut serialization" if you will, delegation is typically controlled through attributes on the request types that indicate a specific JSON formatter to use when serializing a type, or property of a type. Tying all the pieces together, this is why the low level client call does not serialize as you expect; the call will use the RequestResponseSerializer and, because there are no attributes on your document to indicate that the SourceSerializer should be used, the document is serialized with the RequestResponseSerializer. It's not possible to attribute your document to indicate that it should be serialized with the SourceSerializer, because these attributes are internal. This is because the concept of a RequestResponseSerializer and SourceSerializer are high level client constructs, part of the IElasticsearchSerializer used by the high level client. The low level client has no knowledge of these constructs, and will serialize SerializableData<T> with whatever IElasticsearchSerializer it's configured with.

I hope this explanation helps, and happy to go into more detail.