Elasticsearch-net: Unit Testing Search Descriptiors

[markwalsh-liverpool]

NEST/Elasticsearch.Net version: 2.0+

Elasticsearch version: 2.0+

Description of the problem including expected versus actual behavior: Unable to unit test SearchDescriptors

Steps to reproduce:
Previously I used to be able to cast search descriptors into SearchRequests and do this:

var query = ((ISearchRequest)searchDescriptor);
Assert.AreEqual(query.Indices.First().Name, "test");

Assert.AreEqual(query.Size, 10);
Assert.AreEqual(query.From, 1);
Assert.AreEqual(query.Query.Filtered.Filter.And.Filters.First().Bool.Must.Skip(1).First().Term.Value, false);

But this now only offers some accessors (Size and From are still members). How can I test the index I am pointing at and also the query?

[russcam]

If you were looking to unit test in a similar way to how you have been, the nearest _verbatim_ way would be

``` c#
void Main()
{
var searchDescriptor = new SearchDescriptor()
.From(1)
.Size(10)
.Index("a,b,c")
.Query(q => q.MatchAll());

var connectionSettings = new ConnectionSettings();
var client = new ElasticClient(connectionSettings);

var query = ((ISearchRequest)searchDescriptor);

Assert.AreEqual("a,b,c", ((IUrlParameter)query.Index).GetString(client.ConnectionSettings));
Assert.AreEqual(10, query.Size);
Assert.AreEqual(1, query.From);

var expectedQuery = new
{
    match_all = new {}
};

Assert.IsTrue(JObject.DeepEquals(
    JObject.Parse(client.Serializer.SerializeToString(expectedQuery)), 
    JObject.Parse(client.Serializer.SerializeToString(query.Query)))
);

}

How `Indices` is serialized depends on `ConnectionSettings` as index names may be inferred from types, and this setup is on `ConnectionSettings` (in this example they aren't, but just mentioning anyway).

`query.Query` implements the visitor pattern so it is possible to pass an `IQueryVisitor` into `query.Query.Accept()` and make your assertions that way, or alternatively, as I have done here, simply assert that the serialized form of the query matches the serialized form of an anonymous type. The latter is probably more readable and easier to maintain than implementing a solution with a visitor and is what we do for our tests. 

It's a good idea to serialize to a string then load a `JObject` from the string and compare these with `JObject.DeepEquals()` as I've done in this example, in order to reduce brittleness arising from the order in which properties are serialized to json when serializing the _expected query_ and the _actual query_.

Taking the "_asserting that the serialized form matches our expectation_" further, we can just simply assert that the whole query matches an anonymous type

``` c#
var expectedQuery = new
{
    query = new
    {
        match_all = new { }
    },
    from = 1,
    size = 10
};

Assert.IsTrue(JObject.DeepEquals(
    JObject.Parse(client.Serializer.SerializeToString(expectedQuery)), 
    JObject.Parse(client.Serializer.SerializeToString(searchDescriptor)))
);

You may still want to check that you're hitting the right index(es) and type(s), but hopefully this gives you some ideas.

[markwalsh-liverpool]

Brilliant answer, thank you very much.

[i-am-ahmed]

Sorry but this solution needs to have access to ES server witch is not very elegant solution for unit testing

[Mpdreamz]

The solution proposed by @russcam does not do any http calls to an ES server. It only creates the client to access it's serializer.

[i-am-ahmed]

Sure, but will it work even with dummy server settings?
I'm saying that just because I've tried to do something like that and it actually needs valid network setting. Hope I'm wrong!

[russcam]

@i-am-ahmed it's the approach taken in a lot of the unit tests for the clients, which don't need or use a running instance of Elasticsearch; it's checking only the serialized form using the client's serialized and not sending any requests.

[i-am-ahmed]

@russcam you mean I can use it with dummy server url? kind of "http://0.0.0.0" ?

[FrankyBoy]

Honestly idk why this issue is closed. Unit testing search descriptors is a _huge_ pain in the ass if not impossible (at least so far I haven't even managed to simply read back the configured index, much less the actual configured queries).

This issue needs to be addressed properly.

[russcam]

@FrankyBoy it was closed over three years ago. Does the approach in https://github.com/elastic/elasticsearch-net/issues/1994#issuecomment-208681266 work for you?

[FrankyBoy]

@russcam not really. Why is this so horrible to test? Like for instance, why is there no way to iterate through query.Index or various other things?

PS: like ... basically ... Index is an enumeration of strings. So why is it not actually IEnumerable but a custom class that's some sort of union (which is internal) with no support to enumerate over anything so you have to use hacks like the one above?

[FrankyBoy]

So I just went through the code of Indices class ... and I don't see why the whole thing could not simply be replaced with implementing IEnumerable<IndexName>. As far as I can tell the Union is just there for the AllIndicesMarker (how is that even resolved to the actual "_all" index name string?), and that could very probably be replaced with an Instance of IndexName instead being it's own class. That in turn gets rid of the union, which then means you can get rid of MultiIndices, and all the rest falls back to an IEnumerable<IndexName>.

[FrankyBoy]

Played around with the code right now and came up with this: https://github.com/FrankyBoy/elasticsearch-net/commit/f9095b058f0c246a821f63c8fd46a0192f7d570a (hope that's accessible to you)

It's basically more or less for getting ideas what could be done in regards to making the Indices actually nicely testable, nothing I'd consider merge-able. Was just checking if I'd encounter any obvious road blocks along the way but it actually looks doable.