Elasticsuite: Elastic Suite slow comparing to commercial extensions

[tomekjordan]

Preconditions

Why Elastic Suite is so slow in autocomplete comparing to Elastic commercial extensions
from Amasty, Wyomind nad Mirasvit? which is blazing fast when you just try to search.
Even comparing to old mysql Instant Search from Itoris or Mageworx - its just slow.
Customers expect "algolia" style instant results, not results after 1-2seconds....

Magento Version :
2.3

ElasticSuite Version :
latest

Environment :
clean Magneto with sample data

Third party modules :
NO

Steps to reproduce

1. Just install Elastic Suite on vanilla Magento 2.3
2. Install sample date - or even yours
3. Try to search with autocomplete

Expected result

1.

Actual result

1. [Screenshot, logs]

[romainruaud]

Dear ElasticSuite user.

We are always pleased to hear constructive remarks about our product.

However, I'm not completely sure about the objectivity of this one. As you may have already noticed, we are providing this extension for free here.

Actually, ElasticSuite is embedded into more than 1200 Magento2 websites across the world, and we are very proud of it.

At Smile, we have it running on many websites, some with hundreds of thousands products, even some with millions of products, and performances remains very fast.

With a properly sized production platform, we benchmarked several times that autocompletion results are rendered <0.5s and search result page is rendered <2s.

On an architectural point of view, we did chose to embed Elasticsuite as a Magento module and a legacy Search Engine implementation (like Magento does for legacy Elasticsearch or Solr adapters), meaning that :

• we are fully respecting the core principles of Magento
• any customer can benefit our solution without breaking anything or having to subscribe to a third-party SaaS platform, or any additional third-party server to host an additional application
• any Magento2 developer will understand easily how things are done in ElasticSuite.

That being said, such an implementation comes with drawbacks, especially because we are following a legacy Search Engine implementation, we are due to suffer the way Magento is internally built. Eg. for the autocomplete, most of latency you are seeing is due to the time spent by the Magento call-stack to run when hitting the Frontend Controller, and so on... Being tied to a legacy implementation, there are these kind of time-consuming resources that are hard to suppress.

Anyway, I'll be quite happy to review a Pull Request from you to help us achieving better performances.

Best regards.

[gewaechshaus]

Hi @romainruaud,
Hi @tomekjordan,

we are facing this issue also, the autocomplete is kind of slow and unresponsive if you compare it with the mirasvit or wyomind one. Maybe it would be an option, to implement a backend configuration to choose a external payload which doesn't bootstrap the complete system like any conventional controller.

Anyway, I like ES - many thanks for providing such a great piece of code.

http://search-elastic-ultimate.m2.mirasvit.com/
http://elasticsearch.wyomind.com/

Cheers
Jan

[gewaechshaus]

@romainruaud, @tomekjordan

There must be a bottleneck somewhere in the controller.
I did some quick tests in one of our clients installations:

Note: We are using http2, so this if faster as every test in the demostore

Please type in the keyword "barf" at the search box located in the header. If you take a look at the console inspectors xhr request, you will see the request is finished after ~1500ms.
[HTTP/2.0 200 OK 1453ms]

If you open the related URI in another Tab, you get significantly better results like:
Cached result in 31ms:
https://www.haustierkost.de/search/ajax/suggest/?q=barf

[romainruaud]

For me the bottleneck is the price rendering done in PHP, and the fact that it's done via a Magento controller.

If we could :

• move this to a REST Api endpoint
• just return some JSON and let the frontend render them properly

It would be a significant improvement.

And of course, if we were able to completely bypass the Magento stack, it would also be waaaay faster of course.

[gewaechshaus]

@romainruaud - I just inspected the wyomind one, they are sending a POST request to a payloader located under http://elasticsearch.wyomind.com/elastic.php

Did somebody ever perform kind of a profiling to find the bottleneck?

[gewaechshaus]

And for sure, many thanks for your fast reaction! Thumbs up.

[gewaechshaus]

So in conclusion we should build an external loader bypassing m2 combined with frontend rendered json...

[romainruaud]

yes, but you cannot "bypass" M2 easily. At least, there is something you don't want to bypass, it's the ES query building made by Elasticsuite, because it will :

• add any synonyms if accurate
• append optimizers function scoring to the query
• do a pre-fetch termVectors call to ES to know if the query should be fuzzy or not

And this Query building is made.. in a low-level PHP code wrapped into the Magento SearchEngineInterface implementation, so ...

[gewaechshaus]

Hmmm... BrainFactory is working :)

[gewaechshaus]

We need to inspect one of the commercial extensions :)

[gewaechshaus]

@romainruaud

_And this Query building is made.. in a low-level PHP code wrapped into the Magento SearchEngineInterface implementation, so ..._

Are you refering this model and the request/response builders in the search method?

https://github.com/Smile-SA/elasticsuite/blob/afa1a40a792fbba570afb0ef982b3c684979b45c/src/module-elasticsuite-core/Model/Search.php#L24

[southerncomputer]

@romainruaud it might be wise to add caching of AJAX request and disregard "space" to allow faster response time due to people backspacing and forward spacing eating up json calls!
` var cache = {};
$(".ac-remote-caching").autocomplete({
minLength: 2,
source: function(request, response) {
var term = request.term;
if (term in cache) {
response(cache[term]);
return;
}

        $.getJSON("../../../app-assets/data/jquery-ui/search.php", request, function(data, status, xhr) {
            cache[term] = data;
            response(data);
        });
    }
});`

[gewaechshaus]

Hey @southerncomputer - that sounds like a good plan also. I much appreciate your commitment around M2 generaly, we already had some conversion in other issue related threads here on github.

[gulatipriyanka]

@romainruaud Smile Elastic Suite search suggestion taking too much time to populate results in dropdown and is very slow.
Please suggest how can we improve the speed or please provide if any patch is available for this.

Any help will be appreciated.

[simonrl]

Hey,

if anyone could share their findings on this one, it would be really nice. Maybe someone has already profiled autocomplete requests? This would save me some work.

I'd also happily take any hints on which part of the code could be optimized, and I will look into it.

Thanks!

[robsch]

The question of @tomekjordan is not necessarily criticism. I'm also wondering, what do the others do different, that they have such a fast response. I find this very crucial. Nowadays this is something that I would expect from a search engine. Waiting for a second after each key hit is annoying - at least for me, when I enter something into google or amazon or whereever, I expect a instant response. And I think I'm not allone...

This is no criticism. It doesn't make sense to criticize open source projects. It probably doesn't matter you, but personally I will tend to use one of the faster search modules, instead of ElasticSuite. What is a pity.

[davidwindell]

Hi @romainruaud would you consider re-opening this issue or creating a new one especially focused on the performance of the autocomplete to put efforts together on profiling and see if the community can come up with some understanding and suggestions for improving the response times?

[kpitn]

I love elasticsuite module, but i agree, defaut autocomplete way is a bit slow actually.

I try a way to speed up autocomplete in a new module available here : https://github.com/Web200/magento-elasticsuite-autocomplete

The goal is to avoid product loading, and use elasticsearch data to display product.
No modification is done on js or phtml.

More information in module readme file.

[davidwindell]

@kpitn this is awesome, thanks for sharing - will check your module out next week.

[paugnu]

Hi @kpitn, do you think you could propose a PR to this repo? I think you've solved one of the main «issues» with ElasticSuite.