Elasticsuite: Duplicate elasticsearch request calls in default magento setup

[susonwaiba]

It seems like ES calls are increased and upon closely looking at logs, It seems like request is executed twice

Preconditions

• NONE, issue persist in default magento 2.3.5-p1 EE with sample data

Magento Version : "2.3.5-p1" EE - need to test in CE as well

ElasticSuite Version : "~2.9.0"

Environment : Default

Third party modules : NONE

Steps to reproduce

1. Install magento EE 2.3.5-p1 and install sample data
2. Install "smile/elasticsuite": "~2.9.0"
3. Setup Elasticsearch server 6.*

$ curl elasticsearch:9200
{
  "name" : "NUI1U2o",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "K760RtJeQMC_SVhHcIcWGA",
  "version" : {
    "number" : "6.6.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "3bd3e59",
    "build_date" : "2019-03-06T15:16:26.864148Z",
    "build_snapshot" : false,
    "lucene_version" : "7.6.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

1. Enable flat from catalog
2. Enable debug from Elasticsearch config
3. tail -f var/log/system.log and visit product list page

Expected result

• 2 calls to elasticsearch, one for attribute and other to fetch products for list page

Actual result

• 4 calls to elasticsearch with same request twice for attribute and product
  
• Log is attached below showing 4 elasticsearch calls:-

[2020-05-11 05:34:10] report.INFO: Request Success: {"method":"HEAD","uri":"http://elasticsearch/magento2_default_catalog_product","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.011091} []
[2020-05-11 05:34:10] report.INFO: curl -XHEAD 'http://elasticsearch/magento2_default_catalog_product?pretty=true' [] []
[2020-05-11 05:34:10] report.INFO: Request Success: {"method":"GET","uri":"http://elasticsearch/","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.001076} []
[2020-05-11 05:34:10] report.INFO: curl -XGET 'http://elasticsearch/?pretty=true' [] []
[2020-05-11 05:34:10] report.INFO: Request Success: {"method":"POST","uri":"http://elasticsearch/magento2_default_catalog_product/_search","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.004051} []
[2020-05-11 05:34:10] report.INFO: curl -XPOST 'http://elasticsearch/magento2_default_catalog_product/_search?pretty=true' -d '{"size":0,"query":{"constant_score":{"filter":{"bool":{"must":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"bool":{"must_not":[{"term":{"category.is_virtual":{"value":true,"boost":1}}}],"boost":1}},{"terms":{"category.category_id":["23"],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},{"term":{"stock.is_in_stock":{"value":true,"boost":1}}},{"terms":{"visibility":[2,4],"boost":1}},{"bool":{"must_not":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"term":{"category.category_id":{"value":23,"boost":1}}},{"term":{"category.is_blacklisted":{"value":true,"boost":1}}}],"must_not":[],"should":[],"boost":1}},"boost":1}}],"boost":1}},{"terms":{"visibility":[2,4],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},"aggregations":{"attribute_set_id":{"terms":{"field":"attribute_set_id","size":100000,"order":{"_count":"desc"}}},"indexed_attributes":{"terms":{"field":"indexed_attributes","size":100000,"order":{"_count":"desc"}}}}}' [] []
[2020-05-11 05:34:10] report.INFO: Request Success: {"method":"POST","uri":"http://elasticsearch/magento2_default_catalog_product/_search","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.012429} []
[2020-05-11 05:34:10] report.INFO: curl -XPOST 'http://elasticsearch/magento2_default_catalog_product/_search?pretty=true' -d '{"size":12,"sort":[{"category.position":{"order":"asc","missing":"_last","unmapped_type":"keyword","nested":{"path":"category","filter":{"terms":{"category.category_id":["23"],"boost":1}}},"mode":"min"}},{"_score":{"order":"desc"}},{"entity_id":{"order":"desc","missing":"_first","unmapped_type":"keyword"}}],"from":0,"query":{"constant_score":{"filter":{"bool":{"must":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"bool":{"must_not":[{"term":{"category.is_virtual":{"value":true,"boost":1}}}],"boost":1}},{"terms":{"category.category_id":["23"],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},{"term":{"stock.is_in_stock":{"value":true,"boost":1}}},{"terms":{"visibility":[2,4],"boost":1}},{"bool":{"must_not":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"term":{"category.category_id":{"value":23,"boost":1}}},{"term":{"category.is_blacklisted":{"value":true,"boost":1}}}],"must_not":[],"should":[],"boost":1}},"boost":1}}],"boost":1}},{"terms":{"visibility":[2,4],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},"aggregations":{"attribute_set_id":{"terms":{"field":"attribute_set_id","size":100000,"order":{"_count":"desc"}}},"price.price":{"nested":{"path":"price"},"aggregations":{"price.price":{"filter":{"terms":{"price.customer_group_id":[0],"boost":1}},"aggregations":{"price.price":{"histogram":{"field":"price.price","interval":1,"min_doc_count":1}}}}}},"option_text_color":{"terms":{"field":"option_text_color","size":100000,"order":{"_count":"desc"}}},"option_text_material":{"terms":{"field":"option_text_material","size":10,"order":{"_count":"desc"}}},"option_text_size":{"terms":{"field":"option_text_size","size":100000,"order":{"_count":"desc"}}},"option_text_style_general":{"terms":{"field":"option_text_style_general","size":10,"order":{"_count":"desc"}}},"option_text_pattern":{"terms":{"field":"option_text_pattern","size":10,"order":{"_count":"desc"}}},"option_text_climate":{"terms":{"field":"option_text_climate","size":10,"order":{"_count":"desc"}}}}}' [] []
[2020-05-11 05:34:11] report.INFO: Request Success: {"method":"POST","uri":"http://elasticsearch/magento2_default_catalog_product/_search","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.0033} []
[2020-05-11 05:34:11] report.INFO: curl -XPOST 'http://elasticsearch/magento2_default_catalog_product/_search?pretty=true' -d '{"size":0,"query":{"constant_score":{"filter":{"bool":{"must":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"bool":{"must_not":[{"term":{"category.is_virtual":{"value":true,"boost":1}}}],"boost":1}},{"terms":{"category.category_id":["23"],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},{"term":{"stock.is_in_stock":{"value":true,"boost":1}}},{"terms":{"visibility":[2,4],"boost":1}},{"bool":{"must_not":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"term":{"category.category_id":{"value":23,"boost":1}}},{"term":{"category.is_blacklisted":{"value":true,"boost":1}}}],"must_not":[],"should":[],"boost":1}},"boost":1}}],"boost":1}},{"terms":{"visibility":[2,4],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},"aggregations":{"attribute_set_id":{"terms":{"field":"attribute_set_id","size":100000,"order":{"_count":"desc"}}},"indexed_attributes":{"terms":{"field":"indexed_attributes","size":100000,"order":{"_count":"desc"}}}}}' [] []
[2020-05-11 05:34:11] report.INFO: Request Success: {"method":"POST","uri":"http://elasticsearch/magento2_default_catalog_product/_search","port":9200,"headers":{"Host":["elasticsearch"],"Content-Type":["application/json"],"Accept":["application/json"],"User-Agent":["elasticsearch-php/7.6.1 (Linux 5.4.38-1-MANJARO; PHP 7.3.16)"]},"HTTP code":200,"duration":0.009832} []
[2020-05-11 05:34:11] report.INFO: curl -XPOST 'http://elasticsearch/magento2_default_catalog_product/_search?pretty=true' -d '{"size":12,"sort":[{"category.position":{"order":"asc","missing":"_last","unmapped_type":"keyword","nested":{"path":"category","filter":{"terms":{"category.category_id":["23"],"boost":1}}},"mode":"min"}},{"_score":{"order":"desc"}},{"entity_id":{"order":"desc","missing":"_first","unmapped_type":"keyword"}}],"from":0,"query":{"constant_score":{"filter":{"bool":{"must":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"bool":{"must_not":[{"term":{"category.is_virtual":{"value":true,"boost":1}}}],"boost":1}},{"terms":{"category.category_id":["23"],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},{"term":{"stock.is_in_stock":{"value":true,"boost":1}}},{"terms":{"visibility":[2,4],"boost":1}},{"bool":{"must_not":[{"nested":{"path":"category","score_mode":"none","query":{"bool":{"must":[{"term":{"category.category_id":{"value":23,"boost":1}}},{"term":{"category.is_blacklisted":{"value":true,"boost":1}}}],"must_not":[],"should":[],"boost":1}},"boost":1}}],"boost":1}},{"terms":{"visibility":[2,4],"boost":1}}],"must_not":[],"should":[],"boost":1}},"boost":1}},"aggregations":{"attribute_set_id":{"terms":{"field":"attribute_set_id","size":100000,"order":{"_count":"desc"}}},"price.price":{"nested":{"path":"price"},"aggregations":{"price.price":{"filter":{"terms":{"price.customer_group_id":[0],"boost":1}},"aggregations":{"price.price":{"histogram":{"field":"price.price","interval":1,"min_doc_count":1}}}}}},"option_text_color":{"terms":{"field":"option_text_color","size":100000,"order":{"_count":"desc"}}},"option_text_material":{"terms":{"field":"option_text_material","size":10,"order":{"_count":"desc"}}},"option_text_size":{"terms":{"field":"option_text_size","size":100000,"order":{"_count":"desc"}}},"option_text_style_general":{"terms":{"field":"option_text_style_general","size":10,"order":{"_count":"desc"}}},"option_text_pattern":{"terms":{"field":"option_text_pattern","size":10,"order":{"_count":"desc"}}},"option_text_climate":{"terms":{"field":"option_text_climate","size":10,"order":{"_count":"desc"}}}}}' [] []

[romainruaud]

Confirmed on my dev environment.

Will dig more on this one, thank you @susonwaiba for reporting it.

Regards

[romainruaud]

It's due to this : https://github.com/Smile-SA/elasticsuite/pull/1543

Removing the $this->_isFiltersRendered = false; prevents the ES query to be sent twice.

[susonwaiba]

Thank you @romainruaud. Working like a charm. :+1:
Removed $this->_isFiltersRendered = false;