Elasticsuite: Exception: catalog_product index does not exist yet.

[diybook]

Hi Romain, thanks for introducing elasticsearch to me in the post https://github.com/magento/magento2/issues/7027. I think the elasticsuite is really greath and wanted to try it myself.

I´ve installed the ElasticSearch server as well as the magento module but getting the following errors on the frontend.

`1 exception(s):
Exception #0 (LogicException): catalog_product index does not exist yet. Make sure everything is reindexed

Exception #0 (LogicException): catalog_product index does not exist yet. Make sure everything is reindexed

0 /var/www/diyparts.staging/diyparts_dev/vendor/smile/elasticsuite/src/module-elasticsuite-catalog-rule/Model/Rule/Condition/Product/AttributeList.php(166): Smile\ElasticsuiteCore\Index\IndexOperation->getIndexByName('catalog_product', Object(Magento\Store\Model\Store\Interceptor))

1 /var/www/diyparts.staging/diyparts_dev/vendor/smile/elasticsuite/src/module-elasticsuite-catalog-rule/Model/Rule/Condition/Product/AttributeList.php(108): Smile\ElasticsuiteCatalogRule\Model\Rule\Condition\Product\AttributeList->getMapping()

2 /var/www/diyparts.staging/diyparts_dev/vendor/smile/elasticsuite/src/module-elasticsuite-catalog-rule/Model/Rule/Condition/Product.php(77): Smile\ElasticsuiteCatalogRule\Model\Rule\Condition\Product`

Reindexing does not help, so I've no clue how to further go on. Maybe you know what to do.
Installed Magento version: 2.1.0
using Elasticsuite version 2.2.0

I've updated to Magento version: 2.1.1
using Elasticsuite version 2.2.2 showing the same error.

[romainruaud]

hello @diybook and thank you for trying our module.

Did you run the setup:upgrade after adding the module to Magento ?

[diybook]

Yes I did. Initially I had troubles installing the plugins for the elasticuite server, and therefore get error messages. After successful install of the plugins no error occured, but maybe the db is messed up with something.

[romainruaud]

maybe you could give a try to removing Smile_* entries from the setup_module then process again a setup:upgrade. Or now that you have everything running fines regarding ES plugins, try a reinstall from scratch.

Regards.

[diybook]

unfortunately it's not working. I've installed the magento module on a new dump of the production setup. (clean db, and files) But the same error is occuring.

Do you have any other ideas?

[afoucret]

What about reindexinng did you run bin/magento index:reindex as explained into the doc.
if so, no errors ?

Did you check ES server install requirement is OK as explained into : https://github.com/Smile-SA/elasticsuite/wiki/ServerConfig ?

[diybook]

Yes, reindexing runs without any trouble and without error messages.

ES install should be ok. Server is up and running. But ES Server Version is 2.4.1. Maybe that this is a problem because install documentation shows version 2.2.0. I've installed JDK 1.8 as recommended.

curl localhost:9200 shows:
{
"name" : "Shanna the She-Devil",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "noY0LiV_SD20xL2YYy9wkA",
"version" : {
"number" : "2.4.1",
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
"build_timestamp" : "2016-09-27T18:57:55Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}

[afoucret]

Did you install the plugins (icu and phonetic) and did you restart the ES server after the install ?
Then reindex and it should run.

[diybook]

Yes I did this too. Initially I had problems installing the plugins, but in the end it worked. I've restarted the ES beacause otherwise reindexing did not work. But reindexing do not show any error.

[afoucret]

Hi @diybook,

I try to uninstall the phonetic plugin on my own environment to see own the error is reported, and I am able to see the following error :

Catalog Search indexer process unknown error:
index_creation_exception: failed to create index

And I get the following error in my ES logs located into /var/log/elasticsearch/ :

[2016-10-20 11:45:02,762][WARN ][cluster.action.shard     ] [Raymond Sikorsky] [magento2_default_catalog_category_20161017_131650][9] received shard failed for target shard [[magento2_default_catalog_category_20161017_131650][9], node[O3fNYMoIRUae8zvwYSHowQ], [P], v[3], s[INITIALIZING], a[id=VqWN--7lRk6TtYFZxtF1WQ], unassigned_info[[reason=ALLOCATION_FAILED], at[2016-10-20T09:45:02.666Z], details[failed to create index, failure IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown TokenFilter type [phonetic] for [phonetic]]; ]]], indexUUID [CqVnvghoQ8iXXkEvO1MAKA], message [failed to create index], failure [IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown TokenFilter type [phonetic] for [phonetic]]; ]

Tips : enabling debug log of ElasticSuite in the module configuration allow to have the ES error the Magento logs.

I think that your PHP environment is configured to not display errors even in CLI mode.

[diybook]

Hi I gave it another try. What I found out is that the server is running and working as intended. Plugins are installed and working. Reindexing do not show any error.

But the systemlog of magento shows the following error:
[2016-10-22 19:42:25] main.WARNING: Request Failure: {"method":"HEAD","uri":"http://localhost:9200/magento2_repariersdoch_at_catalog_product","headers":{"host":["localhost:9200"]},"HTTP code":404,"duration":0.003469,"error":""} [] [2016-10-22 19:42:25] main.WARNING: Response [null] []

I have no idea why the requestet index is not generated. The elasticsearch log shows the folloging lines:

[2016-10-22 21:17:03,013][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_default_at_catalog_category_20161018_120745][0]] ...]). [2016-10-22 21:46:52,531][INFO ][cluster.metadata ] [Ultimo] [magento2_repariersdoch_at_catalog_category_20161022_194652] creating index, cause [api], templates [], shards [1]/[0], mappings [category] [2016-10-22 21:46:52,788][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_repariersdoch_at_catalog_category_20161022_194652][0]] ...]). [2016-10-22 21:46:55,633][INFO ][cluster.metadata ] [Ultimo] updating number_of_replicas to [0] for indices [magento2_repariersdoch_at_catalog_category_20161022_194652] [2016-10-22 21:46:55,643][INFO ][index.shard ] [Ultimo] [magento2_repariersdoch_at_catalog_category_20161022_194652][0] updating refresh_interval from [10s] to [1s] [2016-10-22 21:46:55,778][INFO ][cluster.metadata ] [Ultimo] [magento2_repariersdoch_de_catalog_category_20161022_194655] creating index, cause [api], templates [], shards [1]/[0], mappings [category] [2016-10-22 21:46:55,901][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_repariersdoch_de_catalog_category_20161022_194655][0]] ...]). [2016-10-22 21:46:56,875][INFO ][cluster.metadata ] [Ultimo] updating number_of_replicas to [0] for indices [magento2_repariersdoch_de_catalog_category_20161022_194655] [2016-10-22 21:46:56,877][INFO ][index.shard ] [Ultimo] [magento2_repariersdoch_de_catalog_category_20161022_194655][0] updating refresh_interval from [10s] to [1s] [2016-10-22 21:46:56,972][INFO ][cluster.metadata ] [Ultimo] [magento2_diyparts_at_catalog_category_20161022_194656] creating index, cause [api], templates [], shards [1]/[0], mappings [category] [2016-10-22 21:46:57,073][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_diyparts_at_catalog_category_20161022_194656][0]] ...]). [2016-10-22 21:46:57,130][INFO ][cluster.metadata ] [Ultimo] updating number_of_replicas to [0] for indices [magento2_diyparts_at_catalog_category_20161022_194656] [2016-10-22 21:46:57,134][INFO ][index.shard ] [Ultimo] [magento2_diyparts_at_catalog_category_20161022_194656][0] updating refresh_interval from [10s] to [1s] [2016-10-22 21:46:57,213][INFO ][cluster.metadata ] [Ultimo] [magento2_default_ch_catalog_category_20161022_194657] creating index, cause [api], templates [], shards [1]/[0], mappings [category] [2016-10-22 21:46:57,297][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_default_ch_catalog_category_20161022_194657][0]] ...]). [2016-10-22 21:46:57,351][INFO ][cluster.metadata ] [Ultimo] updating number_of_replicas to [0] for indices [magento2_default_ch_catalog_category_20161022_194657] [2016-10-22 21:46:57,353][INFO ][index.shard ] [Ultimo] [magento2_default_ch_catalog_category_20161022_194657][0] updating refresh_interval from [10s] to [1s] [2016-10-22 21:46:57,431][INFO ][cluster.metadata ] [Ultimo] [magento2_repariersdoch_at_thesaurus_20161022_194657] creating index, cause [api], templates [], shards [1]/[0], mappings [] [2016-10-22 21:46:57,562][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_repariersdoch_at_thesaurus_20161022_194657][0]] ...]). [2016-10-22 21:46:57,597][INFO ][cluster.metadata ] [Ultimo] [magento2_repariersdoch_de_thesaurus_20161022_194657] creating index, cause [api], templates [], shards [1]/[0], mappings [] [2016-10-22 21:46:57,654][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_repariersdoch_de_thesaurus_20161022_194657][0]] ...]). [2016-10-22 21:46:57,689][INFO ][cluster.metadata ] [Ultimo] [magento2_diyparts_at_thesaurus_20161022_194657] creating index, cause [api], templates [], shards [1]/[0], mappings [] [2016-10-22 21:46:57,739][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_diyparts_at_thesaurus_20161022_194657][0]] ...]). [2016-10-22 21:46:57,780][INFO ][cluster.metadata ] [Ultimo] [magento2_default_ch_thesaurus_20161022_194657] creating index, cause [api], templates [], shards [1]/[0], mappings [] [2016-10-22 21:46:57,832][INFO ][cluster.routing.allocation] [Ultimo] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[magento2_default_ch_thesaurus_20161022_194657][0]] ...]).

No indexes for catalog_product are generated. But why? There are 5 products in the database? Any Ideas? Maybe there is something wrong with the database.

The default main category is empty because we use another main category for the configured shops. But the configuration is ok, because magento is working correct in this setup. How can I find out why the catalog_product index is not there?

Best regards,
Gerald

[diybook]

To close this issue some update: Installation worked on a new Magento installation. I'm not sure what was wrong but definitly something was wrong with the magento-db. Best regards, and thanks for your help.

[rbostan]

Same issue in Magento 2.1.2 with 5-6 catalog rules.
All indexes are fine except catalog rule product and catalog product rule.

There is about 1 million single (both configurable and children) products so catalog rule indexers takes too long.

Why should we wait to complete all indexers to be completed ?

[blewinsky]

I've got that issue too and discovered that this functions doesnt index data for admin area (Store 0):
Magento\CatalogSearch\Model\Indexer\Fulltext::executeFull()
Smile\ElasticsuiteCatalog\Model\Category\Indexer\Fulltext::executeFull()

For the test when i added $storeIds[] = 0; after this assigment $storeIds = array_keys($this->storeManager->getStores()); in both functions and reindexed again the warning was disappeared.

Magento ver. 2.1.2
smile/elasticsuite 2.3.2

[ericrisler]

Perhaps helpful: if you do not change the search engine setting in Stores > Configuration > Catalog > Catalog Search then the catalog_product index will not be generated in ElasticSearch.

[amenk]

as written before:
https://github.com/Smile-SA/elasticsuite/issues/173#issuecomment-321014140

Run the index, while elasticsuite is selected in the backend.

n98-magerun2 config:store:set catalog/search/engine elasticsuite
n98-magerun2 cache:flush
n98-magerun2 index:reindex

[matheusgontijo]

In most cases solution provided from @amenk works great.

Another possible scenario is that the value for status column from indexer_state table is working for some (or many) indexes as catalogsearch_fulltext.

That was making the command index:reindex SKIP that index. In other words, it was not processing that index due to a past interrupted command that never updated the status from working to invalid.

Make sure you set it as invalid ;- )

[ksaho2]

I also want to share my solution on this. I am running ES under Docker, and found that the above solutions did not work until after I restarted ES.

[jasonhildebrand]

Thanks for the tip @matheusgontijo
To expand a bit for others with this issue, it can happen that some indexes don't get rebuilt even if you do 'bin/magento index:reindex'.

To know for sure, go into your database and run select * from indexer_state, look at the updated column.
To force all indexes to be updated, run update indexer_state set status = 'invalid'
Then re-run bin/magento index:reindex.

[jtomaszewski]

For me that issue started to appear when my bin/magento index:reindex script exploded with PHP Fatal error: Allowed memory size of ... bytes exhausted error. That caused an index to be forever in a "processing state".

Also, instead of doing raq sql queries, you can force all indexes to be updated with bin/magento indexer:reset command. (Magento 2.3.4)

[robsch]

Thank you @ericrisler! The problem is the documentation. Should be a screenshot like the other configuration explanations above.

[nuovecode]

Thanks for the tip @matheusgontijo
To expand a bit for others with this issue, it can happen that some indexes don't get rebuilt even if you do 'bin/magento index:reindex'.

To know for sure, go into your database and run select * from indexer_state, look at the updated column.
To force all indexes to be updated, run update indexer_state set status = 'invalid'
Then re-run bin/magento index:reindex.

Thanks, you helped me a lot