Elasticsuite: Automatic and/or bulk purge of ghost indices

[scottsb]

Is your feature request related to a problem? Please describe.
It was great to get the admin panel that shows ghost indices in a recent version. However, if they have built up for a while (we had over a thousand), it is nearly impossible to delete all of them manually.

Describe the solution you'd like
A background job that automatically deleted ghost indices and/or an admin feature to bulk delete them on demand.

This was previously brainstormed by @romainruaud here:
https://github.com/Smile-SA/elasticsuite/issues/1278#issuecomment-458858078

Describe alternatives you've considered
Using Curator, which is possible but has greater operations overhead.

[stephanie-ehrling]

I would love to have a feature like that. We also have had almost thousand ghost indices. I deleted all indices and recreated them but a job to delete them would be great.

[stephanie-ehrling]

I implemented it here: https://github.com/stephanie-ehrling/elasticsuite/tree/delete-ghost-indices
and I will test it for the next days to see if that prevents generating ghost indices for us.

[romainruaud]

That's a good job @stephanie-ehrling.

I don't want to push additional pressure but what do you think about publishing this as a standalone module for Elasticsuite ?

That's not something I want to include in the Elasticsuite core because I don't want us to be responsible for automatic indices deletion, because "with great power comes great responsibilities" and I'm concerned about some "not really ghost" indices suffering an automatic deletion if we include this in core and it gets delivered to all our users on next update : in my opinion people should install this separate module on purpose and assume responsibilities.

=> That's why having it as a separate module is fine, it will be installed only when needed.

Regards

[stephanie-ehrling]

Hi @romainruaud ,
that's a good point and it totally makes sense. Since it is a very simple module I think I can manage it to put this in a separate module during this week.

[stephanie-ehrling]

Created a pull request #2004

[romainruaud]

Oh, I was expecting rather a separate module that will not get merged into the Elasticsuite core :)

You can publish it under your name (or company name if you prefer to), and we'll add a link into this on the homepage with the other community/third-party modules.

Regards

[scottsb]

@stephanie-ehrling Do you anticipate being able to turn your PR into a separate module in the near future? If not, then I may take that on, but I don't want to interfere with something you're working on or expect to do soon.

[stephanie-ehrling]

Hi Scott, the module is now available here: https://github.com/stephanie-ehrling/elasticsuite-ghost-cleaner

[scottsb]

Great, thanks!