Elasticsearch: Wildcard index templates can impact security index working

[bizybot]

Elasticsearch allows the creation of index templates using wildcards which may be applicable to security index. We have seen this happening time to time and when that happens, the settings and mappings from these templates may override our security index template. The changes can be catastrophic, impacting the security of the system. It is difficult to debug when this happens.

This issue exists to discuss and find a solution:-

• to identify when this happens via failure or warning message
• to prevent this from happening

We may use the solution once available for the core issue related to index template inheritance (#21105), but that seems unlikely as ES security does not use the template for security index.

[elasticmachine]

Pinging @elastic/es-security

[jaymode]

I really think these are the types of issues that drive the need for system indices as a concept within elasticsearch. I would love to see us put together a list of things that we’d like system indices to solve, this being one of them.

[bizybot]

Yes, I think collecting all such requirements would be a good way to identify the need for system indices concept and what the expectations are. I will try to go through the issues/support tickets to identify common things and create a team discuss-issue for it.

[ppf2]

+1

ILM introduces another feature (via index templates) where users can inadvertently cause havoc on our system indices dot indices. @jakelandis

[droberts195]

The problem of customer templates applying to internal indices has been previously raised in #29605. There was an idea for what to do in https://github.com/elastic/elasticsearch/issues/29605#issuecomment-383100069.

It would be nice if the eventual solution could be used by all plugins that require internal indices, and not just security.

I think there are also two types of internal indices that we may wish to have slightly different restrictions on:

1. Indices that need to be searchable externally, for example by the UI, but need to be protected from having customer index templates applied to them
2. Indices storing highly sensitive data that need to be protected from having customer index templates applied to them _and also_ protected from easy grants of read or write access to non super users

I guess security indices would fall into category 2, but ML has indices in both categories (and I think Watcher does too).

For indices in category 1 it would be nice for debugging purposes if wildcarded searches were still possible - see https://github.com/elastic/elasticsearch/issues/29605#issuecomment-383551818.

[elasticmachine]

Pinging @elastic/es-core-infra

[jasontedor]

@droberts195 The rough idea for system indices is that each plugin that manages a system index will also dictate the API that is available on top of that index (e.g., maybe some plugins expose search, and others do not). It would likely be via dedicated APIs though, not the general APIs.

[droberts195]

It would likely be via dedicated APIs though, not the general APIs

I think for ML that means we'd keep some of our indices as standard indices and just make a subset into system indices (especially the ones that store security headers in configs).

/cc @davidkyle @hendrikmuhs

[gwbrown]

This issue has been rolled into the effort to formalize and protect system indices (see https://github.com/elastic/elasticsearch/issues/50251), and in particular is actively being addressed as part of https://github.com/elastic/elasticsearch/issues/61656 to allow plugins to provide settings and mappings directly, rather than via index templates.

As this issue has been superseded by issues describing more concrete features and changes to address this issue, I'm going to close this.