Elasticsearch: indices starting with - (dash) cause problems if used with wildcards

[yehosef]

I was looking through the indices on one of our hosts and saw some indices that started with - (dash), eg "-2016.04.15". I'm not sure why it was there - but not to critical, elasticsearch lets you do it.

POST -2016.12.12/test
{
  "name":"abc"
}

I tried to delete these indices by issuing
DELETE -2016.*

The problem is that this was interpreted as
DELETE everything except for indices starting with 2016.
which basically means delete the entire database - and after a few poignant seconds, that's what it did.

I have since become acquainted with https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html and the ability to include or exclude indices with the + or - operator, but it seems that this is more dangerous than useful, at least if you are unfortunate enough to have indices that start with -.

I understand that it's a "feature", but it doesn't seems practically so useful. Perhaps there could be a special query string for DELETE like "wildcard=inclusive" or "=exclusive"... As it is now, I'm not even sure how I would delete the indices that start with "-2016." I can't do "+-2016.*"

[clintongormley]

So curiously, you can DELETE -whole_index_name. It's only if you specify the wildcard that the +/- behaviour kicks in. This in itself sounds like a bug.

In order to remove ambiguity, I think we should prevent index names starting with + or -.

Related https://github.com/elastic/elasticsearch/issues/9059

[yehosef]

yes.. that would be fine.

[colings86]

Discussed in Fix It Friday and we agreed that we should fix the bug that the +/- behaviour does not work unless there is a wildcard, and also prevent index names starting with a + or -

[makeyang]

@colings86
I'd like to pull request for this issue. but I have some questions to ask:

1. what do u mean by "the +/- behaviour does not work unless there is a wildcard", let's say I had only one index "twitter", and I delete with name "+twitter", what will happen? take as index not found or delete twitter?

[makeyang]

@colings86
also is there anywhere ES define "wildcard"?

[yehosef]

FWIW - the case of using an exclusion in the index name in the docs was together with an inclusion - +test*,-test3 I question the usefulness of an exclusion by itself. How often do you really want to do DELETE -test-* and doing a +test* is not needed, because it's inherently "+". You would just do DELETE test*

I would offer that because the implications of someone misunderstanding and and its questionable need, perhaps you should consider putting it in a query string option, then it's easier that it's more intentional. eg to do the command that would now be DELETE -test-*, instead do a DELETE *?exclude=test-*. Then it's much more obvious what you're doing but you still have the same power.

The truth is that this is really mainly a problem with DELETE, perhaps these changes should just be made here.

[yehosef]

note - some of this discussion is still relevant even if you remove dashes from the start of queries. DELETE -test* is still just as dangerous and might not be obvious to some users what would happen.