Elasticsearch: changing search_analyzer on existing field returns error

[javanna]

Trying to change the search analyzer for an existing field through put mapping api:

PUT test/my_type/1
{
  "field":"value"
}

PUT test/_mapping/my_type
{
  "properties": {
    "field" : {
      "type": "string",
      "search_analyzer": "keyword"
    }
  }
}

I get back "analyzer on field [field] must be set when search_analyzer is set". The error seems to imply that I have to add the "analyzer" too, although it is the default one already. Adding "analyzer": "standard" as follows:

PUT test/_mapping/my_type
{
  "properties": {
    "field" : {
      "type": "string",
      "search_analyzer": "keyword",
      "analyzer": "standard"
    }
  }
}

causes a "Merge failed with failures {[mapper [field] has different [analyzer]]}". I am not sure what I am doing wrong or whether there is a bug somewhere, but at least the errors here really need to be improved and clarified.

[nik9000]

That sounds like trouble. I'm pretty sure I remember changing the search analyzer on the fly a few times.

[jimczi]

By default the analyzer is set with the name "default" (and not "standard" though default points to "standard" ...). If you change the analyzer name to "default" you'll be able to change the search_analyzer ;)

[javanna]

cool thanks @jimferenczi ! still I don't get why one has to specify the analyzer (and has to know to use default instead of standard). It might make sense for new mappings to be explicit, not for fields that are already defined. Seems like bad user experience to me.

[rjernst]

I agree it makes sense only for new mappings. I think the problem is simply that standard and default are not considered "the same" in the validation code.

[rjernst]

Scratch my last statement. We have no way to distinguish when this is a "new mapping" vs an "update".

[rjernst]

I also think this is actually a good thing. It makes it very clear to the user that they now have 2 analyzers acting on the field.

[djschny]

Agree with @javanna that this is a bad user experience. I fail to see how this makes it very clear to the user.

Perhaps instead make it so when you GET a mapping instead of having implied values that are not present, we always explicitly include in the json the analyzer so that way users will be able to see the default vs standard situation.

[rjernst]

@djschny I was not referring to the problem with default vs standard. The request in first post here that included analyzer should have worked. We should fix that. I was only referring to the fact that analyzer must be included when updating search_analyzer.

[javanna]

I understand in general that being explicit is better, but if we just want to change the search_analyzer, and an analyzer is already defined for the field, and we don't want to change that analyzer, why should one be forced to specify again that same analyzer, which is also the only one that will be accepted (as the index analyzer cannot be changed)? That seems misleading from a user perspective, the fact that technically it's challenging to distinguish between new mappings and updates is a different story, that we may or may not have to deal with internally.

[jpountz]

Mappings have a way to apply a default source. For instance this is how the _default_ mapping is applied. On the contrary when a mapping already exists for a given type, we don't apply any defaults. I'm wondering that maybe this issue could be solved by applying the current mapping as a default source when mappings for a type already exist?

[clintongormley]

@jpountz that sounds like a plan

[clintongormley]

And honestly the requirement to have to be explicit about setting an analyzer when adding a search_analyzer is just annoying - I see no real benefit in this requirement.

[jpountz]

cc @elastic/es-search-aggs