Elasticsearch: Doc: Clarify count for index.mapping.total_fields.limit

[ppf2]

https://www.elastic.co/guide/en/elasticsearch/reference/5.3/mapping.html#mapping-limit-settings

index.mapping.total_fields.limit
The maximum number of fields in an index. The default value is 1000.

Currently, documentation simply says index.mapping.total_fields.limit is the max number of fields in an index. For end users who are looking at the mappings api output and doing a count for validation, they will notice that ES will return a limit exception before the perceived # of fields hit 1000.

This is because index.mapping.total_fields.limit includes the count of meta fields and most of these will not show up in the mappings api output. It will be nice to clarify this in the documentation, similar to how we do clarify that index.mapping.depth.limit counts also the root level as 1 level.

[jpcarey]

This through me for a loop. I had set the limit to 10, in which it is impossible to post a document to an index. If you use a limit of 20, you can have a total of 4 fields in your document. Not entirely sure how we have 16 meta type fields...

DELETE test
PUT test
{
  "settings": {
    "index.mapping.total_fields.limit": 20
  }
}
POST test/t/1
{
  "0": "hello",
  "1": "hello",
  "2": "hello",
  "3": "hello"
}
# ^ works
POST test/t/2
{
  "0": "hello",
  "1": "hello",
  "2": "hello",
  "3": "hello",
  "4": "hello"
}
# hits the 20 field limit

What is also confusing: Using the 20 field limit (4 real fields), you can create 4 unique fields for as many types as you can create. The field limit is being applied to the _type, not the total _mapping.

[jimczi]

This through me for a loop. I had set the limit to 10, in which it is impossible to post a document to an index. If you use a limit of 20, you can have a total of 4 fields in your document. Not entirely sure how we have 16 meta type fields...

Since you use dynamic mapping in your example, each field will create an extra ".raw" field.
So you have 8 fields in your mapping and there are 12 metadata fields in es 5.x:

• _routing
• _ttl
• _index
• _type
• _all
• _parent
• _field_names
• _source
• _id
• _version
• _timestamp
• _uid

Some of these fields are deprecated or not activated by default but we count them in the limit.

This is because index.mapping.total_fields.limit includes the count of meta fields and most of these will not show up in the mappings api output. It will be nice to clarify this in the documentation, similar to how we do clarify that index.mapping.depth.limit counts also the root level as 1 level.

Maybe it would be simpler if we remove the metadata fields from the computation. We don't want to list all metadata fields in this doc just for the sake of counting and they are in the mapping anyway.

What is also confusing: Using the 20 field limit (4 real fields), you can create 4 unique fields for as many types as you can create. The field limit is being applied to the _type, not the total _mapping.

This will fix itself with the removal of types so not sure if we should do something here.

[elasticmachine]

Pinging @elastic/es-search-aggs

[nfsec]

A moment ago I had the same problem and only information from github clarified the principle of operation.