Kibana: Can't work with index patterns with significant number of fields

[davekonopka]

Kibana version: 5.6.3

Elasticsearch version: 5.6.3

Server OS version: Amazon Linux

Browser version: Chrome Version 62.0.3202.89 (Official Build) (64-bit)

Browser OS version: macOS 10.12.6

Original install method (e.g. download page, yum, from source, etc.): Kibana Chef Cookbook via download url

Description of the problem including expected versus actual behavior:
I have a set of time series indices with a significant number of nested objects in Elasticsearch. I have a wildcard index pattern in Kibana covering these indices. I've hit a wall and can no longer view the index pattern on the Discover or index management pages. It seems to be due to field metadata being passed to Elasticsearch overrunning a post payload limit.

The error message is "Payload content length greater than maximum allowed: 1048576" Neither the discovery or index management pages ever load anything inside the main panel.

Looking in the inspector, a call is made to _fields_for_wildcard. A message with metadata on all the fields returns. And then a call to the index pattern name fails with a 400 error.

The payload looks to be all the fields from the index pattern mappings along with metadata for each field. It is large covering roughly 7,000 fields. We end up with a lot of repetitive field names due to nested objects in the Elasticsearch documents. For example:

• level1.level2.level3.level4.level5.some_field1
• level1.level2.level3.level4.level5.some_field2
• level1.level2.level3.level4.level5.some_field3
• level1.level2.level3.level4.level5.some_field4

Is this a bug? Is there anything we can do to avoid having to pass all this field data around?

Steps to reproduce:

1. Have an index pattern with a lot of nested objects resulting in large number of fields with long, repetitive field names in Kibana's index pattern configuration.
2. Attempt to open the index pattern on the Discovery or Management index pattern pages.

Errors in browser console (if relevant):

_fields_for_wildcard
{{index name}}

Error on accessing index:

index: 400 bad request
    error: "Bad Request"
    message: "Payload content length greater than maximum allowed: 1048576"
    statusCode: 400

Provide logs and/or server output (if relevant):

Describe the feature:

[Bargs]

The limit you're hitting is in the Kibana server. You can adjust this in kibana.yml with the server.maxPayloadBytes setting.

https://www.elastic.co/guide/en/kibana/current/settings.html