Kibana: discover reporting no index patterns despite >1 existing

[graphaelli]

Kibana version: 7.2.0 bc4

Elasticsearch version: 7.2.0 bc4

Original install method (e.g. download page, yum, from source, etc.): docker

Describe the bug:

Unable to get past the discover "you need index patterns screen" despite index patterns existing. Maybe important - none are starred.

Steps to reproduce:

1. Start 7.2 bc4 with kibana/apm-server/metricbeat
2. Navigate to discover and see prompt for index patterns even though some already exist, though none are starred.

Expected behavior: able to discover

Screenshots (if relevant):

Errors in browser console (if relevant): none

Provide logs and/or server output (if relevant):

Requested by the page with no patterns:

http://localhost:5601/api/saved_objects/_find?type=index-pattern&fields=title&per_page=10000

{
  "page": 1,
  "per_page": 10000,
  "saved_objects": [
    {
      "attributes": {
        "title": "apm-*"
      },
      "id": "apm-*",
      "migrationVersion": {
        "index-pattern": "6.5.0"
      },
      "references": [],
      "type": "index-pattern",
      "updated_at": "2019-06-03T23:09:45.841Z",
      "version": "WzIsMV0="
    },
    {
      "attributes": {
        "title": "metricbeat-*"
      },
      "id": "metricbeat-*",
      "migrationVersion": {
        "index-pattern": "6.5.0"
      },
      "references": [],
      "type": "index-pattern",
      "updated_at": "2019-06-03T23:09:57.043Z",
      "version": "WzMsMV0="
    }
  ],
  "total": 2
}

Any additional context:

This does not happen without metricbeat also starting. Removing one of the patterns resolves the issue, as does starring one of them. Kibana should probably just pick one instead?

[graphaelli]

in 7.1, this is shown:

removed the bug label for now in case that user messaging was removed intentionally.

[jasonrhodes]

Seems like this happens when 2+ index patterns are created programmatically (outside the UI), since that seems to be the only way to create index patterns without specifying as a default. If you create one in the UI and don't have a default already, it makes the one you just created "default".

I agree with @graphaelli that it seems like Kibana should just pick one, or at least show the message from 7.1…

[elasticmachine]

Pinging @elastic/kibana-app

[jen-huang]

@mattkime / @elastic/kibana-app

With the revamped Index Pattern UI, the messaging about picking a default index pattern from a list of existing ones is removed. It's also less obvious how to make an existing index pattern default in this flow, since the list of patterns no longer shows on the side next to the create index pattern wizard.

Maybe we can 1) put the messaging about choosing a default back and 2) redirect to the main Index Pattern page (with list and Create button), instead of automatically going into the wizard?

IMO this is a high priority bug. Maybe even a blocker for 7.2.

[mattkime]

Takin' a look.

[mattkime]

I think this deserves a good ramble -

There are two reasons to want a default index pattern in place. First, I just want to play with something on the screen and second I want to use a particular data set. I suspect that having a good default default-index-pattern is to address the first case. Thats what I'm focusing on.

We can resolve this indeterminate state in one of two places - either at read or write of index patterns.

I've gone searching through the code to see where we create index patterns and I'm finding the code a little indirect and I'm therefore concerned that I'm not finding all the cases. I'm looking at you, bulkCreate!

As for reading default index patterns, grepping for requireDefaultIndex appears reliable, pointing to discover, dashboard, and visualize. We should check to see if dashboard and visualize suffer from similar problems. Ideally we'd have shared code to solve this in all defects. It appears that savedObjectClient is used anytime index patterns are accessed so it might be a bit extra work.

[wylieconlon]

There is some suspicious logic in load_default.js which is used by the route parameter requireDefaultIndex. Since this logic only defaults if there is one index, it is probably failing if there are two default indicies:

          if (!defined) {
            // If there is only one index pattern, set it as default
            if (patterns.length === 1) {
              defaultId = patterns[0];
              config.set('defaultIndex', defaultId);
            } else {
              throw new NoDefaultIndexPattern();
            }
          }

[AlonaNadler]

Can we default on the first index pattern created for now?
I think sending all users to the index pattern page when there are already index patterns because no default index pattern was selected is worse than the risk of defaulting on the wrong index pattern (which admins can change). It will also be worse with Feature Control when users might not have access to index patterns and they will get stuck.

In general, the concept of a default index pattern is a bit weird, why do we must specify a default index pattern?

[wylieconlon]

I've put up a PR which we can discuss- this behavior has been around for a while based on the git history, but it seems like the recent issues with the apm-* index have made it more noticeable. The fix for the apm-* is related: #37965