Kibana: performance regression in area visualizations

[hydrapolic]

Kibana version:
7.9.0

Elasticsearch version:
7.9.0

Server OS version:
Linux Gentoo

Browser version:
Firefox 79.0

Browser OS version:
Linux OpenSuse

Original install method (e.g. download page, yum, from source, etc.):
Gentoo packages

Describe the bug:
Hello,just upgraded kibana oss from 7.8.1 to 7.9.0 and my area visualizations take forever to load (cpu is burning). The y-axis is set to Count, the x-axis has buckets aggregated by @timestamp and the series are split by time ranges. It worked like a charm in 7.8.1.

Steps to reproduce:

1. Upgrade kibana to 7.9.0 and check the dashboards.

Expected behavior:
Reasonable load times

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[nreese]

Maybe related to https://github.com/elastic/kibana/issues/76208?

[filippog]

Maybe related to #76208?

Can confirm the issue, seems to affect most/all types of visualizations

[hydrapolic]

Is it possible to easily downgrade to 7.8.1? Just trying to revert the binaries back didn't allow it, somewhere I've found you need to restore from a backup.

[insatomcat]

my solution to rollback was to:

• stop kibana
• revert the .kibana alias to the previous index (in this example I go from 10 to 9, list your indices to find your latest/previous numbers):

curl -X POST -k https://xxx:[email protected]:9200/_aliases -H 'Content-Type: application/json' -d '{ "actions" : [ { "remove" : { "index" : ".kibana_10", "alias" : ".kibana" } }, { "add" : { "index" : ".kibana_9", "alias" : ".kibana" } } ] }'

• remove the latest index:

curl -X DELETE -k https://xxx:[email protected]:9200/.kibana_10

• change your kibana binary to the previous version you used (I personnaly use docker images)
• start kibana

[wylieconlon]

Fixed with https://github.com/elastic/kibana/pull/75850, will release in 7.9.1

[filippog]

Thanks @wylieconlon and all for the update!

As far as I can see #75850 fixed defaultsDeep usage only in one call site. During my testing I found other spots in the visualization library that experienced the same behaviour, e.g. src/plugins/vis_type_vislib/public/vislib/lib/types/point_series.js that I believe need some kind of the same fixes (cc @mbondyra as author of the PR above, what do you think?)

[hydrapolic]

Fixed with #75850, will release in 7.9.1

@wylieconlon just tried to apply the patch, but it didn't help, it's still slow.

[wylieconlon]

Okay it seems like this needs more triage.

[mbondyra]

@hydrapolic could you run a performance profiler from Chrome Dev Tools while executing a critical action (eg. refreshing the chart) and send the file?
@filippog thanks for pointing it out - you are right, there are several places where we use defaultsDeep and we might need to revisit if the other places also need fixing. I'll take a look 👀

[hydrapolic]

@mbondyra sent to your email. Thanks

[mbondyra]

Hi @hydrapolic I received your files, thank you! What surprises me is that the patch didn't help, because it looks exactly like the problem I fixed in my PR. The initVisConfig function runs VisConfig constructor that runs lodash function that takes around 8 seconds with each execution (so in the end, the chart renders for more than 16 seconds). Could you please double-check with the patch? If you make sure it didn't help, could you also send me the performance json file after applying the patch?

When it comes to other places that use _.defaultsDeep, the object we're merging there are much much smaller (for comparison, the object from initVisConfig from the original PR holds 11MB of data and the others (axisConfigArgs and gridConfig) are no bigger than ~100bytes) so they are not noticeable when it comes to performance. It can be worth to apply the same change, but it's not the reason for performance regression.

[filippog]

Thank you @mbondyra for taking a look! I've applied 446c5237d5 on top of v7.9.0 and can confirm I'm not seeing the CPU spiking up anymore and slow render times.

[hydrapolic]

@mbondyra I've applied the patch to 7.9.0, even removed the optmize/bundles and restarted kibana, but still seeing the same slowness. The HAR/profile sent was already using this patch.

Can the patch be applied to https://artifacts.elastic.co/downloads/kibana/kibana-7.9.0-linux-x86_64.tar.gz or you need to rebuild kibana?

[hydrapolic]

The performance is restored after upgrading to 7.9.1.

Thanks for the help and support.