Kibana: Saving rearranged objects in dashboard does not always persist

[ppf2]

Not sure if this is the same as https://github.com/elasticsearch/kibana/issues/1830.
But I have a dashboard where the rearrangement of objects on it does not persist after a save. Certainly not reproducible in all dashboards.

@rashidkpc I will send you the repro site and video for it.

[theNewFlesh]

I'm getting the same problem. The panelsJSON data is correct, both from within Kibana's object editor and from an Elasticsearch query of the .kibana index. However, the URL specifies a different numbers for the row field of each graph, rather than what it should do (ie whatever is listed in the actual dashboard document).

[theNewFlesh]

Here is the info:

Kibana URL:
http://10.10.152.27:5601/#/dashboard/DEV-7-Version-Comparison?_g=()&_a=(filters:!(),panels:!((col:1,id:'DEV-7-Average-Ray-Count-(per-ray-type-over-versions)',row:5,size_x:12,size_y:4,type:visualization),(col:1,id:'DEV-7-Memory-Consumed-(per-type-over-versions)',row:13,size_x:12,size_y:4,type:visualization),(col:1,id:'DEV-7-Render-Node-Cores-(over-versions)',row:21,size_x:4,size_y:4,type:visualization),(col:5,id:'DEV-7-Render-Node-CPU-speed-(over-versions)',row:21,size_x:4,size_y:4,type:visualization),(col:9,id:'DEV-7-Render-Node-Threads-(over-versions)',row:21,size_x:4,size_y:4,type:visualization),(col:1,id:'DEV-7-Render-Time-(by-type-over-versions)',row:9,size_x:12,size_y:4,type:visualization),(col:1,id:'DEV-7-Render-Time-(per-phase-over-versions)',row:1,size_x:12,size_y:4,type:visualization),(col:1,id:'DEV-7-Textures-Read-vs-Total-(over-versions)',row:17,size_x:12,size_y:4,type:visualization)),query:(query_string:(analyze_wildcard:!t,query:'*')),title:'DEV%20-%207%20-%20Version%20Comparison')

ES query:
{u'_id': u'DEV-7-Version-Comparison', u'_index': u'.kibana', u'_score': 1.0, u'_source': {u'description': u'', u'hits': 0, u'kibanaSavedObjectMeta': {u'searchSourceJSON': u'{"filter":[{"query":{"query_string":{"analyze_wildcard":true,"query":"*"}}}]}'}, u'panelsJSON': u'[{"col":1,"id":"DEV-7-Average-Ray-Count-(per-ray-type-over-versions)","row":21,"size_x":12,"size_y":4,"type":"visualization"},{"col":1,"id":"DEV-7-Memory-Consumed-(per-type-over-versions)","row":9,"size_x":12,"size_y":4,"type":"visualization"},{"col":1,"id":"DEV-7-Render-Node-Cores-(over-versions)","row":13,"size_x":4,"size_y":4,"type":"visualization"},{"col":5,"id":"DEV-7-Render-Node-CPU-speed-(over-versions)","row":13,"size_x":4,"size_y":4,"type":"visualization"},{"col":9,"id":"DEV-7-Render-Node-Threads-(over-versions)","row":13,"size_x":4,"size_y":4,"type":"visualization"},{"col":1,"id":"DEV-7-Render-Time-(by-type-over-versions)","row":5,"size_x":12,"size_y":4,"type":"visualization"},{"col":1,"id":"DEV-7-Render-Time-(per-phase-over-versions)","row":1,"size_x":12,"size_y":4,"type":"visualization"},{"col":1,"id":"DEV-7-Textures-Read-vs-Total-(over-versions)","row":17,"size_x":12,"size_y":4,"type":"visualization"}]', u'title': u'DEV - 7 - Version Comparison', u'version': 1}, u'_type': u'dashboard'}

Kibana Object Editor:

[
  {
    "col": 1,
    "id": "DEV-7-Average-Ray-Count-(per-ray-type-over-versions)",
    "row": 21,
    "size_x": 12,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 1,
    "id": "DEV-7-Memory-Consumed-(per-type-over-versions)",
    "row": 9,
    "size_x": 12,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 1,
    "id": "DEV-7-Render-Node-Cores-(over-versions)",
    "row": 13,
    "size_x": 4,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 5,
    "id": "DEV-7-Render-Node-CPU-speed-(over-versions)",
    "row": 13,
    "size_x": 4,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 9,
    "id": "DEV-7-Render-Node-Threads-(over-versions)",
    "row": 13,
    "size_x": 4,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 1,
    "id": "DEV-7-Render-Time-(by-type-over-versions)",
    "row": 5,
    "size_x": 12,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 1,
    "id": "DEV-7-Render-Time-(per-phase-over-versions)",
    "row": 1,
    "size_x": 12,
    "size_y": 4,
    "type": "visualization"
  },
  {
    "col": 1,
    "id": "DEV-7-Textures-Read-vs-Total-(over-versions)",
    "row": 17,
    "size_x": 12,
    "size_y": 4,
    "type": "visualization"
  }
]

[passing]

simliar to #3328 & #2945

[spalger]

@GustavoHoyer made an interesting point in #3328; sorting the panels in the JSON object by row/col may help prevent this behavior.

[cphoover]

This is really really frustrating

[Funbit]

I'm facing with the same problem. It does not matter what browser I'm using. I hope that next versions will sort this JSON automatically so gridster plugin works as it should...

[asatsi]

Is this fix tagged to any release already? This is annoying and frustrating to re-arrange the dashboard everytime we launch it.

[gohoyer]

Hi @asatsi ,

As mentioned in #3328, there is a workarround. You can edit the Dashboard on the settings tab and sort your visualizations by line on the panelsJSON attribute.

[marcinkubica]

@GustavoHoyer @rashidkpc

I got it reproduced in 4.2.0- when 'Store time with dashboard' is on, then it doesn't save the layout.

However it seems to be a kind of a memory leak or something, because you need to save your dashboard several times when keep adding visualizations.

[skeller88]

The workaround @GustavoHoyer suggested did not fix my issue, but I found another workaround. Create a new dashboard with a different name, and add the visualizations to that dashboard in the desired order, then save. Then overwrite the problematic dashboard with the new dashboard.

[andyford]

Is a fix for this issue planned? It makes grouping related visualizations on a dashboard impossible.

[markwalkom]

Ran into this same thing with a customer, the fix that @GustavoHoyer mentioned here worked.

[dowdell]

@GustavoHoyer's fix also worked for me

[vikrim1]

Really sad about this.. Happens almost every time I add something to our dashboard. :( :(

[vikrim1]

Hi @rashidkpc, is there any plan for fixing this? I simply can't modify our dashboard anymore. @GustavoHoyer's fix sort of works, but every time I re-add a visualization a different one gets rearranged.

[bharatbhatia]

I have worked with Elastic Support team, and provided them enough material to be able to reproduce it. They will work on it hopefully soon.

[ppf2]

This has been reproduced internally. Removing not reproducible label. @Bargs is looking into it.

[vikrim1]

Great, thanks!

[Bargs]

I've recreated a troublesome dashboard using dummy data. Here are the steps to reproduce:

1. Create some fake logstash data and a Kibana index pattern for it using makelogs.
2. Import these visualizations via the Settings -> Objects tab.
3. Import this dashboard in the same way.
4. Open the dashboard you just imported.
5. Grab foo8 and swap places with foo13, then save the viz.
6. Reload the page, notice that foo8 and foo13 are now back in their original positions.

[Bargs]

I wasn't successful in using the previously mentioned workaround to fix this dashboard, so I'm going to assign this a P2 unless someone can show that a workaround exists for the example I just posted.

[rashidkpc]

I'm going to bump this to P1, its a really annoying bug and we should fix it asap

[vikrim1]

Thanks Rashid. Will this be fixed in just the newest kibana version or can you also fix it in a ES 1.x compatible release? (We haven't upgraded to ES 2 yet)

[pemontto]

:+1: seem to be hitting this issue a lot lately. My workaround it to delete and the save the problematic panel. In my case it's always been a single panel that causes issues.

[tbragin]

Actually bumping to P1, per Rashid's comment above. I just ran into this with one of the dashboards I'm playing with as well.

[niempy]

+1
Since we have severe issues with this (running Kibana 4.4.2 and Elastic 2.2.1).
Our users complain a lot.
Thanks.

[nl06019]

+1
After upgrading to the latest versions it does not work anymore.

[aowen5000]

+1
Tiles moving after dashboard is saved

[baddogdown]

+1 occurring on Kibana 4.5 running on Found.

[asiletto]

+1
same problem

[janbernhardt]

+1
same issue

[calumah]

+1 same issue on Firefox 47.0 and Kibana 4.5.1 Only after a moment with a very large dashboard some elements (specially at bottom) return to their previous place after saving. I will try @pemontto fix or divide dashboard by theme to see.

[citricut]

+1
We are Gold subscription clients and this is really important to us. The users are already complaining.

[asiletto]

does a Gold subscription help to solve this issue? I would be happy to buy it if so. Same problem as @citricut here, big organization with IE11 only desktops

[sethyes]

Pemontto's suggestion to delete and re-add the offending visualization to the dashboard has resolved my issue -- it's a pain but it works right now.

[climbingsavage]

Aaaaarrrgh this problem! Causing a lot of havoc in my organization, and a lot of hours lost. Workarounds do not seem to solve problems consistently.
Can we expect it to be fixed in 4.x or must we wait until 5?

[ckmcd]

+1 Very frustrating.

[GojiSoft]

+1
Same issue

[nicholaskuechler]

+1

Experienced the issue with a newer version of kibana:

Version 4.5.4
Build 10000
Commit SHA 00d0f49

[tedivm]

This bug is going to hit it's second birthday in a couple of months. I'm not sure people understand what P1 means . . .

[vtipparam]

+1
Please fix it. I'm afraid of modifying dashboards because of this annoying bug.

[syedyali]

Hello, did this issue get fixed?
The dashboard get messed up and not in it's original organized way when it was saved. This causes real issues when the operations team is monitoring 24/7 in production.

[tedivm]

It's fixed in Kibana 5- I just upgraded a few days ago and a lot of really annoying bugs are gone.

[gskema]

@gustavohoyer Fix worked for me, what I did:

• Order the dashboard the way I want in one browser window,
• Edit the dashboard JSON in settings, order the array items the same way they were displayed in the other window.
• Update panelIndex, starting from 1

[jfiore-ibm]

I implemented the workaround that @gskema mentioned above in a groovy function and it seems to work pretty well.
Hopefully this will save some people some agony over this bug if they cannot upgrade to Kibana 5 yet (like I can't)

/** Fixes up the Dashboard so that the ordering is actually the way it should be (so the dashboard actually renders correctly).
 * There is a bug in Kibana that when you save the Dashboard after moving and resizing visualizations, when you load the dashboard, things are all out of place.
 * This bug: https://github.com/elastic/kibana/issues/2138
 * This function implements the workaround stated in comment: https://github.com/elastic/kibana/issues/2138#issuecomment-303046596
 * This basically takes the `panelsJSON` and reorders the array sorting by row then by column. And then it renumbers the `panelsIndex` for each visualization by this ordering starting from 1.
 * After doing this, the dashboard shows things in the right order and sizing.
 *
 * @param dashboardJsonArrayStr A JSON String representing an array whose single object is the Kibana Dashboard config JSON object (i.e. which has a top-level property of `_source`)
 * @return A new JSON Array string representing the fixed up dashboard.
 */
def fixKibanaDashboardOrdering(dashboardJsonArrayStr) {
    def panelsByRowAndColumn = new TreeMap<Integer, SortedMap<Integer, Object>>()
    def jsonSlurper = new JsonSlurper()
    def dashboard = jsonSlurper.parseText(dashboardJsonArrayStr)

    // Extract the embedded JSON in the panelsJSON field and unescape it.
    def panelsJsonStr = StringEscapeUtils.unescapeJavaScript(dashboard[0]._source.panelsJSON)
    // Parse the panelsJSON
    def panelsJson = jsonSlurper.parseText(panelsJsonStr)

    // Sort the panelsJSON by row then by column and renumber the panelIndex starting at 1
    panelsJson.each { panelObj ->
        def row = panelObj.row
        def col = panelObj.col
        def mapForRow = panelsByRowAndColumn.get(row)
        if(mapForRow == null) {
            mapForRow = new TreeMap<Integer, Object>()
            panelsByRowAndColumn.put(row, mapForRow)
        }
        mapForRow.put(col, panelObj)
    }
    def sortedPanels = []
    def panelIndex = 1
    panelsByRowAndColumn.each { row, rowMap ->
        rowMap.each { col, panelObj ->
            panelObj.panelIndex = panelIndex++
            sortedPanels.add(panelObj)
        }
    }

    // Serialize the sortedPanels back to JSON
    def sortedPanelsJsonStr = JsonOutput.prettyPrint(JsonOutput.toJson(sortedPanels))

    // Attach the sortedPanels JSON String back to the dashboard JSON object
    dashboard[0]._source.panelsJSON = sortedPanelsJsonStr
    // Serialize it back out to JSON
    def fixedDashboardJsonStr = JsonOutput.prettyPrint(JsonOutput.toJson(dashboard))

    return fixedDashboardJsonStr
}

[julienbeclin]

Hello @jfiore-ibm,

This would be really useful to us as we did not migrate to ES 5 yet. Where should we place this code in order for it to work ?
Thank you