Godot: Sorting of tiles with new TileSet editor is unpredictable when using atlas

[Krumelur]

Godot version:
3.1 Alpha 5

OS/device including version:
mac OS High Sierra

Issue description:
Sorting of tiles with new TileSet editor is unpredictable when using atlas.

Steps to reproduce:

• Add a TileMap
• Set a TileSet and edit
• Add a texture with all the tiles
• Create a "New Atlas"
• Select all tiles
• Return to TileMap

-> Order of tiles seems to be completely randomised. Tiles should be processed left to right and top to bottom or even better, ordering should be configurable.

Quick video showing the problem: https://www.youtube.com/watch?v=OMxrGUmZgTk

Minimal reproduction project:
https://github.com/Krumelur/GodotTesting/tree/godot31_issue_24751
Open the project and go to res://Helpers/LevelConverter/Importer.tscn and select the "TemplateTileNode" node in there.

[Krumelur]

Just noticed that this might be a duplicate of https://github.com/godotengine/godot/issues/24517

[Calinou]

@Krumelur I think I'll keep this issue open as it clearly outlines a bug.

[vortexofdoom]

I think what it's doing is prioritizing columns first, and displaying them as rows. I looked at my atlas and the first row, at size x, was the first tile of the first x columns in my texture. So when I adjusted the view so that the row was as long as the number of columns, suddenly I was seeing full rows as columns and vice versa. Still far from ideal from a usability standpoint, but I don't think it's random.

I'm pretty sure that the original video shows the same behavior.

Seems like some for loops probably got flipped around somewhere, if I had to guess lol.

[rzllmr]

You guessed right. The coordinates given in Vector2 variables throughout the TileSet class are associated to column x and row y. That way the sorting algorithms of the Vector and Map class as used for filling the TileMap selection window are prioritizing the columns x. But filling it rows-first then swaps the order and makes it hard to use.

I got a fix for this symptom by using a custom sorting in editor/plugins/tile_map_editor_plugin.cpp line 524 (of the latest revision) or line 486 (of release 3.1.1-stable):

// replace
entries2.sort();
// by
struct SwapComparator {
    _FORCE_INLINE_ bool operator()(const Vector2 &p_l, const Vector2 &p_r) const {
        return p_l.y != p_r.y ? p_l.y < p_r.y : p_l.x < p_r.x;
    }
};
entries2.sort_custom<SwapComparator>();

This works as a quick workaround for those using the source version of the engine. Until there is someone more familiar with the plugin to tell whether the row-column-swap could be fixed in the tileset itself without confusing other systems.

[vortexofdoom]

Column x, row y actually makes some sense if you look at it from the perspective of the column and row numbers representing position on the x and y axes, it's just that it doesn't draw them the way it reads them, I think.

[rzllmr]

Right. The order in the TileSet makes sense then. And as the tiles fetched from it are filled into the widely used ItemList control that draws rows-first, the suggested fix may be the best place to intervene. I could try a PR then.