Dynmap: Not rendering pre 1.14 map tiles

[Mecroscope]

We've been running dynmap for a while now, but at the moment can't fully render the explored world (which is getting reasonably large), as older tiles cannot be forced to render. The map is easily pre 1.12 (it's a number of years old, so probably pre 1.09 or more), though we had dynmap rendering until relatively recently (1.12 maybe, or possibly 1.13). No amount of purging, full renders, or updating the dynmap client seems to help (currently running dynmap-3.0-snapshot-spigot.jar, spigot-1.14.1.jar - both 1.14.2 spigot or craftbukkit haven't proven stable for us so far).
We really haven't modified the server config (at least recently), so normal dynmap defaults.

The only thing that has worked is the chunksrefresher discussed in a previous post below:
https://github.com/webbukkit/dynmap/issues/2359
The catch is that runs out of memory after about 8-10 512x512 block refreshes, at which point I restart the server, and start over. At this rate it will take weeks or months to refresh the server as there are a LOT of tiles to go through. Is there any way of incorporating a more memory friendly version of this approach?

An example of the map is below (partially completed process above):

[korhox]

Dynmap is not generating the world, only reading.

Try installing WorldBorder plugin, set world radius and then using fill command pregenerate the world. After that you can do dynmap fullrender.

Command order:
/wb set /wb fill
/wb fill confirm
When wb fill has completed you can use /dynmap fullrender

Edit:
Sorry if i'm wrong, but i think it seems like this is the issue.

[Mecroscope]

Thanks for the suggestion, but Worldborder isn't really either an option or the solution.

We did look at it previously but the issue isn't that we need to generate the chunks, we've done that for years by walking on them. It's convincing dynmap that they exist. The world is also highly non-circular so while I could possibly force fill/render the central areas, it wouldn't help with anything else, and we have tendrils going out to 30-50000 in various directions.

[korhox]

Oh, that's the case. Sorry for my misunderstanding.

I don't know is this going to help with your issue at all, but how about uncommenting render trigger playermove in render triggers?

render-triggers:
  - playermove
  #- playerjoin
  - blockplaced
  - blockbreak
  - leavesdecay
  - blockburn
  - chunkgenerated
  - blockformed
  - blockfaded
  - blockspread
  - pistonmoved
  - explosion
  #- blockfromto
  #- blockphysics
  - structuregrow
  - blockgrow
  #- blockredstone

[mikeprimm]

At this point, I've given up on using the Spigot chunk loading APIs - couldn't get to work without leaking loads of memory and otherwise just not functioning properly (e.g. they always cause chunk generation that leads to unbounded map growth). The current Dynmap builds load chunk data directly, which means that we're not triggering chunk migration logic (which is tied to the chunk generation logic....). For 1.13, I'd put in a hack to drive chunk migration (since that had similar problems with pre-1.13 chunks not being migrated when you do a chunk load-without-generate), but the chunk loading calls didn't have the memory leak issues I've seen with 1.14, so that was more practical.

I'll give some thought to seeing if I can come up with a way to force the migration safely in 1.14 - I can tell that the chunk data I'm loading is pre-1.13 easily enough: it's just a matter of driving the chunk migration logic safely, and not having the server memory bleed to death in the mean time.

[Mecroscope]

Thanks Mike, I rather guessed as much based on your previous comments about spigot loading, but thanks for looking at it. I'm manually driving the migration now using that tool above, it's just spectacularly time consuming so anything would help!

[mikeprimm]

OK - I've got something VERY experimental in the latest dev build (212 or later). I've done a quick set of testing on it on a not particularly big map, but it seems to be working. It will, when enabled, attempt to drive chunk migration when pre-1.13 chunks are found. I don't suggest trying it without backing up your world data first, just in case....

To enable it, add 'migrate-chunks: true' to configuration.txt on the new 212+ build (it's off by default). If it works out, I suggest doing a 'dynmap fullrender world:flat' to prime the migrated world, and then turn the feature off once the render has driven the migration - it's really not an ideal thing to leave on full time. Lates dev build is always at https://dynmap.us/builds/dynmap/Dynmap-3.0-SNAPSHOT-spigot.jar

[Mecroscope]

Thanks Mike. Running it now, I backed up the existing dynmap folder and full world first (which was what took so long to get started as it's about 30 gigs worth of tiles!), I'll let you know what happens. Initial indications are promising as the local tiles are starting to fill in correctly now. Whether the memory holds up? We'll see.

[Mecroscope]

Ok, interestingly enough, it didn't fully fix all the local tiles, AND it actually erased all the tiles that I'd updated using the chunk refresher. It didn't however run out of memory or anything. See below for example (only a small section of the map obviously). I'll leave it alone for a moment as I do still have a full backup of the previous dynmap/server I can swap back to, if you have anything else you'd like me to try.

[iridinite]

Having the same issues, so I'll subscribe to this thread. I'm running 3.0-beta4, on the latest available Spigot dev build. The worlds I'm using are very, very old, some of them are from before Minecraft 1.0 was a thing.

I tried setting migrate-chunks to true, however it doesn't really appear to do anything. When running dynmap fullrender world:flat it still only renders the area around the spawn (which I briefly visited after upgrading to 1.14.2)

Worse, it apparently purged the rest of the map. In this screenshot, only a tiny square in the middle is rendered, but almost everything up until the red border used to be rendered.

[iridinite]

Small update: I ran WorldBorder's wb fill on the primary world, with the force option set to true. I don't know if that's actually necessary, but it seemed like a good idea to force it to load all chunks. As it appears to me that just having the chunk load and re-saved is sufficient to have it converted.

It appears to work. I'm now watching dynmap do a full render of the flat map, and I think it's not stopping anymore just a few chunks off the spawn like in the above image, as it's going on for much longer. Will edit this post once I know for certain. EDIT: This confirmed works for my use case.

However of course this isn't an option for worlds that don't make use of the worldborder plugin.

[razaqq]

Most the map rendered fine using this, however it left some weird artifacts on the borders.

I manually teleported there and did a radiusrender to fix it for now

[aaronfranke]

I have the same problem, but /dynmap radiusrender fixed it. I just had to use a really big radius number that covers the map.

[FedUpWith-Tech]

Duplicate of #2282