Godot: Add warning/docs for .res/.tres for binary files

[WizzardMaker]

We use the PckPacker in our project to add image resources at runtime, but they waste a lot of space when not compressed.

There should be a way to compress the .pck files made with PckPacker, like the .pck files made with the editor.

[Calinou]

There should be a way to compress the .pck files made with PckPacker, like the .pck files made with the editor.

Are PCK files created with the editor compressed? As far as I know, they're not compressed because this makes random seeking difficult. (Try comparing the file size when exporting a project to a PCK file and a ZIP archive.)

[WizzardMaker]

These are the sizes I get for my project

Pckpacker.zip - 885,883KiB
PckPacker.pck - 885,883KiB

Export.zip - 59,834KiB
Export.pck - 885,883KiB

[avencherus]

As far I understand it, you will want to do the compression on your image files, and not the PCK.

Game archive files aren't used for compression, they're used to keep files in the same disk space and eliminate or control file fragmentation. This allows reads to be faster since the disk doesn't have to jump around to far reaching sectors as much. (Probably less of an issue with SSDs)

Also, added compression in itself is an increase of load time to the players, because on top of the time of retrieving data off disk, it will have to decompress it as a separate operation every time something needs to be accessed.

I'm not sure about the target methods, but I imagine, as Calinou said, you couldn't just randomly pick out the file you wanted from the archive, the whole thing would have to be decompressed for each access. Then the unused files (and the work to get them) would get tossed out.

[WizzardMaker]

This all makes sense, but the image files are already as compressed as they can be. The problem is, I have around 900 images, and each and every one is saved in the pck with the tres format, which adds a rather big overhead to the file size with its redundant file header.

Modern games also have all their games compressed and we load most of the data at launch into memory. You have a point with the compression, but I can't force my players to have a 800+ MB file, when it can easily be a 40MB file.

The other godot game files, which are not packed by me, seem to also get compressed when exporting to, say an exe.

I think there should be an option for the compression in the Pckpacker, as its already implemented in Godot. Whether it's useful or not should be up to the developer in the end, shouldn't it be?

[akien-mga]

Are you saving image data as TRES (base64 text, huge) instead of PNG (compressed, binary format, small size)?

[WizzardMaker]

We are forced to use the TRES format.

We are making an open source remake of a game and because of legal reasons have to remove all original data before publishing. We import all images of that game into godot as TRES files, delete the "data" content before export and refill them with a .pck made at the first launch of the exported game. We have to use TRES, because godot forces all other formats to be moved to the .import folder and we don't see any other way to modify those .import files.

[Calinou]

@WizzardMaker Couldn't you use the binary .res format in this case? I'm not sure if you could easily save and load a compressed version though.

[WizzardMaker]

Well, saving all files as *.res does help... I shrinks everything to the size of the ziped export. Thank you for that

But it would have been nice to have known about that. I did some research before implementing all that and I didn't find any concrete mention of the differences between .res and .tres in the context of saving resources.

[Calinou]

@WizzardMaker I wonder if we could throw a warning whenever a (large enough) binary resource is being saved as Base64 in a .tscn or .tres file. This could also be displayed in the editor, as it can harm the user experience due to long saving/loading times.

[WizzardMaker]

Sounds like a good idea, we should also suggest in the warning to use the non base64 .t* version.

There should also be a mention of the difference between those two variants of saving (.tres/.res) in the documentation.