Open-ps2-loader: Support for cso/gzip compressed ISOs

[eadmaster]

CSO (aka CISO) can be converted with maxcso.

They could have 2 advantages:

• save space on USB and HDD storage devices;
• speedup loading from slow storage devices like USB, since less data needs to be transferred.

[TnA-Plastic]

There would be a disadvantage however...

The support would need code to run on the IOP (to decompress it) and the filesystem-support in-game on USB was already stripped (hence defragmenting ISOs and VMCs), due to it needing too much code and causing incompatibilities with a lot of games.

I would like to have support for it as well tho'.

[AKuHAK]

http://psx-scene.com/forums/f19/%5Bhelp%5D-loader-compressed-isos-103805/

[rickgaiser]

I really like this idea and have already been playing with lz4 since the decompression requires so little resources. I didn't know about the ZSO and CSOv2 formats that also use lz4 (and deflate) for compression.
Perhaps I'll do a quick experiment to see what's possible with the iop's low speed and lz4...

[eadmaster]

The reason why i suggested CSO it's because it is supported by PCSX2 as well, so we could have a good archival format for both real hardware and emulation. Gzipped isos would be another option.

[TnA-Plastic]

SMB could also benefit a bit, since it wouldn't be un-/decompressed on the host-side (given someone uses something like NTFS-Compression) [before the transfer], thus the transfered game-data is compressed there as well, unlike a filesystem-based compression on the host.

It depends if it would dive bandwidth more down than it can yield in compression, or not.

[AKuHAK]

I really like this idea and have already been playing with lz4 since the decompression requires so little resources. I didn't know about the ZSO and CSOv2 formats that also use lz4 (and deflate) for compression.
Perhaps I'll do a quick experiment to see what's possible with the iop's low speed and lz4...

According to this lz4 (default compression level) decompression is only 3 times slower than simple memcpy so maybe PS2 will be possible to handle this.
Some tips:

• it will be possible to compress most of the titles less then FAT32 limit, so no need to split image for USB storage
• @rickgaiser if you do testing maybe it will be useful to not compress media files. CISO plus has the feature to not compress multimedia files: Leave PMF and AT3 files alone.

[TnA-Plastic]

Well, the PS2 uses AT3 sometimes I suppose, but others like PSS are more common!

It would be neat to selectively choose the files which should not be compressed and even more so would 'profiles' for games (which files to compress and which shouldn't be compressed, if it shouldn't compress the entire ISO) be a cool thing, IMO.

[sp193]

Long ago, PS2ESDL used to support compressed disc images. Not CSO, but its own format. It used LZO and compressed only sectors that could be sufficiently compressed.
I eventually removed it because it seemed like nobody wanted to use a different format.

Speed was never really improved. Sometimes it did, but it almost never helps FMVs (a benchmark by some people) and so on because videos are usually compressed (and compressing compressed data is usually a bad idea). So I think we should not consider a speed improvement as a condition for considering whether to add this feature.

[AKuHAK]

Speed was never really improved. Sometimes it did, but it almost never helps FMVs (a benchmark by some people) and so on because videos are usually compressed (and compressing compressed data is usually a bad idea).

@sp193 Yeah, it is why I mention the ability to skip movies. As I know CSO also has the ability to skip sectors that are not compressable (for example if the compression ratio is less then 95%).
I don't think that this will improve speed, but it can save a lot of space (maybe about 20% can be saved even with the lowest/fastest compression ratio).

This can be very useful for internal HDDs cause they are very fast but sometimes has not so much free space.

[eadmaster]

I expect this to speedup loading times of vector and textures data of the disk (if they are not already compressed).

Also i could skip removing zero-filled/padding files from the Isos, because they are easily compressed.

[gingerbeardman]

This is really exciting!