Crystal: [RFC]Qt like File and Path interface for embed resources

[babyface0-0]

crystal is good for single binary deployment, but resources such as static assets, locales, database migrations must copy with folder, some shards try to solve this by bake file system, but lack common interface, it 's hard to do integration.

can we implement a qt like resource interface :/path/to/file
then someone can choose backend, such as Base64, ByteArray, or tar and zip archive.

then this interface can used by all File, Path, Dir.

eg:

File.virtual_filesystem = Base64FileSystem.new()
  .add("../public")
  .add("../locales")
  .add("../db")

I18n.load_paths << ":/locales"

Migration.magrations_path = ":/db/migrations"

StaticHandler.new(":/public")

[jhass]

There are several shards providing a virtual filesystem. The more popular ones I know are https://github.com/schovi/baked_file_system and https://github.com/busyloop/rucksack.

There's existing related talk at #1649, #2751, #2791, #2886, #5792, #6967.

I think we can close this as a duplicate for #1649 and focus discussion there.

[straight-shoota]

Embedding resources is one problem, but how to make them accessible is a different thing. So maybe this could be a separate discussion about integration of virtual filesystems.
This is maybe not super important for stdlib. I think the only component (except for the direct file APIs) that works with file system features is HTTP::StaticFileHandler.

[jhass]

Well, then the first question is whether stdlib should provide this at all. My personal answer would be no. Crystal should provide a primitive for baking binary data into the binary efficiently, but I think the ideas of how an API ontop of this should look like are too conflicting too easily.

So I'd love somebody replacing read_file with something incbin based and making it return a reference to a Bytes instead of a StringLiteral. That should provide enough of a primitive for shards to build nice abstractions on top. Given Crystal's monkey patchability, a shard could even provide an API as suggested in the OP. But I don't see this necessary in stdlib, really.

[straight-shoota]

I agree that this doesn't really fit into stdlib. But maybe stdlib could help providing a more pluggable API.
The problem I see with the approach to just prepend paths with a colon is that it only works with a single virtual file system. If an application uses a second virtual file system (for example mounting remote resources), that's already out of limit.
So my idea would be to consider having an abstract API for generic file system access. Stdlib would provide an implementation for the local file system. Other shards could provide implementations for baked files, remote filesystems, and layering them.
The abstraction could also be defined by a shard and the local file system implementation build upon the stdlib features. But having the abstraction in stdlib would ensure compatibility between shards. Otherwise, some shards working with file sytems would only target the stdlib API and it would be hard to make it pluggable.
This is just an undeveloped idea, but it might be worth giving it some though.

[babyface0-0]

yes we need abstract API !!!