Crystal: [RFC] per-platform low-level abstractions

[ysbaddaden]

I propose to introduce a low-level platform abstraction for features that require platform-specific bindings or implementations. These abstractions would be grouped under a Sys namespace in the core library, and be private to the core/stdlib (the API can be changed at any time). This RFC is influenced by the sys namespace in Rust.

These abstractions are currently either missing (i.e. tied to POSIX) or implemented using compile-time flags and inlined in the source code, which already proves hard to read. See for example callstack.cr where it's quickly hard to know for sure which platform is targeted by the code currently visible on screen; adding support for Windows would only make things badder.

Some use cases:

• raising exceptions, building a stack trace for exceptions (platform specific, sometimes arch-specific);
• getting the current time, changing or removing TZ+DST offsets;
• getting random numbers (/dev/null, arc4random, ...);
• asynchronous File and IO read/writes;
• and many more topics.

Formally, I propose to introduce a src/sys entry with platform specific subfolders, and a generic file that would require the correct implementation based on compile flags. For example a Sys::Random module could be defined as:

• sys/unix/random.cr (UNIX generic: read from /dev/null /dev/urandom)
• sys/linux/random.cr (use getrandom on Linux if available, otherwise fallback to /dev/urandom)
• sys/openbsd/random.cr (use arc4random on OpenBSD)
• sys/windows/random.cr
• with sys/random.cr implemented as follow:

{% if flag?(:linux) %}
  require "./linux/random"
{% elsif flag?(:openbsd) %}
  require "./openbsd/random"
{% elsif flag?(:windows) %}
  require "./windows/random"
{% else %}
  require "./unix/random"
{% end %}

Porting Crystal to support more platforms could be achieved by adding any required C bindings to src/lib_c then implement the API for the platform under src/sys —in the hopeful case that it won't require changes in the Sys API.

This RFC may not solve everything, some topics may be harder (e.g. threads, fiber context switches), but I'm confident it would help simplify and cleanup the core/stdlib source code, and ease the review and merge of per-platform features (i.e. incremental Windows support).

[jhass]

I'm all in, in fact I proposed something similar in the past

[bew]

@ysbaddaden for unix generic method to get some random, I think you mean /dev/urandom instead of /dev/null

[ysbaddaden]

@bcardiff proposed Platform as an alternative name, and it seems to gain some traction.

My only concern is that I can foresee myself defining a Platform type someday, whereas I don't foresee using a Sys namespace. But no strong feelings. If we prefer Platform, then long lived Platform!

[luislavena]

I've been a bit split about the usage of Sys, System and Platform being used by default by any Crystal project. Been thinking more and more that Crystal runtime elements might deserve their own namespace (like RT for Runtime or similar).

I see that extend to things like the Scheduler and other internal elements of Crystal that don't need to be exposed directly to the global namespace.

Sorry, couldn't resit the bike shedding :stuck_out_tongue_winking_eye:

[ysbaddaden]

No, you're perfectly right Luis. The crystal runtime (e.g. Scheduler, ...) should live under the Crystal namespace, even maybe Crystal:: Runtime. It was proposed and accepted under a POSIX or libc related pull request. It's a good idea.

We could introduce Crystal::Platform or Crystal::System (we are making system calls after all) in order to boostrap this change?

[mverzilli]

It looks like we settled on Crystal::System. Can we consider this done and close it? Also, maybe we should distill the conclusions of this in a wiki article for future reference for contributors.