Brew: Rewrite Homebrew in Crystal?

[Jens0512]

NB: Although you very likely to find this suggestion outrageous, please bear with me, and consider it seriously. Please do not instantly close this issue, on behalf of the community, let it be open for people to share their thoughts on this.

We will close this issue or ask you to create a pull-request if it's something the maintainers are not actively planning to work on.

Surely, improving Homebrew's speed is something planned, or at least hoped for. And multiplying it tenfolds of times should be strived for.

Motivation

Edit: This first statement is mostly wrong apparently
Homebrew is slow, that is because Ruby is slow, there are no arguments against this.

You might have heard about Crystal, its syntax is heavily inspired by ruby. Ruby's syntax is slick, and so is Crystals'. Crystal goes by the catchphrase

Fast as C, slick as Ruby

If you are skeptical, know that it is very understandable, unless you are familiar with crystal of course, in which you certainly would find the results tempting.

Relevance

This would greatly boost the Homebrews performance. This repository's issue-guidelines says

Explain how the feature would be relevant to at least 90% of Homebrew users.

This would be relevant to 100% of Homebrew users, no doubt about it.

If you are now thinking: I am not going to learn a new language for this! Do not do so. Because if you're reading this, chances are you'll know Ruby; read through this little wiki-page; Crystal for Rubyists, and you'll be pretty much ready to go.

Tl;Dr

• Rewrite Homebrew in the language Crystal, which has syntax almost identical, and at least a 100 times faster.
• Do so in a way, that keeps the current Formulas working.

What exactly

These are my points positive of crystal:

1. It is at least a 100 times faster than Ruby (Take a look at the benchmarks).
2. The syntax is extremely similar to ruby - Therefore the front-end Homebrew (Formulas) will require no alterations.
3. Most Ruby code is valid Crystal code, the transition from Ruby to Crystal will be mainly small changes.

Against this:

1. Code can not be dynamically loaded in Crystal. Hence the formulas will be a bit of trouble.
2. This will require some work.
3. Unlike Ruby, Crystal is not pre-installed on macOS

Do you want to get started, but don't know what to do? Or are you just very doubtful about this? First take a look at Crystal and Crystal for Rubyists

[bararchy]

Sounds like an interesting proposal, does MacOS always comes with Ruby installed at all versions ?

[RX14]

This sounds like a better idea to propose as a parallel project instead of as a rewrite under the same name. I doubt any of the current homebrew maintainers know or would be willing to put in the effort to learn Crystal.

Personally i'd benchmark how much time that homebrew actually spends in ruby code before making a proposal like this. I think you'd be surprised. Any performance benefits are more likely to come from rearchitecturing, not the new language itself.

[vitorgalvao]

Explain how the feature would be relevant to at least 90% of Homebrew users.

This would be relevant to 100% of Homebrew users, no doubt about it.

[emphasis added]

Do doubt. I use Homebrew and this would be irrelevant to me. I also don’t see slowness as a common complaint. The only noticeably slow parts in Homebrew are the parts out of its control, such as downloading packages.

Unlike Ruby, Crystal is not pre-installed on macOS

Which is huge.

This sounds like a better idea to propose as a parallel project instead of as a rewrite under the same name.

Agreed.

Homebrew isn’t done. I see only disadvantages in having the core team spend time porting the large codebase to a different language, rather than continue to improve the current one.

I’d see it more of a challenge to a third-party. If it’s true the languages are so similar and little code changes would be required to reap the alleged monumental benefits, then it should be easy for a third-party already familiar with the language to build the proof.

does MacOS always comes with Ruby installed at all versions ?

Every macOS versions comes with a specific ruby version, that is higher than the one bundled with the previous version.

[RX14]

To be fair to the proposal, there's no need to have crystal be installed because it compiles to native executables. However, you'd have to build that executable from scratch every time you changed a formula because the formulas would be written in crystal, which in practical terms, means crystal installed on every machine (If I started a homebrew-alike in crystal I'd have the formulas be in shell script).

The only noticeably slow parts in Homebrew are the parts out of its control, such as downloading packages.

That's what I would suspect too: ruby vs crystal makes little difference since most of the time would be IO.

[woodruffw]

Homebrew is slow, that is because Ruby is slow, there are no arguments against this.

Homebrew's performance is dominated by networking and housekeeping operations, not the Ruby runtime. Fetching large archives over the network and exploding them is a fundamental constraint on every package manager's performance. Ruby is slow, but Homebrew isn't slow because of it.

I like Crystal a lot (I've made a few contributions to the main language repository, and written some libraries in it), but I don't think this is going to happen for two different reasons (off the top of my head, there are probably others):

• Homebrew's codebase is large, and makes liberal use of Ruby's metaprogramming and DSL facilities. Crystal's macros are great, but would probably require a fundamental redesign of the Formula DSL.

• Crystal is still in an alpha stage of development, with frequent standard library and compiler changes. Maintaining Homebrew is already a huge task, and mixing that with a steady stream of language changes would require us to take time away from feature development and package maintenance.

[Jens0512]

Ok. I realize I am wrong about this.

Homebrew is slow, that is because Ruby is slow.

[asterite]

• Crystal is not production ready (and who knows what will happen with Crystal in the future, but Ruby is here and it's production ready).
• When a formula changes you'd need to recompile that formula, or the entire homebrew. Either will take much more time than Ruby.
• Homebrew is not slow. It takes about 0.6s on my machine to start up, but then most of the time will be spent in IO and disk, not CPU computations.

[MikeMcQuaid]

The only noticeably slow parts in Homebrew are the parts out of its control, such as downloading packages.

ruby vs crystal makes little difference since most of the time would be IO.

Homebrew's performance is dominated by networking and housekeeping operations, not the Ruby runtime. Fetching large archives over the network and exploding them is a fundamental constraint on every package manager's performance.

Homebrew is not slow. It takes about 0.6s on my machine to start up, but then most of the time will be spent in IO and disk, not CPU computations.

Thanks for the write up and cool idea but passing on this due to the above. Sorry!

[Jens0512]

Don't be sorry, I was fully aware that my suggestion was crazy, but I wanted to give it a go.

[reitermarkus]

Rust-rewrite, anyone? 😛

[RX14]

Haha, I hope the crystal community doesn't get that reputation!

[Jens0512]

Sorry 😅