Straight.el: Add comparison to nix

[raxod502]

See https://github.com/raxod502/straight.el/issues/95#issuecomment-317199087.

[raxod502]

@vyp

Can you also add a comparison to nix?

Sure. It's a little odd since nix is a system package manager, but I can see why the comparison would be useful, since straight.el also claims to be a functional package manager. Do you know of any resources that would be useful for me to get an understanding of the nix internals, so that I can write a proper comparison? From what I can tell it's a pretty complicated piece of software.

[vyp]

Ah haha ! So I had assumed that with "functional package manager" in the description, plus with guiding principles of "no persistent state kept elsewhere" and "100% reproducible package management", you would have had experience with nix! Assumption there on my part so my bad.

First of all, documentation of emacs package management for nix is close to non-existent, or at least I haven't found any. So, without actually trying nix out yourself or something, it'd be pretty hard to make a comparison, I think. So I'm not asking you to learn about and use something if you don't want to or don't have the time. (And feel free to close this issue too then!)

I would help with it, except I haven't really used nix extensively for emacs packages, so I still have to learn a lot about it myself, and see if it fulfills the features I want (with regards to emacs packages). At the moment I'm still mainly using quelpa.

But given straight.el, I think you hopefully might at least be interested in it, so I'm going to expand a little bit into the rabbit hole! :smiley: But before you read anymore, nix doesn't currently work on Windows. (There's no technical limitation, it's just low priority. It might work with WSL or cygwin or something, I have no clue.) But looking at your dotfiles I think you're on a mac. :wink:

---

There's a few terms which might be confusing:

• Nix
  
  
  
  • The package manager itself.
  
  
• NixOS
  
  
  
  • A GNU/Linux distribution that uses Nix the package manager.
  
  
• Nix Expression Language
  
  
  
  • The (domain specific) programming language that Nix uses to describe packages.
  
  
  • Sometimes called "nixlang" or confusingly even just "nix".
  
  
• "nixpkgs"
  
  
  
  • The GitHub repository which contains the 'Nix Expressions' for basically all packages that Nix knows about. (Of course you can also use your own modified or completely private packages provided you write a 'Nix expression' to describe it.)
  
  

It does have it's issues, but all things considered, Nix is pretty wonderful compared to basically all other package management systems, in my opinion. You can read about more about from the links above, but in short, it's functional package management model allows for some pretty neat features. "Functional" is sort of borrowed from the concept of functional programming languages (no state, same inputs result in same output etc). Some of them include:

• Atomic upgrades and rollbacks.
• Installing multiple versions of the same package.
• Reproducibility.
  
  
  
  • Via declarative configuration with nix expression files.
  
  
  • Similar to your "init file as sole source of truth" guideline.
  
  
• Reliability.
  
  
  
  • The functional model of having no state means that, for example, if the power goes out whilst you're upgrading packages, when you reboot again your Nix packages will still be as they were, and not "borked" or something.
  
  

I love having all these and now that I've been using nix for a few years, having anything less "feels" 'impure' haha. (So basically I've become a smug functional programmer almost. :stuck_out_tongue_closed_eyes:)

So if you're actually interested in trying it out now, I suggest starting with the Nix manual. You can use Nix on GNU/Linux distributions or on macOS (or you can use NixOS!). Packages are stored under /nix/store so installing/uninstalling nix and nix-managed packages doesn't affect the rest of the system. (So uninstalling Nix and all of it's packages is as simple as rm -rf /nix.) IRC channel is on #nixos on freenode. BTW, there's separate manuals for Nix, NixOS, nixpkgs etc! (So make sure that you're reading the right manual if you start looking into it more cause they all look similar and can be confusing.)

Having it's functional "stateless" nature, Nix doesn't conform to the FHS (as I said, it uses /nix), so there is some maintenance overhead when installing and maintaining packages because a lot of packages assume that user's systems do conform to it. However, Nix now actually automates a lot of that overhead away anyway for a lot of things, so it's not that bad. (It really depends on the package.) And honestly the FHS is outdated anyway. (Maybe this is sort of like how straight.el doesn't associate with package.el haha, because it gets in the way with the goals of straight.el.)

Despite it's conception back in 2003, I think Nix only really got more popular about a decade or so later. So in my opinion Nix is like a breath of fresh air compared to everything else, I'm not aware of anything else that compares to it* (including docker, snappy, flatpak, appimage etc). There was only one other package manager which I remember being fairly sophisticated and had similar features to Nix, but I don't remember what it's name was. :/

---

So with that being said, the place to look for documentation about emacs package management with nix would be https://nixos.org/nixpkgs/manual/#chap-language-support (but as I said it isn't there yet, or I may be blind). There is some rudimentary emacs related documentation at https://nixos.org/nixos/manual/index.html#module-services-emacs, but note that it's part of the NixOS documentation so a lot of it might be specific to NixOS users only.

At the moment, the level of documentation and support for specific-language package management is dependent on the userbase of Nix and how many people use that language/tool. For example, the haskell nix infrastructure is quite sophisticated because haskell didn't really have anything (until stack), so a lot of haskell users and developers started using it. However OTOH for example, the rust language support isn't very "Nix-y" last I checked. :(

---

I would be willing to help with this issue if it's something you're interested in.

One of the main reasons I like quelpa is because it allows one to locally build whatever version of a package I want to even if it's not up on melpa yet. So for example you as a user can test a patch that a developer has made quickly without having to wait on melpa or anything. (Of course you there's other ways and you don't need quelpa, but quelpa's way is a lot easier for reproduciblity in my opinion.)

One pain point (I think) with using Nix for emacs packages could be that it needs the sha256 of the package source in it's nix expression. So there might be some overhead with keeping emacs packages up to date compared to say using something like quelpa or (I presume) straight.el. (However, people have made tools to help with some sort of automation of this.) Also the current lack of windows support could be another disadvantage.

An advantage of course is that nix can reliably install system dependencies if an emacs package requires it. For example pdf-tools, haskell-mode etc. This probably isn't a big deal with emacs packages anyway because if an emacs package does require some other system library or package, it's common for an emacs package to test for it's existence before attempting to use it. But still, in the interest of total reproducibility, it is a nice advantage to have in my opinion.

I mean maybe there could be the best of both worlds where one uses some sort of combination of the two (straight.el + nix). I've heard of haskell people using both stack and nix together (there's actually a --nix flag or something in stack now for nix integration). So maybe it doesn't have to be just one or the other, and it could be useful to straight.el. Or maybe that would overcomplicate things and be undesirable, I don't know yet! :sweat_smile:

In any case, let me know if you try out Nix, and what you thought of it if you do! I'm happy to try to answer other questions as well. (And if it's not relevant to this issue I'm on twitter as well, see profile.) Just a note, unfortunately these days I just don't have the time to hack on things that much, so I'm not sure how much more insight about nix + emacs I can provide really. :/

---

*You should know there's also Guix, which is basically Nix in Scheme (GNU Guile) instead of the nix expression language, which I think is a lot better because it de-duplicates a lot of logic and tooling across all the different language platforms, among other reasons. (Nixlang isn't as capable because remember it's domain specific, whereas scheme is of course general purpose.) So it has it's own set of packages described in scheme instead of with nix expressions. The creator was a core developer of Nix I believe.

But it's only for GNU/Linux (or eventually other GNU systems). It also has a lot less manpower than Nix. However, being scheme, it has a decent amount of emacs users! (It's creator being one of them!) So the point being you can also use Guix to manage emacs packages much the same way as you would use Nix, if you're on a GNU system, so maybe there could be a comparison with Guix too!. :stuck_out_tongue: BTW, GNU Guile has an elisp interpreter! (You may have heard of guile-emacs?) I'm not sure if Guix takes advantage of it though.

[raxod502]

Wow this is quite an impressive summary! Since Nix can be used on macOS without having to be using NixOS, I'll definitely want to try it out at some point. As you can see I'd like to get around to it before the 1.0 release. Thanks so much!

If you like Quelpa, I really think you will like straight.el since IMO it accomplishes the same thing but with fewer layers of indirection and with more reliability, while being more flexible.

From what I can tell, the place where straight.el wins against Nix (and all other package managers that I know of) is how focused it is on local modifications. It certainly doesn't require a sha256 hash of the upstream, because such a concept doesn't even make sense in the face of allowing arbitrary local changes! (OTOH if you are really a reproducibility fanatic and are willing to sacrifice some flexibility to gain 100% assurance that it's impossible to break your config, then this would be a disadvantage of straight.el, and an advantage of Nix. It's all a matter of perspective.)

The point about system dependencies, though, is really powerful and definitely something that is out of scope for straight.el. So you may be right that using them together could be possible and/or useful. (I have my own plans for system package management…)

Anyway—long story short, I'll be checking out Nix at some point. Thanks again.

[vyp]

From what I can tell, the place where straight.el wins against Nix (and all other package managers that I know of) is how focused it is on local modifications.

This sounds promising! And yes it's also what I was trying to get at a little bit too. I've always wanted some sort of way to easily reliably reproduce all my elisp (whether it be other packages, my own modifications, a combination of the two, etc). The default package.el + elpa/melpa method meant that for third party packages you're basically limited to downloading packages, instead of being able to point package.el to a local package. (package.el has an install-from-file command but nothing for multi-file packages so it's lacking.) Not only would that be good for users, but also for maintaining and developing your own packages (so developors). i.e. Imagine pointing emacs to your local development version of a package.

There's cask and pallet, but again it didn't fulfill that need because it still fetched and installed dependencies using package.el/melpa (so what if I wanted my package to depend on another local package of mine). And el get + el-get-lock I couldn't get to work. So I ended up writing quelpa's file fetcher to alleviate this a bit, but if you use it, it basically means you have to roll your own update/rollback method for packages that use the file fetcher. (Which I did for myself, but it's sort of dependent on my own dotfiles and setup so it's not really a solution.) So I look forward to using straight.el when I can find the time, thank you!

(I have my own plans for system package management…)

Also consider contributing to Nix if it works for you! Or maybe even make dotman use Nix underneath if it can?! (if the platform supports it) :stuck_out_tongue:

[vyp]

From what I can tell, the place where straight.el wins against Nix (and all other package managers that I know of) is how focused it is on local modifications.

So just to be sure, these modifications are a "form of state" right? As in, if I was to use my init.el on another machine, these local changes would not be there right? And I can just do straight-normalize-package/straight-normalize-all to remove these changes?

[raxod502]

That is (almost) correct. There is one canonical state, and then arbitrary local modifications on top of that. To revert to the fully canonical state, you'll want to do M-x straight-thaw-versions. Running M-x straight-normalize-all will e.g. make sure you've checked out master, but it won't reset master to the exact revision. More likely, you'll want to do M-x straight-freeze-versions, which will perform normalization (prompting you to merge in feature branches and the sort) and then write the current commits into your lockfile.

[vyp]

So I looked into this a bit more and sadly it looks like nix manages emacs packages as a wrapper around package.el or something like that, so I think this issue should be closed. I spoke way too soon. (It's possible guix is more clever than nix, especially since guile, the language that guix is written in, has an elisp interpreter, but I don't use guix anymore so I wouldn't be able to say.)

For reference, my evidence to support this include, from https://nixos.org/nixos/manual/index.html#module-services-emacs:

Under NixOS, you can continue to use package-list-packages and package-install to install packages. You can also declare the set of Emacs packages you need using the derivations from Nixpkgs.

You can check that it works by executing this in a terminal:

$ nix-build emacs.nix
$ ./result/bin/emacs -q

and then typing M-x package-initialize.

Note, needing to do package-initialize isn't that big of a deal, but it's already worse than straight.el, and it's imperative and non-functional.

The Emacs init file should be changed to load the extension packages at startup:

(require 'package)

;; optional. makes unpure packages archives unavailable
(setq package-archives nil)

(setq package-enable-at-startup nil)
(package-initialize)

After the declarative emacs package configuration has been tested, previously downloaded packages can be cleaned up by removing ~/.emacs.d/elpa (do make a backup first, in case you forgot a package).

Note that straight.el is better with a single source of truth, the config file, rather than the state of the filesystem.

Finally, the example config given here also uses package.el.

I'm not sure if it's strictly true that nix managed emacs packages rely on using package.el, but from the above it seems like it. If anyone reading this knows otherwise, of course please say.

Nix still has the significant advantage of being able to install non-elisp parts of a package, but for me at least having something that's functional is much more important. Further, I'm pretty sure that the vast majority of emacs packages are pure elisp, so it's not that much of a pain to install and manage any non-elisp parts yourself as you would on any non-NixOS system.

[raxod502]

Nevertheless I think this will be useful information to put in the README! Thanks.

[AndersonTorres]

Hello! Some cents here. As I understand it, Nix can be used in a more generalistic way. After all, anyone can write a custom Nix build expression for any Emacs configuration.

I haven't read the relevant files, but I think Nixpkgs just repackages the files in order to "emulate" the look and feel of package.el.

[vlaci]

I have created a tool which hooks into straight and generates the repository from nix package derivations. I don't know if it is viable as a general purpose tool outside of my use-case of packaging doom-emacs for nix.
The idea is to use two phases for installing:

1. Disable package installation and create a json report of all packages would be installed by straight.
2. Populate the straight repository from the nix store instead of cloning the recipes from git

My current implementation is somewhat limited as packages are matched by name and e.g. versions cannot be overridden from recipes. A more correct approach would be to generate the nix packaging from the straight recipe if needed.

[kototama]

One pain point (I think) with using Nix for emacs packages could be that it needs the sha256 of the package source in it's nix expression.

I think the Nix packages for Emacs are populated from melpa so it's transparent for the user.

I'm using Emacs with home-manager (it uses Nix but it's designed for your home files and non-system binaries).

You can define a package from a repository like here: https://github.com/kototama/nix-home/blob/master/home.nix#L4
but most of the time you just need to write the name of the packages which are included in Nix like here: https://github.com/kototama/nix-home/blob/master/home.nix#L39

The advantage of the Nix package manager is that you can install from any sources: github, tarball, local file whatever. All of these scenario are supported by the Nix language, in which you express your dependencies/packages. You can even combine different Nix channels (a channel is where you get the packages from, like a debian repository) for example stable and unstable repository. So it's possible to have magit in its latest version (unstable) and paredit from stable (for example). If the channels are pinned to a specific version then the configuration is totally reproducible between different machines. It's functional so you can always jump back to a previous iteration of your configuration.

Disadvantages are: poor documentation, even if the Nix language is easy on its own it's not easy to find how to achieve something with it. concepts and terminology are unfamiliar for mosts making it harder to learn.

[maisiliym]

It's a little odd since nix is a system package manager

The system can only be correctly analyzed as a whole, hence to make such a distinction is a symptom of incorrectness. Nix is flawed, guix fixes some of those flaws, but they are currently both the pinnacle in this area.

[kototama]

The system can only be correctly analyzed as a whole, hence to make such a distinction is a symptom of incorrectness.

Can you develop?