Nixpkgs: All stdenvs should have a generic libc attribute

[copumpkin]

Currently, the linux stdenv has

extraAttrs = {
      inherit (stage4.pkgs) glibc;
      ...
};

which gives it a glibc attribute, but now that we're getting a proper darwin stdenv, it would be nice to be able to refer to its standard library generically. Darwin's stdenv already has a libc in its extraAttrs, so I think the solution to this would be as simple as putting that attribute into the linux stdenv's extraAttrs, or into the generic stdenv builder. Adding attributes to the various bootstrap phases will cause a massive linux rebuild though...

[copumpkin]

As an example, this would help with

  wrapCC = wrapCCWith (makeOverridable (import ../build-support/cc-wrapper)) glibc;

which could turn into

  wrapCC = wrapCCWith (makeOverridable (import ../build-support/cc-wrapper)) stdenv.libc;

Same with many other unnecessary glibc assumptions throughout the package universe.

[vcunat]

General stdenv.libc certainly makes sense to me.

If we just add libc attribute to linux stdenv without removing glibc attribute, then nothing should break. Even after changing stdenv.glibc occurences to stdenv.libc, nothing should even need to rebuild.

[wmertens]

+1

BTW, sidetrack: this would help with better specifying dependencies instead
of whitelisting platforms.

Right now we need to provide platforms for packages, but if we instead
clarified that a package depends on glibc vs libc then it won't build on
Darwin because there's no glibc. If we also define (symlinked)
linux-proc-mount and linux-sys-mount dependencies and other things like
that we can do away with the platforms altogether, the dependencies will
either be available or not. Hypothetically, if someone made a Linux
compatible /sys for Darwin, that would suddenly make programs available on
Darwin. Same applies for Cocoa in the other direction.

So making libc mean "just some sort-of-standard libc like in
http://en.wikipedia.org/wiki/C_standard_library#Standardization" would make
most packages work, and special things like libiconv are extra buildInputs
that refer to glibc or an actual libiconv or whatever.

This makes package specification purer. If you decide to build a system
based on uclibc or musl, it would mostly "just work".

So basically, stdenv should not have the glibc attribute, it should be
required separately and probably via "feature-interfaces" like libiconv and
glibcLocales. If a package fails to build on Darwin because it's trying to
use some library that doesn't exist, that means stdenv has hidden
Linux-only dependencies.

(glibcLocales is null when not available, indicating that it's an optional
requirement normally. In case it's not an optional dependency it needs to
be specified as requiring glibc as buildInput)

On Sun Feb 08 2015 at 10:52:06 AM Vladimír Čunát [email protected]
wrote:

General stdenv.libc certainly makes sense to me.

If we just add libc attribute to linux stdenv without removing glibc
attribute, then nothing should break. Even after changing stdenv.glibc
occurences to stdenv.libc, nothing should even need to rebuild.

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/issues/6221#issuecomment-73404127.

[copumpkin]

It looks like this is already available as stdenv.cc.libc, but perhaps it should be one level up as well?

[copumpkin]

Also, @wmertens, generally agree with everything you said, and have had to deal with much of the pain from what you describe.

[gitfoxi]

+1

I have tried and failed to get nix going on alpine linux which uses musl in place of glibc. At this point in history, linux != glibc.

It would be cool to have musl or uclibc based binary channels. This avoids glibc's kernel requirements which shuts out kernels prior to 2.6.32 among other things. Musl can use any 2.6 kernel and uclibc has threading implementations that can support even 2.4 kernels.

[vcunat]

Personally I don't see enough justification to have another whole platform built by hydra.nixos.org (many thousand builds) just to use a different libc. We even removed

[gitfoxi]

@vcunat I'm using

[vcunat]

Sure, you can submit pull requests that add support to optionally use musl instead of glibc (for some Linux platforms), and I'm pretty sure that will be accepted (in principle). I'm just not convinced (yet) to significantly increase the load of hydra.nixos.org because of it (by also building all against it).

[gitfoxi]

@vcunat An added benefit is musl strives to be standards compliant and in doing so surfaces bugs. I've just got nix running under alpine this morning and already found bugs in xz, autoconf, boehm-gc and nixpkgs itself. I've just got to hello building as stdenvNative (not a full bootstrap) so I'm sure to encounter many more as I pursue this.

[edolstra]

@vcunat I wouldn't be in favor of adding musl support at this point because it just increases maintenance burden. My experience with other alternative libcs (like dietlibc, klibc, ...) is that you just end up with a gazillion patches to packages that then need to be maintained.

[wmertens]

How about adding it but not accepting patches due to it, instead requesting
they be upstreamed?

On Mon, Jun 29, 2015, 18:38 Eelco Dolstra [email protected] wrote:

@vcunat https://github.com/vcunat I wouldn't be in favor of adding musl
support at this point because it just increases maintenance burden. My
experience with other alternative libcs (like dietlibc, klibc, ...) is that
you just end up with a gazillion patches to packages that then need to be
maintained.

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/issues/6221#issuecomment-116754223.

Wout.
(typed on mobile, excuse terseness)

[gitfoxi]

@wmertens Tricky to upstream because problems arise due to musl being more posix-compliant than glibc. They're not interested in making musl bug-compatible with glibc.

[wmertens]

In that case I think musl NixOS will have to live in a fork for a while...
We have few enough resources as it is...

On Mon, Jun 29, 2015, 19:00 Michael Fox [email protected] wrote:

@wmertens https://github.com/wmertens Tricky to upstream because
problems arise due to musl being more posix-compliant than glibc. They're
not interested in making musl bug-compatible with glibc.

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/issues/6221#issuecomment-116761738.

Wout.
(typed on mobile, excuse terseness)

[vcunat]

I don't have any experience with that, but generally I can't see why upstream should be opposed to improving POSIX compatibility. Most packages are supported also on darwin and BSD (i.e. without glibc).

[wizeman]

@gitfoxi I think the point was that we'd require patches to _packages_ (to improve musl/POSIX compatibility) be upstreamed, not patches to _musl_ (to make it bug-compatible with glibc). Although, I don't think we'd want to apply lots of patches to musl either...

[joncfoo]

+1 on the proposed patch. This would at least allow some experimentation with alternative libc implementations. The new flag could be documented to say something along the lines of: "here be dragons if not using gnulibc".

As far as support goes, I can see that supporting patches could quickly become a nightmare. I don't think it's unreasonable to not provide a separately maintained channel [and all the work that entails].

[vcunat]

@joncfoo: as noted, for now there's at least stdenv.cc.libc attribute (working for darwin and linux at least).

[joncfoo]

@vcunat, sorry I should have read more carefully! I didn't realize the option was already available.

@gitfoxi, perhaps we can work together on setting up an alternative libc channel (musl in this case) and see how far we get.

[Profpatsch]

(triage) what’s the status?

[wmertens]

Alright so to do this we want to:

• remove stdenv.glibc
• add stdenv.libc
• do a gigantic rename

This shouldn't cause rebuilds.

Then, on Darwin, ongoing:

• fix packages that need glibc with "feature packages"

These can be found by compiling with uclibc as well methinks.

Right?

[copumpkin]

Sounds right! You volunteering? 😄

[wmertens]

Well, I could do the scripting for the renames… not sure about the darwin
build fixing.

On Fri, Jul 29, 2016, 5:39 AM Daniel Peebles [email protected]
wrote:

Sounds right! You volunteering? 😄

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/6221#issuecomment-236087209, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADWluAsZw35uoXrJFh3nsat_bKqSaniks5qaXYKgaJpZM4DdQZ7
.

[Ericson2314]

In https://github.com/NixOS/nixpkgs/pull/21268 I add a libc attribute in all-packages with the libc derivation for the current stage. When cross compiling (crossDrvs) this subsumes libcCross, and is chosen according to crossSystem.libc. I'd like to follow-up and make a buildPlatform.libc for choosing the non-cross-compiling one---I prefer that to stdenv.libc as I think we are already shoving too much functionality into stdenvs.

[Ericson2314]

https://github.com/NixOS/nixpkgs/pull/24610 will make a {build,host,target}Platform.libc be a thing. We can use that when bootstrapping and otherwise to choose the libc.

[Ericson2314]

@LnL7 I think we'll want a libcxx too.

[LnL7]

@Ericson2314 yeah, that should make the clangStdenv definitions nicer.

[Ericson2314]

OTOH, I'm we might be fine with stdenv.cc.lib{c,cxx} rather than stdenv.lib{c,cxx}. For example, why should stdenvNoCC have any opinions about these libraries?

[Ericson2314]

stdenv.cc.libc and *Platform.libc are implemented and good enough.