Futures-rs: futures-macro (pulled in by futures-util) appears not to compile on musl

Created on 25 Oct 2020  ·  30Comments  ·  Source: rust-lang/futures-rs

I have a CI pipeline that tests my project on glibc and musl and it appears that when I pull in futures-util as a dependency, this compiles just fine on glibc but does not compile on musl.

Here is the build log: https://travis-ci.org/github/jbaublitz/neli/jobs/738716295

This appears to be due to futures-macro exporting all of its procedural macros with #[proc_macro_hack] which is not allowed on musl.

I know that rust and musl have had problems in the past related to proc macros and I'm wondering if this is the right place to report this or if this should go into the rust repo.

C-question
Source
picture jbaublitz

Most helpful comment

For future NixOS users googling this error message, this can also happen when using rustup and updating your system / glibc version. Clearing ~/.rustup and reinstalling the needed toolchains solved it for me.

picture kristoff3r on 28 Feb 2021
👍3

All 30 comments

I believe this issue was fixed in Rust 1.44+ (see https://github.com/rust-lang/rust/issues/40174). Can you check musl-dev is installed?

taiki-e picture taiki-e on 25 Oct 2020

This appears to be a different issue as I do have musl-dev installed (see here) and it is able to find the proc-macro crate. It is simply complaining about the usage of #[proc_macro_hack] which is not what the linked issue seems to be about. I read that one over before filing this one.

jbaublitz picture jbaublitz on 25 Oct 2020
👀1

Hmm, I'm not sure what caused this problem, but this should eventually be fixed by https://github.com/dtolnay/proc-macro-hack/pull/64.

It should be able to compile by disabling futures-util's default features until that pr is merged or another workaround is found.

futures-util = { version = "0.3", default-features = false, features = ["std"] }
taiki-e picture taiki-e on 25 Oct 2020

Oh thanks for putting up a PR! Do you want me to test anything for now?

jbaublitz picture jbaublitz on 25 Oct 2020

I'm having the same or at least a similar issue. Adding

futures-util = { version = "0.3", default-features = false, features = ["std"] }

to Cargo.toml doesn't fix it. Here's my output:

❯ cargo build
   Compiling futures-macro v0.3.7
   Compiling pin-project-internal v0.4.27
   Compiling serde_derive v1.0.117
   Compiling thiserror-impl v1.0.21
error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:31:1
   |
31 | pub fn join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:37:1
   |
37 | pub fn try_join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:43:1
   |
43 | pub fn select_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:49:1
   |
49 | pub fn select_biased_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-macro-0.3.7/src/lib.rs:24:5
   |
24 | use proc_macro_hack::proc_macro_hack;
   |     ^^^^^^^^^^^^^^^

error: aborting due to 5 previous errors

error: could not compile `futures-macro`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

I don't know what the deal is with that specific version of glibc being required. The version I have available is 2.31.

loewenheim picture loewenheim on 28 Oct 2020

@loewenheim I believe you have another dependency that enables async-await feature of futures-util. Can you show me the output of cargo tree -i -p futures-util?

❯ cargo build
Compiling futures-macro v0.3.7

taiki-e picture taiki-e on 28 Oct 2020

Here's the output:

❯ cargo tree -i -p futures-util
futures-util v0.3.7
├── futures v0.3.7
│   └── teloxide v0.3.2
│       └── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
├── futures-executor v0.3.7
│   └── futures v0.3.7 (*)
├── h2 v0.2.7
│   └── hyper v0.13.8
│       ├── hyper-tls v0.4.3
│       │   └── reqwest v0.10.8
│       │       └── teloxide v0.3.2 (*)
│       └── reqwest v0.10.8 (*)
├── hyper v0.13.8 (*)
├── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
└── reqwest v0.10.8 (*)
loewenheim picture loewenheim on 28 Oct 2020

@loewenheim

Here's the output:

Seems async-await feature is enabled by teloxide.

taiki-e picture taiki-e on 28 Oct 2020

Since I merged #2243, this should be fixed on the master branch.

taiki-e picture taiki-e on 28 Oct 2020

Unfortunately, that hasn't seemed to fix it for me. I now get

❯ cargo tree -i -p futures-util
futures-util v0.3.7 (https://github.com/rust-lang/futures-rs?rev=4122f3d5#4122f3d5)
├── futures v0.3.7
│   └── teloxide v0.3.2
│       └── latexbot-rs v0.1.0 (/home/sebastian/git/latexbot-rs)
├── futures-executor v0.3.7
│   └── futures v0.3.7 (*)
├── h2 v0.2.7
│   └── hyper v0.13.8
│       ├── hyper-tls v0.4.3
│       │   └── reqwest v0.10.8
│       │       └── teloxide v0.3.2 (*)
│       └── reqwest v0.10.8 (*)
├── hyper v0.13.8 (*)
└── reqwest v0.10.8 (*)

but the build still fails with exactly the same error as before.

loewenheim picture loewenheim on 28 Oct 2020

@loewenheim Replacing dependencies with git doesn't replace dependencies of dependencies with git. It needs to add a patch section like the following to workspace root's Cargo.toml.

[patch.crates-io]
futures = { git = "https://github.com/rust-lang/futures-rs" }
futures-core = { git = "https://github.com/rust-lang/futures-rs" }
futures-channel = { git = "https://github.com/rust-lang/futures-rs" }
futures-executor = { git = "https://github.com/rust-lang/futures-rs" }
futures-io = { git = "https://github.com/rust-lang/futures-rs" }
futures-macro = { git = "https://github.com/rust-lang/futures-rs" }
futures-sink = { git = "https://github.com/rust-lang/futures-rs" }
futures-task = { git = "https://github.com/rust-lang/futures-rs" }
futures-util = { git = "https://github.com/rust-lang/futures-rs" }

(In any case, I should create a new release...)

taiki-e picture taiki-e on 28 Oct 2020

Still no luck, I'm afraid. Though I'm not entirely sure that the problems are caused by this issue.

❯ cargo build
   Compiling futures-macro v0.3.7 (https://github.com/rust-lang/futures-rs#4122f3d5)
   Compiling pin-project v1.0.1
   Compiling tokio v0.2.22
   Compiling serde v1.0.117
error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libpin_project_internal-71a017b8e79917db.so)
  --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/pin-project-1.0.1/src/lib.rs:85:9
   |
85 | pub use pin_project_internal::pin_project;
   |         ^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: could not compile `pin-project`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libserde_derive-428f432050fd432d.so)
   --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.117/src/lib.rs:286:1
    |
286 | extern crate serde_derive;
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:31:1
   |
31 | pub fn join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:37:1
   |
37 | pub fn try_join_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:43:1
   |
43 | pub fn select_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: `proc-macro` crate types currently cannot export any items other than functions tagged with `#[proc_macro]`, `#[proc_macro_derive]`, or `#[proc_macro_attribute]`
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:49:1
   |
49 | pub fn select_biased_internal(input: TokenStream) -> TokenStream {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)
  --> /home/sebastian/.cargo/git/checkouts/futures-rs-b0bea7d4c3745ece/4122f3d/futures-macro/src/lib.rs:24:5
   |
24 | use proc_macro_hack::proc_macro_hack;
   |     ^^^^^^^^^^^^^^^

error: aborting due to 5 previous errors

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libtokio_macros-b907bffc258ed627.so)
   --> /home/sebastian/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/lib.rs:403:13
    |
403 |     pub use tokio_macros::select_priv_declare_output_enum;
    |             ^^^^^^^^^^^^

error: aborting due to previous error

error: build failed
loewenheim picture loewenheim on 28 Oct 2020

@loewenheim
Hmm, it seems all proc-macros failed to compile. Can you show me the version of rustc (rustc --version --verbose)?
Also, if you installed rustc using a way other than rustup, that may be the cause.
If updating the compiler and using the toolchain installed via rustup doesn't fix the problem, you may be able to fix it by using -C target-feature=-crt-static (https://github.com/rust-lang/rust/issues/59302):

RUSTFLAGS="-C target-feature=-crt-static" cargo build
taiki-e picture taiki-e on 30 Oct 2020

I installed rust via rustup. rustc --version --verbose outputs

rustc 1.47.0 (18bf6b4f0 2020-10-07)
binary: rustc
commit-hash: 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39
commit-date: 2020-10-07
host: x86_64-unknown-linux-gnu
release: 1.47.0
LLVM version: 11.0

Adding the target-feature flag didn't fix the problem. I'm very thankful for your assistance, but I'm honestly not sure investigating this further is worth your time—I'm using NixOS and there's just been a major update, the problem may be related to that.

loewenheim picture loewenheim on 30 Oct 2020

Possibly related: https://github.com/rust-lang/rust/issues/78210
I think it's a slightly different issue, but the same workaround (specify target explicitly) may work:

RUSTFLAGS='-C target-feature=+crt-static' cargo build --target x86_64-unknown-linux-gnu
taiki-e picture taiki-e on 6 Nov 2020

I finally figured out what the issue on my end was—it was a NixOS problem, nothing to do with the library at all.

loewenheim picture loewenheim on 7 Nov 2020

@loewenheim

it was a NixOS problem,

Is the problem tracked on the NixOS side? (I searched for it a few days ago but I couldn't find any related issues...)

taiki-e picture taiki-e on 7 Nov 2020

I'm sorry, I was being unclear. It was a problem with my specific NixOS configuration.

loewenheim picture loewenheim on 7 Nov 2020

Ah, ok, thanks for clarifying.

taiki-e picture taiki-e on 8 Nov 2020

@loewenheim: It seems some people encountered the same problem as you encountered (e.g., #2253). If you could tell us information about what configuration was the cause is very helpful.

taiki-e picture taiki-e on 11 Nov 2020

@jbaublitz: We have published 0.3.8. Can you see if it fixed the problem you encountered?

taiki-e picture taiki-e on 11 Nov 2020

This may help:

I've encountered the same issue (on NixOS) in an older rust nightly version (I believe it was the nightly from 22th October), after I've updated to the most recent (8th November) (using rustup) the issue was gone. @loewenheim and all NixOS users have you tried the most recent nightly rust toolchain?

I'm not sure though if the rust toolchain update was the only necessary fix. For me it stopped working (with the same issue as @loewenheim had), and after some time (I didn't actively pursue this issue) it started working again...

Philipp-M picture Philipp-M on 11 Nov 2020
👀1

error: /nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6: version `GLIBC_2.32' not found (required by /home/sebastian/git/latexbot-rs/target/debug/deps/libproc_macro_hack-59bcb376d337da05.so)

This error is likely https://superuser.com/questions/537683/how-to-fix-lib-x86-64-linux-gnu-libc-so-6-version-glibc-2-14-not-found.

That means the program was compiled against glibc version 2.14, and it requires that version to run, but your system has an older version installed. You'll need to either recompile the program against the version of glibc that's on your system, or install a newer version of glibc (the "libc6" package in Debian).

taiki-e picture taiki-e on 11 Nov 2020

In my case, it worked like this: my system depends on the 20.09 channel of nixpkgs.

  # flake.nix
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09";

I also make local dev environments for Rust projects. I use nix + direnv for this. I put this in the file flake.nix in the project directory:

{
  description = "Rust dev environment";
  outputs = { self, nixpkgs }: {
    # setup the devShell for x86_64-linux.
    devShell.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux;
      mkShell {
        nativeBuildInputs = [ pkgconfig rust-bindgen ];
        buildInputs = [
          # dependencies go here
          alsaLib
          libudev
          openssl
          …
        ];
      };
  };
}

and this in .envrc:

# reload when these files change
watch_file flake.nix
watch_file flake.lock
# load the flake devShell
eval "$(nix print-dev-env)"

This automatically rebuilds the dev environment whenever flake.nix changes.

The flake.nix above implicitly depends on nixpkgs. I assumed that it would use the same version as my system configuration above, but this was actually not the case—it used the unstable version. This was the cause of the problem. Some things were compiled with glibc 2.32 from unstable nixpkgs and some with 2.31 from 20.09. There are two ways to fix this.

  1. Add an explicit line to the project's flake.nix that specifies the version of nixpkgs to use, like at the start of this post.
  2. Modify the flake registry; see this blog post. By default, it says global flake:nixpkgs github:NixOS/nixpkgs. You can modify this using e.g. nix registry add flake:nixpkgs github:NixOS/nixpkgs/nixos-20.09. Running nix registry list now shows user flake:nixpkgs github:NixOS/nixpkgs/nixos-20.09 instead of the previous line. This means that you have overridden the standard definition of the nixpkgs flake and nixpkgs will refer to the 20.09 branch unless specified otherwise.

After doing either 1. or 2., you also have to update the flake using nix flake update --update-input nixpkgs --commit-lock-file.

loewenheim picture loewenheim on 11 Nov 2020

Thanks @loewenheim for the detailed information!
If I understand correctly, it can also be fixed by removing the old build artifacts that were generated before updating to nixos-20.09 (i.e., run cargo clean), right? EDIT: This guess seems not correct

taiki-e picture taiki-e on 13 Nov 2020

cargo clean wasn't sufficient in my case at least.

loewenheim picture loewenheim on 13 Nov 2020
👍1

@jbaublitz: I think this is the solution to the error you encountered. (tl; dr, It seems that it can be fixed by installing libc6-compat or gcompat.)

taiki-e picture taiki-e on 13 Nov 2020

@taiki-e I'll try to test this out when I get the change in my CI pipeline.

jbaublitz picture jbaublitz on 19 Nov 2020
❤1

Hi @taiki-e, sorry this took so long! I was previously testing on a very handspun Ubuntu docker image. I've switched over to alpine with libc6-compat and it works perfectly. Thanks for your help! Closing.

jbaublitz picture jbaublitz on 25 Nov 2020
👍1

For future NixOS users googling this error message, this can also happen when using rustup and updating your system / glibc version. Clearing ~/.rustup and reinstalling the needed toolchains solved it for me.

kristoff3r picture kristoff3r on 28 Feb 2021
👍3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

Matthias247 picture Matthias247  ·  4Comments
jedisct1 picture jedisct1  ·  3Comments
FSMaxB-dooshop picture FSMaxB-dooshop  ·  4Comments
seanmonstar picture seanmonstar  ·  4Comments
MajorBreakfast picture MajorBreakfast  ·  3Comments