Rust: under latest MinGW, cannot link with C code using stdout

[tspiteri]

I'm having an issue under MinGW and I'm not sure if it's a MinGW issue or a rustc issue. On the one hand the bug crops up after an update to two MinGW packages, but on the other hand I cannot reproduce with only C, and I don't know if rustc is using MinGW in some undocumented unsupported way.

say_hi.c:

#include <stdio.h>
void say_hi(void) {
    fprintf(stdout, "hi!\n");
}

c_main.c

void say_hi(void);
int main(void) {
    say_hi();
    return 0;
}

r_main.rs:

extern "C" {
    fn say_hi();
}
fn main() {
    unsafe {
        say_hi();
    }
}

Under the latest MinGW, specifically with packages

• mingw-w64-x86_64-crt-git-6.0.0.5066.61efe559-1 and
• mingw-w64-x86_64-headers-git-6.0.0.5066.61efe559-1,

using only gcc like this works:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ gcc c_main.c -L. -lsay_hi -o c_main.exe && ./c_main.exe
hi!

But it doesn't work with rustc:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ rustc r_main.rs -L. -lsay_hi -o r_main.exe && ./r_main.exe
error: linking with `gcc` failed: exit code: 1
  |
  = note: "gcc" "-Wl,--enable-long-section-names" "-fno-use-linker-plugin" "-Wl,--nxcompat" "-nostdlib" "-m64" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\crt2.o" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "-L" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "r_main.r_main0.rust-cgu.o" "-o" "r_main.exe" "r_main.crate.allocator.rust-cgu.o" "-Wl,--gc-sections" "-nodefaultlibs" "-L" "." "-L" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-l" "say_hi" "-Wl,-Bstatic" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-60d4a252bdda9250.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc_system-a05404e5e059417a.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librand-81cbf6f1d5a19206.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-0b63e20460046692.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-f824a78e715239af.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_unicode-2abaea46f194a6e1.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-7daa32005d584d8a.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-49ff25caaa6e945f.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-f2bd2d524c6ace24.rlib" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-c6139fac5e899293.rlib" "-Wl,-Bdynamic" "-l" "advapi32" "-l" "ws2_32" "-l" "userenv" "-l" "shell32" "-Wl,-Bstatic" "-l" "gcc_eh" "-l" "pthread" "-Wl,-Bdynamic" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "C:\\Users\\Trevor\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: ./libsay_hi.a(say_hi.o):say_hi.c:(.text+0x10): undefined reference to `__imp___acrt_iob_func'


error: aborting due to previous error

If I downgrade those two packages to

• mingw-w64-x86_64-crt-git-5.0.0.5002.34a7c1c0-1 and
• mingw-w64-x86_64-headers-git-5.0.0.5002.34a7c1c0-1,

it works with rustc too:

$ gcc -c say_hi.c
$ ar cr libsay_hi.a say_hi.o
$ rustc r_main.rs -L. -lsay_hi -o r_main.exe && ./r_main.exe
hi!

One last thing, if I compile the library using the old packages, and then upgrade the packages, both gcc c_main.c ... and rustc r_main.rs ... work. If I compile the library using the new packages, and then downgrade the packages, both gcc c_main.c ... and rustc r_main.rs ... fail.

[mati865]

Mingw-w64 seen some huge changes lately, if recompiling Rust with latest mingw-w64 doesn't work or still gives errors like that then it's probably a regression they introduced.

[cretz]

Got anyone else arriving here across the web w/ new MinGW breaks, in addition to:

pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-crt-git-5.0.0.4745.d2384c2-1-any.pkg.tar.xz
pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-headers-git-5.0.0.4747.0f8f626-1-any.pkg.tar.xz

You might also need to downgrade winpthreads (granted your version numbers may be different):

pacman -U /var/cache/pacman/pkg/mingw-w64-x86_64-winpthreads-git-5.0.0.4741.2c8939a-1-any.pkg.tar.xz /var/cache/pacman/pkg/mingw-w64-x86_64-libwinpthread-git-5.0.0.4741.2c8939a-1-any.pkg.tar.xz

Had to do both at once due to codeps. I ran into this Rust+MinGW issue when statically building latest Tor and deps w/ latest MinGW.

[tspiteri]

The old packages are also available at http://repo.msys2.org/mingw/x86_64/, where you can download the latest versions of the packages in question that include git-5.0.0 in their version rather than the new incompatible git-6.0.0. This could be required for example on a new MinGW installation where there are no git-5.0.0 packages in /var/cache/pacman/pkg.

[lins05]

Reached here by googling.

@cretz thanks for the tip, it works

[mati865]

Should be fixed with crt/headers packages in version 6.0.0.5114.b9400293. They are already available in the repository.

[tspiteri]

I upgraded to version 6.0.0.5114 and reproduced the issue. Downgrading to 5.0.0.5002 still works.

(Keep in mind that if the library is built using 5.0.0.5002, linking will work using 6.0.0.*.)

[mati865]

Similar issue was fixed recently https://github.com/Alexpux/MINGW-packages/issues/3469.

Maybe rust internals have to be recompiled.

[NovemberZulu]

Still happens for with rustc 1.25.0 (package in question is cargo-update). May be Rust build machine has older MinGW?

[mati865]

Yeah, I just forgot that Rust is using mingw-w64-builds toolchain which is still based on 5.x mingw-w64 releases.
That basically makes official Rust unusable within MSYS2.

[NovemberZulu]

The build instructions for for windows-gnu specifically suggest using MSYS2 :( . Could you please elaborate a bit about that mingw-w64-builds toolchain?

[mati865]

Bundling /mingw64/x86_64-w64-mingw32/lib/libcrtdll.a from MSYS2 toolchain could help but somebody would have to test it properly.

Could you please elaborate a bit about that mingw-w64-builds toolchain?

https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/

[NovemberZulu]

I looked around the link, and it seems that master already has __acrt_iob_func, but v5.x does not. I wasnot able to find any kind of roadmap or ETA for v6.x release.

[NovemberZulu]

I built current Rust master using MSYS2, as described in README, and now I am able to build cargo-update without tinkering with include files. May be switching the official builds to MSYS2 is an option?

[petrochenkov]

May be switching the official builds to MSYS2 is an option?

What MSYS2 exactly?
The mingw-w64 toolchain in MSYS2 is constantly updated and these updates regularly break something.
That's why Rust uses a stable predictable standalone version from Mingw-builds.
I'd personally recommend to do the same thing and use a standalone mingw toolchain instead of MSYS2 package (at least that's what I do).

It may make sense to bump the version used by Rust from 6.3.0 to something up-to-date though.

[NovemberZulu]

What MSYS2 exactly?

Some reasonably stable reasonably recent version :) I know this is self-contradictory.

That's why Rust uses a stable predictable standalone version from Mingw-builds.

I totally see the point, the issue is the build environment is not very well documented. Updating README with proper setup instructions for Windows GNU will help a lot.

[mati865]

Using mingw-builds toolchain with MSYS2 is not perfect either.
If you want to use libs from MSYS2 with different toolchain you can also run into this error, like here.

[tspiteri]

This build script can work around the issue. It is a bit hacky; basically it tests for the bug using this issue's test case, and if it detects the bug it creates and links libworkaround_47048.a which contains the __imp___acrt_iob_func symbol as defined in newer MinGW, which is apparently not included when linking using rustc. This only works around the original issue, and does not handle the further issues for winpthreads raised in https://github.com/rust-lang/rust/issues/47048#issuecomment-362892587.

use std::env;
use std::fs::{self, File};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::Command;

fn main() {
    let target = env::var_os("TARGET")
        .expect("TARGET")
        .into_string()
        .expect("TARGET");
    if target.contains("-windows-gnu") {
        mingw_check_47048();
    }
}

fn mingw_check_47048() {
    let rustc = env::var_os("RUSTC").expect("RUSTC");
    let out_dir = PathBuf::from(env::var_os("OUT_DIR").expect("OUT_DIR"));
    let try_dir = out_dir.join("try_47048");
    let mut cmd;

    fs::create_dir_all(&try_dir).expect("create directory");
    create_file(&try_dir.join("say_hi.c"), SAY_HI_C);
    create_file(&try_dir.join("c_main.c"), C_MAIN_C);
    create_file(&try_dir.join("r_main.rs"), R_MAIN_RS);
    create_file(&try_dir.join("workaround.c"), WORKAROUND_C);

    cmd = Command::new("gcc");
    cmd.current_dir(&try_dir).args(&["-fPIC", "-c", "say_hi.c"]);
    execute(cmd);

    cmd = Command::new("ar");
    cmd.current_dir(&try_dir)
        .args(&["cr", "libsay_hi.a", "say_hi.o"]);
    execute(cmd);

    cmd = Command::new("gcc");
    cmd.current_dir(&try_dir)
        .args(&["c_main.c", "-L.", "-lsay_hi", "-o", "c_main.exe"]);
    execute(cmd);

    // try simple rustc command that should work, so that failure
    // really is the bug being checked for
    cmd = Command::new(&rustc);
    cmd.arg("--version");
    execute(cmd);

    cmd = Command::new(&rustc);
    cmd.current_dir(&try_dir)
        .args(&["r_main.rs", "-L.", "-lsay_hi", "-o", "r_main.exe"]);
    let status = cmd
        .status()
        .unwrap_or_else(|_| panic!("Unable to execute: {:?}", cmd));
    let need_workaround = !status.success();

    // build and test libworkaround_47048.a
    if need_workaround {
        cmd = Command::new("gcc");
        cmd.current_dir(&try_dir).args(&["-fPIC", "-O2", "-c", "workaround.c"]);
        execute(cmd);

        cmd = Command::new("ar");
        cmd.current_dir(&try_dir)
            .args(&["cr", "libworkaround_47048.a", "workaround.o"]);
        execute(cmd);

        cmd = Command::new(&rustc);
        cmd.current_dir(&try_dir).args(&[
            "r_main.rs",
            "-L.",
            "-lsay_hi",
            "-lworkaround_47048",
            "-o",
            "r_main.exe",
        ]);
        execute(cmd);

        let src = try_dir.join("libworkaround_47048.a");
        let lib_dir = out_dir.join("lib");
        fs::create_dir_all(&lib_dir).expect("create directory");
        let dst = lib_dir.join("libworkaround_47048.a");
        fs::rename(src, dst).expect("move file");

        let lib_dir_str = lib_dir.to_str().expect("unsupported characters");
        println!("cargo:rustc-link-search=native={}", lib_dir_str);
        println!("cargo:rustc-link-lib=static=workaround_47048");
    }

    fs::remove_dir_all(try_dir).expect("remove directory");
}

fn create_file(filename: &Path, contents: &str) {
    let mut file = File::create(filename)
        .unwrap_or_else(|_| panic!("Unable to create file: {:?}", filename));
    file.write_all(contents.as_bytes())
        .unwrap_or_else(|_| panic!("Unable to write to file: {:?}", filename));
}

fn execute(mut command: Command) {
    let status = command
        .status()
        .unwrap_or_else(|_| panic!("Unable to execute: {:?}", command));
    if !status.success() {
        if let Some(code) = status.code() {
            panic!("Program failed with code {}: {:?}", code, command);
        } else {
            panic!("Program failed: {:?}", command);
        }
    }
}

const SAY_HI_C: &'static str = r#"/* say_hi.c */
#include <stdio.h>
void say_hi(void) {
    fprintf(stdout, "hi!\n");
}
"#;

const C_MAIN_C: &'static str = r#"/* c_main.c */
void say_hi(void);
int main(void) {
    say_hi();
    return 0;
}
"#;

const R_MAIN_RS: &'static str = r#"// r_main.rs
extern "C" {
    fn say_hi();
}
fn main() {
    unsafe {
        say_hi();
    }
}
"#;

const WORKAROUND_C: &'static str = r#"/* workaround.c */
#define _CRTBLD
#include <stdio.h>

FILE *__cdecl __acrt_iob_func(unsigned index)
{
    return &(__iob_func()[index]);
}

typedef FILE *__cdecl (*_f__acrt_iob_func)(unsigned index);
_f__acrt_iob_func __MINGW_IMP_SYMBOL(__acrt_iob_func) = __acrt_iob_func;
"#;

[mati865]

Recent mingw-w64 GCC 8.1.0 build contains __acrt_iob_func so it can be hit by this issue as well.

[mati865]

I'm not sure who is responsible for Windows GNU, @retep998?

I assume libs from https://github.com/rust-lang/rust/issues/47359 are necessary for people who don't have GCC in their PATH. However they are an issue for people with GCC/MinGW versions different from Rust ones.
They are built in specific environment and the fact they work in (some) other environments is just a coincidence that shouldn't be relied upon. Yet Windows GNU target prioritises them over host libraries and relies on host headers causing conflicts like undefined reference to__imp___acrt_iob_func'` from this issue.

It can be solved by:

• using host libraries over bundled if they exists
• bundling headers (66 MiB) and using them instead of host

[retep998]

Rustup does not currently offer a way to opt out of the bundled MinGW, but there is a way to work around that. If you use the x86_64-pc-windows-msvc toolchain and add the 86_64-pc-windows-gnu-target you will be able to cross compile to x86_64-pc-windows-gnu without any bundled MinGW and instead relying entirely on your locally installed MinGW.

[rivy]

@retep998 , Can you elaborate on how you would go about avoiding the built-in MinGW, using the locally installed MinGW? I'm trying to construct an AppVeyor configuration that will build, avoiding this type of stdio incompatibility bug.

[rukai]

@rivy this is my work around https://github.com/rukai/PF_Sandbox/blob/master/appveyor.yml

[rivy]

After looking at AppVeyor configurations by @pkgw and @rukai, I spent some time constructing a relatively generic AppVeyor CI configuration which includes a workaround to allow compilation / testing with the "gnu" ABI.

I've included the config file (.appveyor.yml) in a PR for uutils/coreutils (uutils/coreutils#1318). I think it's generic enough that it could be useful in other rust repos with only minor modifications.

This is also a workaround for #48272.

ref: #53454

[buybackoff]

Tomorrow Rust 2018 goes live, but the beta version still has problem with latest MinGW. The only fix that worked for me was from here: https://github.com/racer-rust/racer/issues/919#issuecomment-413536972

Will it be fixed so that the latest mingw-w64 just work without wasting hours on finding workarounds? Msvc version works ok, but CLion supports debugging only with gnu target and debugging is quite useful feature, you know...

The linked solution feels very hacky and even if that works for now I feel uncomfortable with just replacing some files which I have no idea what are used for. Could someone at least confirm that that should work OK without unexpected problems down the road or propose a different solution that works with debugger?

[mati865]

@buybackoff yes it works for now (in the future probably more libs will require replacing). Rust is packaging old libraries that aren't compatible with headers from your system (that's why replacing libs does fix it).

[rivy]

I'm copying all libs from the installed MinGW system to the rust area in the above .appveyor.yml config for AppVeyor testing. It works now and should act as a canary if a conflict arises.

I'm doing the same thing as a local workaround with every rustup update...

::  ## * workaround for rust-lang/rust#47048 / rust-lang/rust#53454  ## !maint: remove when resolved
::  # ** ref: <https://github.com/rust-lang/rust/issues/47048>, <https://github.com/rust-lang/rust/issues/53454>
::  # ** egs: <https://github.com/pkgw/tectonic/commit/29686db533d8732d7d97fc94270ed33b77f29295>, <https://github.com/rukai/PF_Sandbox/blob/e842613cf9ff102dfb3fbd87381319e6e6dfe3ae/appveyor.yml>
( setlocal
rem :: update gcc libraries
for /f %%G in ('where gcc') do (
    set "gcc_arch=i686"
    set "ERRORLEVEL="
    "%%G" --version | findstr /C:"i686" >NUL 2>&1
    if ERRORLEVEL 1 ( set "gcc_arch=x86_64" )
    call set "d=%%gcc_arch%%-pc-windows-gnu"
    cd "%USERPROFILE%\.rustup\toolchains"
    for /f %%H in ('dir /b/s/a:d %%d%% ^| findstr pc-windows-msvc') do (
        call copy "%%G\..\..\%%gcc_arch%%-w64-mingw32\lib\*.o" "%%H\lib" >NUL
    )
    call echo gcc libraries copied ^(%%gcc_arch%%-type; workaround for rust-lang/rust#47048 + rust-lang/rust#53454^)
)
endlocal )

It would be great to have at least an official mention of the workaround until a fix is implemented.

cc: https://github.com/rust-lang/rust/issues/48272, https://github.com/rust-lang/rust/issues/53454

[weiby3]

@retep998 I have try your method, such as cargo install --force --path . --target x86_64-pc-windows-gnu.
But it seems that rust use /usr/bin/link instead of /mingw64/bin/gcc, and it use msvc linker's flag:

/usr/bin/link: 额外的操作数 “/LIBPATH:C:\\Users\\Administrator\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib”

This makes the build failed.
Are there any solutions?

[retep998]

@weiby3 Well, the downside of my method is that you're cross compiling from -msvc and so you need to actually have VC++ installed to link build scripts...

[weiby3]

@retep998 Well, it seems that it still get undefined reference...

  = note: C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../
x86_64-w64-mingw32/bin/ld.exe: C:\Users\Administrator\.rustup\toolchains\stable-
x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-gnu\lib\crt2.o:crtexe.c:(.r
data$.refptr.__onexitbegin[.refptr.__onexitbegin]+0x0): undefined reference to `
__onexitbegin'
          C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../
x86_64-w64-mingw32/bin/ld.exe: C:\Users\Administrator\.rustup\toolchains\stable-
x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-gnu\lib\crt2.o:crtexe.c:(.r
data$.refptr.__onexitend[.refptr.__onexitend]+0x0): undefined reference to `__on
exitend'
          collect2.exe: error: ld returned 1 exit status

I have install VC++ build tools, but why shouldn't I directly use msvc toolchain to compile it instead of cross compile?:joy:

[weiby3]

Founded a hacky way to solve this problem.
Original solution from https://zhuanlan.zhihu.com/p/52524621

• edit ~\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\multirust-channel-manifest.toml

[[pkg.rust.target.x86_64-pc-windows-gnu.components]]
pkg = "rust-mingw"
target = "x86_64-pc-windows-gnu"

[[pkg.rust.target.i686-pc-windows-gnu.components]]
pkg = "rust-mingw"
target = "i686-pc-windows-gnu"

replace components to extensions, and then execute rustup component remove rust-mingw.

• add the following to ~\.cargo\config

[target.x86_64-pc-windows-gnu]
linker = "C:\\msys64\\mingw64\\bin\\gcc.exe"
ar = "C:\\msys64\\mingw64\\bin\\ar.exe"

• copy crt2.o and dllcrt2.o from C:\msys64\mingw64\x86_64-w64-mingw32\lib to ~\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\lib

Now it's OK. But you should do the above steps again once you use rustup update successfully.

[mati865]

Easiest workaround is to just copy crt2.o and dllcrt2.o from C:\msys64\mingw64\x86_64-w64-mingw32\lib to ~\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\lib every time you update.

[DezzK]

Thanks @mati865, I overwritten all *.o files in the gnu toolchain with corresponding files from Msys2 and all linking problems gone!

[weiby3]

@mati865 After update to 1.34, I have try your solution, but it still can't work. Finally do the things again and it works.

[mati865]

@rustbot modify labels: -O-windows +O-windows-gnu

[cr1901]

Easiest workaround is to just copy crt2.o and dllcrt2.o from C:\msys64\mingw64\x86_64-w64-mingw32\lib to ~.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlibx86_64-pc-windows-gnu\lib every time you update.

I upgraded to 1.35.0/rustc 1.36.0-nightly (8869ee03d 2019-05-23) today and this did _not_ solve the issue. I receive the following error when linking the code in the original post:

$ rustc r_main.rs -L. -lsay_hi -o r_main.exe && ./r_main.exe
error: linking with `gcc` failed: exit code: 1
  |
  = note: "gcc" "-Wl,--enable-long-section-names" "-fno-use-linker-plugin" "-Wl,--nxcompat" "-nostdlib" "-m64" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\crt2.o" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "-L" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "r_main.r_main.7rcbfp3g-cgu.0.rcgu.o" "r_main.r_main.7rcbfp3g-cgu.1.rcgu.o" "r_main.r_main.7rcbfp3g-cgu.2.rcgu.o" "r_main.r_main.7rcbfp3g-cgu.3.rcgu.o" "r_main.r_main.7rcbfp3g-cgu.4.rcgu.o" "-o" "r_main.exe" "r_main.4rwig1jpuebwbe6m.rcgu.o" "-Wl,--gc-sections" "-nodefaultlibs" "-L" "." "-L" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-lsay_hi" "-Wl,--start-group" "-Wl,-Bstatic" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-c954a2d44059a032.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-44ed309e7c0a24e8.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libbacktrace_sys-5562a4bb4d42cef6.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_demangle-f8fcbc8dafb5c415.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libhashbrown-a84a86328accb241.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_alloc-ec9721f567a35b2f.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-06daa6384d17722d.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-acd44a96ed61810f.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-57bfdb57672b491d.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_core-4843b04b2fd1dcfe.rlib" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-51f4f445efc13497.rlib" "-Wl,--end-group" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-eacb47c3ec0f8830.rlib" "-Wl,-Bdynamic" "-ladvapi32" "-lws2_32" "-luserenv" "-Wl,-Bstatic" "-lgcc_eh" "-lpthread" "-Wl,-Bdynamic" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-lmsvcrt" "-luser32" "-lkernel32" "C:\\Users\\William\\.rustup\\toolchains\\nightly-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: C:\Users\William\.rustup\toolchains\nightly-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\lib\crt2.o: In function `pre_c_init':
          C:/repo/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:138: undefined reference to `__p__fmode'
          C:\Users\William\.rustup\toolchains\nightly-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\lib\crt2.o: In function `__tmainCRTStartup':
          C:/repo/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:291: undefined reference to `__p__acmdln'
          ./libsay_hi.a(say_hi.o):say_hi.c:(.text+0x10): undefined reference to `__imp___acrt_iob_func'


error: aborting due to previous error

I copied the libraries just fine it seems:

William@William-THINK MINGW64 ~/src/rust-crt
$ diff -q /c/msys64/mingw64/x86_64-w64-mingw32/lib/crt2.o /c/Users/William/.rustup/toolchains/nightly-x86_64-pc-windows-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/crt2.o

William@William-THINK MINGW64 ~/src/rust-crt
$ diff -q /c/msys64/mingw64/x86_64-w64-mingw32/lib/dllcrt2.o /c/Users/William/.rustup/toolchains/nightly-x86_64-pc-windows-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/dllcrt2.o

The C link demonstrated in the original post works fine:

William@William-THINK MINGW64 ~/src/rust-crt
$ gcc c_main.c -L. -lsay_hi -o c_main.exe && ./c_main.exe

GCC version is 7.3.0:

William@William-THINK MINGW64 ~/src/rust-crt
$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/7.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-7.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
gcc version 7.3.0 (Rev2, Built by MSYS2 project)

And here is my ~/.cargo/config:

William@William-THINK MINGW64 ~/src/rust-crt
$ cat /c/Users/William/.cargo/config
[target.x86_64-pc-windows-gnu]
linker = "C:\\msys64\\mingw64\\bin\\x86_64-w64-mingw32-gcc"

Am I missing something obvious to get Rust working again?

[mati865]

Am I missing something obvious to get Rust working again?

It's still not fixed and this workaround is recommend.

[cr1901]

It's still not fixed and this workaround is recommend.

But the workaround _did not_ work. Even after I copied the relevant libraries, I still get the undefined reference to__imp___acrt_iob_func'` error, among others. The information in my previous message is to try to isolate why the workaround didn't work.

Anyways, I'm fully upgrading my msys2 distribution to see if this helps.

[mati865]

It could mean you have to copy more libs like libmsvcrt, libmingw* when using not latest mingw-w64 or there is discrepancy between your crt and it's headers.

[cr1901]

~I have fully upgraded my MSYS2 installation. I have copied over crt2.o and dllcrt2.o to Rust's directory and the problem still remains.~

~What further steps should I take? E.g. where is libmsvcrt and libmingw* I can copy to the Rust directory?~

EDIT: Oops, I forgot to do rustup component remove rust-mingw, sorry! Just for my own curiosity, why is this step required?

[retep998]

EDIT: Oops, I forgot to do rustup component remove rust-mingw, sorry! Just for my own curiosity, why is this step required?

Because the mingw bits in that component are a different version than the mingw used to compile the c code you're linking to and therefore incompatible. You don't actually need to copy over anything if you just remove that mingw component, and rely on your own external mingw.

[marmistrz]

Probably a duplicate of https://github.com/rust-lang/rust/issues/53454

[mati865]

@marmistrz this issue was opened earlier so #53454 is the duplicate.

[daniel-shimon]

For whoever is getting here with the original problem, the following worked for me:

(undefined reference to '__imp___acrt_iob_func')

1. Create workaround.c with the following

(thanks @tspiteri for the workaround code)

/* workaround.c */
#define _CRTBLD
#include <stdio.h>

FILE *__cdecl __acrt_iob_func(unsigned index)
{
    return &(__iob_func()[index]);
}

typedef FILE *__cdecl (*_f__acrt_iob_func)(unsigned index);
_f__acrt_iob_func __MINGW_IMP_SYMBOL(__acrt_iob_func) = __acrt_iob_func;

2. Compile and create the library

gcc -fPIC -O2 -c workaround.c
ar cr libworkaround_47048.a workaround.o

3. Move the library to rust's lib folder

mv libworkaround_47048.a ~/.rustup/toolchains/stable-x86_64-pc-windows-gnu/lib/rustlib
/x86_64-pc-windows-gnu/lib/

rm workaround.c workaround.o

4. Add the following to ~/.cargo/config:

[target.x86_64-pc-windows-gnu]
rustflags=["-Clink-args=-lworkaround_47048"]

Enjoy 😄

[Migi]

@daniel-shimon & @tspiteri: this workaround is not compiling for me. It gives the following 2 warnings and 2 errors:

workaround.c: In function ‘__acrt_iob_func’:
workaround.c:7:14: warning: implicit declaration of function ‘__iob_func’ [-Wimplicit-function-declaration]
     return &(__iob_func()[index]);
              ^
workaround.c:7:26: error: subscripted value is neither array nor pointer nor vector
     return &(__iob_func()[index]);
                          ^
workaround.c: At top level:
workaround.c:11:1: warning: parameter names (without types) in function declaration
 _f__acrt_iob_func __MINGW_IMP_SYMBOL(__acrt_iob_func) = __acrt_iob_func;
 ^
workaround.c:11:1: error: function ‘__MINGW_IMP_SYMBOL’ is initialized like a variable

Updating from gcc 5.4.0 to gcc 7.4.0 did not fix this (on MinGW 64 bit, Windows 7).

EDIT: The issue was with how I started MSYS2. I don't know precisely what I did wrong (I was messing with the msys2_shell.cmd that launches MSYS2), but it's working now that I use the stock batch script with arguments -mingw64 -use-full-path (I had to add that -use-full-path argument to get cargo working).

[bluretrece]

For whoever is getting here with the original problem, the following worked for me:

(undefined reference to '__imp___acrt_iob_func')

1. Create workaround.c with the following

(thanks @tspiteri for the workaround code)

/* workaround.c */
#define _CRTBLD
#include <stdio.h>

FILE *__cdecl __acrt_iob_func(unsigned index)
{
    return &(__iob_func()[index]);
}

typedef FILE *__cdecl (*_f__acrt_iob_func)(unsigned index);
_f__acrt_iob_func __MINGW_IMP_SYMBOL(__acrt_iob_func) = __acrt_iob_func;

2. Compile and create the library

gcc -fPIC -O2 -c workaround.c
ar cr libworkaround_47048.a workaround.o

3. Move the library to rust's lib folder

mv libworkaround_47048.a ~/.rustup/toolchains/stable-x86_64-pc-windows-gnu/lib/rustlib
/x86_64-pc-windows-gnu/lib/

rm workaround.c workaround.o

4. Add the following to ~/.cargo/config:

[target.x86_64-pc-windows-gnu]
rustflags=["-Clink-args=-lworkaround_47048"]

Enjoy 😄

Hi Daniel @daniel-shimon , I tried your solution, but unfortunately I'm getting this error:

= note: ld: cannot find -lworkaround_47048

What am I doing wrong? Any idea of how can I solve this? Thanks in advance.

[jonas-schievink]

Nominating since this affects quite a few users of a tier 1 platform and requires weird and potentially fragile workarounds.

[pnkfelix]

visiting for triage.

We spoke briefly about this at last week's triage meeting.

In that conversation, @nikomatsakis asked whether this was a correct summary of the situation (its drawn largely from this comment from @petrochenkov):

• mingw issues breaking changes
• we are pinned to some older version right now
• it's perhaps not so well documented which version that is
• and newer versions are breaking

So the questions are:

• What do we actually claim to support?
  
  
  
  • Some fixed set of stand alone versions from mingw-builds?
  
  
  • Or do we claim to support (the constantly updated) mingw-w64 toolchain in MSYS2?
  
  
• Separate question: Should we update our CI's build environment to use a more recent version?
• GIven the points above, is this actually a T-compiler issue at all? Or is it a T-infra one?

[pnkfelix]

triage: P-high. Leaving nomination label, with hopes of discussing with T-compiler team the summary above, and take temperature as to whether this should be reclassified as a T-infra bug.

[jonas-schievink]

It sounds like this (and similar) issue could be fixed by doing what's proposed in https://github.com/rust-lang/rust/issues/53454 (unless I misunderstand the root cause of these bugs)

[mati865]

mingw issues breaking changes

Rust ships mingw-w64 libraries but not the headers. So it relies on mingw-w64 headers installed in the system and uses them together with old mingw-w64 libs shipped by rust-mingw component.
This is asking for the troubles because headers version have to match libs version.

we are pinned to some older version right now

GCC 6.3.0 revision 2 provided by mingw-builds, that means mingw-w64 5.0.

and newer versions are breaking

Older mingw-w64 versions happen to work but there are no guarantees due to mentioned earlier headers vs libs versions mismatch. Newer mingw-w64 versions don't work at all.

Or do we claim to support (the constantly updated) mingw-w64 toolchain in MSYS2?

MSYS2 packages master branch of mingw-w64 and updates it often. I doubt Rust could support it.

Should we update our CI's build environment to use a more recent version?

This has proven to not be that easy, previous attempt failed because newer GCC required more memory: https://github.com/rust-lang/rust/pull/51989
mingw-builds stopped providing new versions but it's not that hard to build it.

It sounds like this (and similar) issue could be fixed by doing what's proposed in #53454 (unless I misunderstand the root cause of these bugs)

It's exactly the same issue.
Proposition 1 is already done, proposition 2 is not going to fix it out of the box.

I believe https://github.com/rust-lang/rust/issues/53454#issuecomment-547248821 or https://github.com/rust-lang/rust/issues/53454#issuecomment-547294739 is the only real solution.
MinGW libs shipped by Rust should be last resort used only when there is no other C compiler installed.

[mati865]

Regarding previous discussion there is another argument for updating mingw-w64 version:

I'm able to build Rust+LLVM with Clang (it's already done for Linux and MSVC) by passing few flags.
It passes all the tests (modulo debuginfo for unrelated reasons) but the thing that holds me back from opening a PR is too old mingw-w64 version installed on Rust's CI.

How this can be pushed forward since mingw-builds doesn't prepare new builds any more?

[sam-ka]

I believe #53454 (comment) or #53454 (comment) is the only real solution.
MinGW libs shipped by Rust should be last resort used only when there is no other C compiler installed.

@mati865 You've mentioned here and elsewhere that this issue can be resolved by replacing rust-provided libraries with their distribution-provided counterparts.

Can swapping libraries like this cause problems for the other precompiled dlls (e.g. std-*.dll, test-*.dll, etc) shipped with rust if upstream mingw libraries diverge too much from the version shipped with rust? Is this a stable workaround?

[mati865]

@sam-ka if only import and static libraries are replaced it won't break precompiled dlls.
Shared libraries non available in the system that are required by Rust libs are still provided in bin directory. It can cause different issues like https://github.com/rust-lang/rust/issues/61561 even without using any workarounds and eventually they should be removed but fixing import and static libraries is more important.

Personally I had removed rust-mingw component and run these 2 commands every toolchain update https://gist.github.com/mati865/607049d837cbfe4968d26c811f2e11da since I have proper mingw toolchain in my PATH. I haven't had issues with that workaround so far.

[pnkfelix]

for those watching, this issue was discussed at the T-compiler meeting two weeks ago.

We essentially decided that @mati865 should move forward with their experimentation with fixes.

[bs2kbs2k]

I solved this issue by running

pacman -U https://archive.org/download/archlinux_pkg_mingw-w64-crt/mingw-w64-crt-5.0.0-1-any.pkg.tar.xz https://archive.org/download/archlinux_pkg_mingw-w64-headers/mingw-w64-headers-5.0.0-1-any.pkg.tar.xz https://archive.org/download/archlinux_pkg_mingw-w64-winpthreads/mingw-w64-winpthreads-5.0.0-1-any.pkg.tar.xz

to downgrade those 3 packages on archlinux.