Rust: Cross-compiling to Windows is broken

[emberian]

(See later posts, _Unwind_Resume symbol is missing)

[emberian]

It's not just clang! With gcc:

gcc -E -MMD -MP -MT i686-pc-mingw32/rt/arch/i386/morestack.o -MF i686-pc-mingw32/rt/arch/i386/morestack.d /home/cmr/hacking/rust/src/rt/arch/i386/morestack.S | /home/cmr/hacking/rust/build/x86_64-unknown-linux-gnu/llvm/Release+Asserts/bin/llvm-mc -assemble -filetype=obj -triple=i686-pc-mingw32 -o=i686-pc-mingw32/rt/arch/i386/morestack.o
/home/cmr/hacking/rust/src/rt/arch/i386/morestack.S:3:19: error: unexpected token in directive
.section .note.GNU-stack, "", @progbits
                  ^
/home/cmr/hacking/rust/src/rt/arch/i386/morestack.S:90:2: error: unknown directive
 .hidden __morestack
 ^
/home/cmr/hacking/rust/src/rt/arch/i386/morestack.S:98:8: error: expected absolute expression
 .type __morestack,@function
       ^
/home/cmr/hacking/rust/mk/rt.mk:93: recipe for target 'i686-pc-mingw32/rt/arch/i386/morestack.o' failed
make: *** [i686-pc-mingw32/rt/arch/i386/morestack.o] Error 1

[emberian]

This makes me think it's just wrongness.

[emberian]

To reproduce: ./configure --target=i686-pc-mingw32,x86_64-unknown-linux-gnu; make

[bharrisau]

Also #11312, llvm-mc is borking on some of the assembly.

[mvdnes]

When using the tripple i686-pc-mingw32, I get the same error.

In addition, for i686-w64-mingw32 (of which I am unsure if it supported, but tried anyway) I get the following output:

make[1]: Leaving directory `/home/mvdnes/rust/src/compiler-rt'
cp: i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/libcompiler-rt.a
compile: i686-w64-mingw32/rt/rust_builtin.o
compile: i686-w64-mingw32/rt/rust_android_dummy.o
compile: i686-w64-mingw32/rt/rust_test_helpers.o
compile: i686-w64-mingw32/rt/rust_try.o
compile: i686-w64-mingw32/rt/arch/i386/record_sp.o
link: i686-w64-mingw32/rt/rustrt.lib
touch i686-w64-mingw32/rt/backtrace.lib
oxidize: i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/libstd
error: linking with `i686-w64-mingw32-g++` failed: exit code: 1
note: i686-w64-mingw32-g++ arguments: '-m32' '-L/home/mvdnes/rust/i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib' '-o' 'i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std-31b43f22-0.10-pre.dll' 'i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o' '-lmorestack' 'i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.metadata.o' '-shared-libgcc' '-Li686-w64-mingw32/rt' '-L.' '-L/home/mvdnes/rust/.rust' '-L/home/mvdnes/rust' '-lrustrt' '-lgcc_s' '-shared' '-lcompiler-rt'
note: i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0xd14): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x1e26): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x2426): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x2cdd): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x44a2): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x585d): more undefined references to `__Unwind_Resume' follow
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f284): undefined reference to `__Unwind_RaiseException'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f2fd): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f307): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f518): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f522): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x8f728): undefined reference to `__Unwind_Resume'
i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o:(.text+0x93616): more undefined references to `__Unwind_Resume' follow
/usr/bin/i686-w64-mingw32-ld: i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/std.o: bad reloc address 0x14 in section `.data'
collect2: ld returned 1 exit status

error: aborting due to previous error
make: *** [i686-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/stamp.std] Error 101

[emberian]

Nothing in the mingw-w64 distribution provides that symbol, and compiler-rt doesn't seem to either. What I get today:

PATH=/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib:$PATH   x86_64-unknown-linux-gnu/stage2/bin/rustc --cfg stage2  -O --cfg rtopt --cfg debug -C rpath  --target=i686-w64-mingw32 -C linker=i686-w64-mingw32-g++ -C ar=i686-w64-mingw32-ar   -D warnings -L "i686-w64-mingw32/rt" -L "" -L ""  --out-dir x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib -C extra-filename=-4e7c5e5c /home/cmr/src/rust3/src/librustrt/lib.rs
error: linking with `i686-w64-mingw32-g++` failed: exit code: 1
note: i686-w64-mingw32-g++ '-m32' '-L' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib' '-o' 'x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt-4e7c5e5c.dll' 'x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' 'x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.metadata.o' '-shared-libgcc' '-Wl,--enable-long-section-names' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/libcollections-4e7c5e5c.rlib' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/libunicode-4e7c5e5c.rlib' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/liballoc-4e7c5e5c.rlib' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/liblibc-4e7c5e5c.rlib' '/home/cmr/src/rust3/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/libcore-4e7c5e5c.rlib' '-L' 'i686-w64-mingw32/rt' '-L' '.' '-L' '/home/cmr/.rust/lib' '-L' '/home/cmr/src/rust3/build/.rust' '-L' '/home/cmr/src/rust3/build' '-L' '/home/cmr/.rust' '-Wl,-Bdynamic' '-lgcc_s' '-Wl,-Bstatic' '-lrustrt_native' '-Wl,-Bdynamic' '-shared' '-lcompiler-rt'
note: x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x31): undefined reference to `__gcc_personality_v0'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0xcb5): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0xcbd): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0xde6): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0xeb7): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x1a0b): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x1eec): more undefined references to `_Unwind_Resume' follow
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x6dd3): undefined reference to `_Unwind_RaiseException'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x6e8f): undefined reference to `__gcc_personality_v0'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x70de): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x7578): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x7a4f): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x7b04): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x7dbf): undefined reference to `_Unwind_Resume'
x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o:(.text+0x7e37): more undefined references to `_Unwind_Resume' follow
/usr/lib/gcc/i686-w64-mingw32/4.9.0/../../../../i686-w64-mingw32/bin/ld: x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/rustrt.o: bad reloc address 0x4 in section `.data'
collect2: error: ld returned 1 exit status

error: aborting due to previous error
/home/cmr/src/rust3/mk/target.mk:166: recipe for target 'x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/stamp.rustrt' failed
make: *** [x86_64-unknown-linux-gnu/stage2/lib/rustlib/i686-w64-mingw32/lib/stamp.rustrt] Error 101

[emberian]

cc @klutzy @vadimcn

[emberian]

The _Unwind_SjLj_Resume symbol is defined, should we be using that instead?

[bharrisau]

Anything like this? http://stackoverflow.com/a/22774664

[emberian]

Compiling a C++ program that uses exceptions uses __Unwind_SjLj_Resume, it's possible the mingw-w64 that Arch ships was built with sjlj instead of dwarf/seh.

[emberian]

@bharrisau related but not quite the same, I think.

[emberian]

The Arch pkgbuild indeed uses --disable-dw2-exceptions

[klutzy]

LLVM is converting resume instruction into call _Unwind_Resume which is not available if mingw-w64 package uses sjlj configuration.
I don't know what is clear way to fix this. We may fix llvm to emit _Unwind_SjLj_Resume for certain setting (LLVM currently does for iOS setting), but I don't even know if rust's unwinding works well in sjlj..

[emberian]

Can we add some linker setting or a wrapper function that does the right
thing? I tried to do it (see the patch I sent you) but maybe it was in the
wrong place...

On Tue, Jul 15, 2014 at 11:18 PM, klutzy [email protected] wrote:

LLVM is converting resume instruction into call _Unwind_Resume which is
not available if mingw-w64 package uses sjlj configuration.
I don't know what is clear way to fix this. We may fix llvm to emit
_Unwind_SjLj_Resume for certain setting (LLVM currently does for iOS
setting), but I don't even know if rust's unwinding works well in sjlj..

—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rust/issues/12859#issuecomment-49127610.

http://octayn.net/

[klutzy]

Here's quick-and-dirty hack. I'm sure cross build unwinding is broken, but anyway make rustc-stage1 passed and cross-hello-world.exe worked (which depends on libgcc_s_sjlj-1.dll).

[klutzy]

For 64-bit target (x86_64-w64-mingw32), Debian and Ubuntu switched to seh.
Arch also seems to use seh for 64-bit. Don't know about other repos, but since seh is default of mingw-w64 so any EH incompatibility means it's upstream issue.

However, for 32-bit target, Debian, Ubuntu, Arch use sjlj. Since sjlj is considered as "slow-but-reliable" default and dwarf is "fast-but-unreliable" option, many linux distros has provided sjlj only. (it's not mingw-w64 specific; old mingw has suffered same issue. Debian issue)

[steveklabnik]

I wonder, is this still true today?

[klutzy]

I think it's still true on 32-bit since the core issue is just that most distribution uses "common" unwinding (sjij) different to what we want (dwarf). (It's not a problem for 64-bit.)

[klutzy]

I'm still not sure what's the best action to close this. The "complete" fix is just to build cross gcc from scratch. The simplest fix is "let's add fake sjij-unwinding support". We could add more proper sjlj support on Windows, but I'm not sure how meaningful it is.

[vadimcn]

@klutzy, what makes it "fake"?

But yeah, the bigger question is: Do we have a compelling reason for supporting SjLj in addition to Dwarf on win32?
Can we simply tell Linux cross-builders to use dwarf-based mingw?

[jan-hudec]

Does that mean the Windows MinGW builds use different configuration than the Linux cross-compilers for the same target?

[vadimcn]

MinGW is available in both flavors, but we only support the dwarf one.

[Fedcomp]

Do we have a compelling reason for supporting SjLj in addition to Dwarf on win32?

Just curious, what would it take to add SjLj support? a LOT of work?

[hone]

Hi, I was wondering if there was an update on this issue. Ideally, I'd like to not rebuild gcc from scratch. I'm interested in rust to be used with mruby and particularly with setting up a cross compiling toolchain from linux -> 32/64 bit osx and winodws. I believe I've run into the same issue with trying to build target i686-pc-windows-gnu. This is the error I'm seeing. Thanks!

[vadimcn]

Compatibility with Windows mingw builds and bitrot aside, the main sticking point, as I recall, was figuring out which mingw flavor we have during the compilation of some std::rt modules.
The optimal place to do this would be in the configure script, but since is_sjlj flag needs to be per-target, we need to crate a mapping target : is_sjlj and pass it down to make. And given the [non-existent] level of support of maps in bash and make, it didn't seem a good investment of my time to make it all work...

[alexcrichton]

@vadimcn note that with https://github.com/rust-lang/rust/pull/30448 I believe it should enable us to transition the MinGW targets over to using the SEH exceptions in which case we'll drop all MinGW dependencies and you should be able to use any toolchain to link Windows Rust code (I think)

[alexcrichton]

I'm going to close this as basically "there's nothing we can do here". Cross compiling works for x86_64 where we don't depend on the style of unwinding of the MinGW toolchain, but we can only cross compiling to a MinGW toolchain targeting dwarf exceptions on 32-bit. Unfortunately the toolchains on Linux tend to be SjLj based, but that's just not the binaries we ship for libstd.

[jan-hudec]

@alexcrichton, so what happened to the SEH plan from #30448? Is it not possible for the 32-bit gnu target after all?

[alexcrichton]

Unfortunately there's a combination of:

1) LLVM doesn't support that
2) The MinGW linker somehow produces an invalid binary

(1) may be surmountable but I couldn't make heads or tails of (2) unfortunately :(

[Arnaz87]

Hi, i'm new to rust, and with what I understood of this thread (not that much), I fixed it adding the option -C panic=abort, which as I understand, disables exception handling and just aborts the program on error, and avoids all these problems about incompatible unwinding systems.

[saurik]

I don't think I've ever seen a distribution-shipped version of MinGW that was configured to _not_ use SJLJ exception handling for 32-bit. In fact, it is so rare, I wasn't even aware it existed as an upstream-supported option. If you are going to hard-code a specific variant, you should hard-code SJLJ for 32-bit, or Rust essentially becomes arbitrarily incompatible with the entire rest of the MinGW ecosystem. Like, this is _six years_ after this bug was filed, and I can report that everyone is still using SJLJ, AFAIK no one is talking about changing that, and Rust thereby still fails to compile 32-bit binaries for Windows. :(

[mati865]

@saurik Fedora ships Dwarf-2 i686 MinGW toolchain, basically all popular native Windows MinGW toolchains also default to Dwarf-2.
Also there is performance overhead when using SJLJ.

[saurik]

@mati865 Ok, you know what? I'm sorry. FWIW, both Ubuntu and macOS Homebrew are shipping SJLJ, which is why I'm still experiencing this bubble of SJLJ whenever I do any tests, as those are the operating systems I normally work with; it turns out you are right: the packages on repo.msys2.org, which is what I'd consider the official decision at this point, are using Dwarf-2.

…and what's weird is that _my own projects_ are also using Dwarf-2, and I simply didn't notice, as I'm actually using clang to do my Win32 compilation right now, and it is, in fact, generating Dwarf-2. This is where I was just really dumb: I put the math together all wrong here and came to the wrong conclusions about what toolchains were doing what. While "basically all popular" still doesn't seem quite right (and you actually said the opposite multiple times in #49078... ;P), it certainly is _morally_ correct, so again: I'm sorry.

The issue I'm experiencing is really then just an issue overriding the sysroot of the linker, and I'll take the comments over to #68887, I guess :(.

[mati865]

While "basically all popular" still doesn't seem quite right (and you actually said the opposite multiple times

I might have not been clear enough. I wanted to say "toolchains running on the Windows natively", Linux distros indeed mostly ship SJLJ based toolchain for i686.

This issue is far beyond https://github.com/rust-lang/rust/issues/68887. It'd require adding rustc argument to switch between DWARF-2 and SJLJ, then you'd have to use build-std, xargo or cargo-xbuild to build std library with chosen exception handling. Another solution would be adding new i686 target triple which would use only SJLJ.

[zzeroo]

For a project of mine I hit this error. I can't belive this issue dates from 2014 RLY?

https://gist.github.com/zzeroo/2f4168364eda226c7ec808a9153ad472

I try to set up a cross compilation environment to compile win32 binaries from a linux box. For separation I use docker containers, sorry that's all I can do to simplify the problem. Please see: https://gitlab.com/zzeroo/rust-crosspile the Dockerfile describes my setup: https://gitlab.com/zzeroo/rust-crosspile/-/blob/master/Dockerfile

This are the steps to get to the error

git clone https://gitlab.com/zzeroo/rust-crosspile -b i686
cd rust-crosspile
docker build . -t rust-crosspile

cd /tmp
cargo build --bin hello-world 
cd example 
docker create -v `pwd`:/home/rust/src --name HELLO-build rust-crosspile:latest
docker start -ai HELLO-build

[mati865]

@zzeroo it's your distribution issue.
It's fixed in Fedora 32 but fedora:latest in your Dockerfile still points to Fedora 31 according to this page: https://hub.docker.com/_/fedora?tab=description

[zzeroo]

@mati865 thank you for this fast help. Sorry for that stupid question.

[infinity0]

I am hitting this issue when trying to cross-compile from linux to 32-bit windows - on Debian, the 32-bit mingw still uses sjlj.

@mati865 if I patch src/libpanic_unwind/dwarf/eh.rs so that USING_SJLJ_EXCEPTIONS is true on 32-bit windows, do you see any potential issues resulting from this? As I understand, there should be no issues about compatibility between a linux-host rustc vs a windows-host rustc, since rust puts the metadata hash in the library name, so it is unlikely that someone will accidentally link two libraries together built using different unwinding strategies (if they did give -C prefer-dynamic)?

[infinity0]

Never mind, I found this PR and comment which explains why my intended approach would not be feasible (and it's also not as simple as setting that flag).