Rust: __sync_val_compare_and_swap_ linker error

[lights0123]

nightly-2019-02-06 does not cause this error, but one revision later, nightly-2019-02-07, does. I'm targeting an embedded device that uses newlib and a custom gcc build. However, February 2nd, 2019's build of nightly Rust causes a link-time error, referring to __sync_val_compare_and_swap_1, __sync_val_compare_and_swap_2, and __sync_val_compare_and_swap_4. No atomic types are being used by my project. This seems similar to #56798.

Here's my custom target.json, and a log of the error:

error: linking with `nspire-gcc` failed: exit code: 1
  |
  = note: "nspire-gcc" "-L" "/home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.106gzic61cjkgqlp.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1256m1ff2vnvs2ol.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.132x4qe5hnk2pao4.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.17ql1tmsq2sdjo29.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1brbjkkjh6m7zqb4.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1nf3xvwrin6l23wj.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1qyuts48pmbnmbxn.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1rqn7jvzfiz714rg.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.1tojkli85puhoiat.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.208shlkxhdb3o79p.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.23d3ikab6hy12qxn.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.27wiy15gic05eepn.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.287yolbeazcivsey.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2az4oro1w8050jw7.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2fk4eg0d4u3aqgb3.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2gqzbm4yu0y61stz.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2lhj4dz48fz3rsao.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2q357qg40ikg5yzl.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2tn5jqobibct7dae.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.2vs7yb1bsuliqotr.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.373l6kebgkoh5bqv.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.3hc2jzy2fujzimsg.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.3opz8ssjooq2n2jc.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.3zm4roez5gafffjn.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n_periodic-091ad608a3fb4aed.496swhrgphpf8icu.rcgu.o" "/home/username/Documents/Projects/Calculator/n-periodic/target/armv5te-nspire-eabi/debug/deps/n
  = note: arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicBool as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN67_$LT$core..sync..atomic..AtomicBool$u20$as$u20$core..fmt..Debug$GT$3fmt17h2409f700d8cc79f4E+0x10): undefined reference to `__sync_val_compare_and_swap_1'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicI8 as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN65_$LT$core..sync..atomic..AtomicI8$u20$as$u20$core..fmt..Debug$GT$3fmt17ha561321bae6b6b2bE+0x18): undefined reference to `__sync_val_compare_and_swap_1'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicU8 as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN65_$LT$core..sync..atomic..AtomicU8$u20$as$u20$core..fmt..Debug$GT$3fmt17hf52d3dfa9ec687daE+0x18): undefined reference to `__sync_val_compare_and_swap_1'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicI16 as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN66_$LT$core..sync..atomic..AtomicI16$u20$as$u20$core..fmt..Debug$GT$3fmt17h105deeed344b8d2eE+0x18): undefined reference to `__sync_val_compare_and_swap_2'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicU16 as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN66_$LT$core..sync..atomic..AtomicU16$u20$as$u20$core..fmt..Debug$GT$3fmt17hdb14784a9235bd9bE+0x18): undefined reference to `__sync_val_compare_and_swap_2'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicIsize as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN66_$LT$core..sync..atomic..AtomicI32$u20$as$u20$core..fmt..Debug$GT$3fmt17hc7b07211c37c1defE+0x18): undefined reference to `__sync_val_compare_and_swap_4'
          arm-none-eabi-ld: /home/username/Documents/Projects/Calculator/n-periodic/target/sysroot/lib/rustlib/armv5te-nspire-eabi/lib/libcore-eb37d88bc53bf3ca.rlib(core-eb37d88bc53bf3ca.core.85mjtqkh-cgu.10.rcgu.o): in function `<core::sync::atomic::AtomicUsize as core::fmt::Debug>::fmt':
          core.85mjtqkh-cgu.10:(.text._ZN66_$LT$core..sync..atomic..AtomicU32$u20$as$u20$core..fmt..Debug$GT$3fmt17h2ba30e716d00887fE+0x18): undefined reference to `__sync_val_compare_and_swap_4'
          collect2: error: ld returned 1 exit status

[ericseppanen]

I believe __sync_val_compare_and_swap_* are compiler intrinsics/builtins.

I think that means even though this error appears at the linker stage, it's not the linker's fault: the compiler should have substituted the relevant inline code already.

[ericseppanen]

Hm; maybe things aren't that simple. LLVM's behavior seems to be arch-specific:
https://llvm.org/docs/Atomics.html#libcalls-sync

We probably can't conclude that this linker error means the same thing on different platforms. While on x86_64 I expect some inline assembly, it looks like some other platforms may emit an actual separate function or even a library call.

[jonas-schievink]

Unlike with https://github.com/rust-lang/rust/issues/56798, I'd expect no such function call to be generated at all in this case (since the code apparently doesn't make use of atomics).

@lights0123 can you perhaps provide instructions for reproducing this issue, preferably with minimized code?

[Lokathor]

I hit this case when building for ARMv4T (using cargo-xbuild), despite my program not using any of the AtomicFoo types at all.

[lights0123]

@Lokathor I fixed it with a quick fix: https://github.com/lights0123/ndless-handler/blob/3f3f3150860926e0622e394dbaeea32865c7663f/src/lib.rs#L62

Adding "singlethread": true to my target.json may have helped too.

[Lokathor]

I had some dummy impls, but "singlethread": true seems to have also fixed it.

[lights0123]

@jonas-schievink MCVE: https://github.com/lights0123/rustc-58500

[spastorino]

Tagged as P-medium until we know how specific the target specification has to be for this error to show up.

[ollie27]

I think the correct solution is to add:

"max-atomic-width": 0

to the target JSON.

I don't think there's any bug here. If a target doesn't have native support for atomics then that either needs to be specified in the target JSON as above or implementations for the __sync_* functions need to be provided to emulate them.

[lights0123]

@ollie27 that appeared to fix it (Although I'll probably stick with "singlethread": true for my platform as I don't have threads or interrupts anyways so I can use Arc with Wakers for Future).

I would say that this should be documented somewhere, but there doesn't seem to be a place where the target.json format is really documented other than the compiler docs. There's a single paragraph in the embedonomicon, but everything about target.json files is really is community blog posts. Should this be documented somewhere?

[Lokathor]

If there is no place to document it then we need to make a place and then document it there.

[lights0123]

I'll send a PR probably tomorrow to add a new page to the embedonomicon all about them, as there's no other place really about it.

[lights0123]

Can someone check that my notes about atomics are correct in rust-embedded/embedonomicon#67? I believe that most of what I said is true, but I'm not sure if manually implementing these functions is correct:

• Configure atomics. Pick the first option that describes your target:
  
  
  
  • I have a single-core processor, no threads, no interrupts, or any way for multiple things to be happening in parallel: if you are sure that is the case, such as WASM (for now), you may set "singlethread": true. This will configure LLVM to convert all atomic operations to use their single threaded counterparts.
  
  
  • I have native atomic operations: set max-atomic-width to the biggest type in bits that your target can operate on atomically. For example, many ARM cores have 32-bit atomic operations. You may set "max-atomic-width": 32 in that case.
  
  
  • I have no native atomic operations, but I can emulate them myself: set max-atomic-width to the highest number of bits that you can emulate up to 64, then implement all of the atomic and sync functions expected by LLVM as #[no_mangle] unsafe extern "C". These functions have been standardized by gcc, so the gcc documentation may have more notes. Missing functions will cause a linker error, while incorrectly implemented functions will possibly cause UB.
  
  
  • I have no native atomic operations: you'll have to do some unsafe work to manually ensure synchronization in your code. You must set "max-atomic-width": 0.
  
  

[ollie27]

* to the highest number of bits that you can emulate up to 64,

It can actually go up to 128.

[Lokathor]

So if I have 1 core, and no threads, and no native atomics, but I do have interrupts, then I guess I'm the last bullet point?

perhaps that case needs to be made more explicit

[lights0123]

@Lokathor i just made that up (although the phil-opp blog does use atomic for single-core interrupts), but it does makes sense. A non-atomic increment looks like:

1. Fetch the value, 0
2. Add 1 to it (now 1)
3. Store it, 1, back

If you have an interrupt in there, you might get:

1. Fetch the value, 0
2. Interrupt!
   
   
   
   1. Fetch the value, 0
   
   
   2. Add 1 to it (now 1)
   
   
   3. Store it, 1, back
   
   
3. Add 1 to it (now 1)
4. Store it, 1, back

You could manually implement atomic instructions though to disable interrupts, do the operation, and then re-enable them.

[lights0123]

So if I have 1 core, and no threads, and no native atomics, but I do have interrupts, then I guess I'm the last bullet point?

perhaps that case needs to be made more explicit

@Lokathor bolded "no interrupts"

[Lokathor]

right, your list has bullet 1 with "no interrupts" required, but i do have interrupts, so i guess that makes me fall to the "otherwise bullet 4" case.

[lights0123]

Yep, I just took your suggestion to make it more obvious.

[lights0123]

:tada: that page is now live at https://rust-embedded.github.io/embedonomicon/custom-target.html! I'll consider this fixed (although it's not live at the main embedonomicon page yet, I'm hoping that'll get automatically pushed).