Linux: alignment must be power of 2

[nickdesaulniers]

ld.lld: error: ./arch/x86/kernel/vmlinux.lds:185: alignment must be power of 2

after applying every workaround for every known issue we have with LLD, this is the final error I hit (may not be reproducible once we fix the rest). Not sure about the source line, since it looks the C preprocessor gets run on this linker script (hence the .lds suffix).

[nickdesaulniers]

Might be orc unwinder related, the preprocessed source L185 looks like:

. = ALIGN(4); .orc_unwind_ip : AT(ADDR(.orc_unwind_ip) - 0xffffffff80000000) { __start_orc_u    nwind_ip = .; KEEP(*(.orc_unwind_ip)) __stop_orc_unwind_ip = .; } . = ALIGN(6); .orc_unwind :     AT(ADDR(.orc_unwind) - 0xffffffff80000000) { __start_orc_unwind = .; KEEP(*(.orc_unwind)) __    stop_orc_unwind = .; } . = ALIGN(4); .orc_lookup : AT(ADDR(.orc_lookup) - 0xffffffff80000000)     { orc_lookup = .; . += (((SIZEOF(.text) + (1 << 8) - 1) / (1 << 8)) + 1) * 4; orc_lookup_end     = .; }

[tpimh]

I was able to link the kernel with the following fix:

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 3d7a6a9c2370..f687c834810c 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -733,7 +733,7 @@
        KEEP(*(.orc_unwind_ip))                 \
        __stop_orc_unwind_ip = .;               \
    }                               \
-   . = ALIGN(6);                           \
+   . = ALIGN(8);                           \
    .orc_unwind : AT(ADDR(.orc_unwind) - LOAD_OFFSET) {     \
        __start_orc_unwind = .;                 \
        KEEP(*(.orc_unwind))                    \
-- 
2.19.1

I'm sure this is not how it should be done and LLVM side should be fixed instead, but at least I was able to compile the kernel and link it with LLD.

[E5ten]

I get the same error even when I apply that patch

[E5ten]

Actually it's not quite the same error, I get

ld.lld: error: <internal>: section sh_addralign is not a power of 2

[dileks]

Workaround this by:

scripts/config -d CRYPTO_SHA512_SSSE3

CRYPTO_SHA512_SSSE3 m -> n

[E5ten]

Any idea what in the sha512_ssse3 module is causing problems?

[dileks]

No.
When building with make V=1 CC=clang-9 HOSTCC=clang-9 LD=ld.lld -j1 I saw something similiar like here.
Currently, my focus is to get the beast built with ld.lld (and boot on bare metal).
I am still building (maybe I hit this one later).

[dileks]

Hurrying on to the train (compile/link tested only):

$ readelf --string-dump .comment vmlinux

String dump of section '.comment':
  [     0]  Linker: LLD 9.0.0 (https://github.com/llvm-mirror/lld.git 7013b9a9ac88072094243c15349dffa910b85cfd)
  [    64]  clang version 9.0.0 (https://github.com/llvm-mirror/clang.git 6a9d9dddc230e461a95921b306558553c955f010) (https://github.com/llvm-mirror/llvm.git c08de5fe639302cec42bd72d2a30b0ce22797e66)

[nickdesaulniers]

Re: https://github.com/ClangBuiltLinux/linux/issues/218#issuecomment-439202811, I'd be curious to know if git blame of that line points to a commit that explains why alignment is not a power of 2? I'm curious if there's a good reason for it; then LLD should not error on NPOT (non-power-of-two) ALIGN statements, or if the kernel is wrong here?

[nathanchance]

Introduced in ee9f8fce99640811b2b8e79d0d1dbe8bab69ba67 by @jpoimboe.

[jpoimboe]

Hm, speaking as the author of that code, that 6-byte alignment shouldn't be necessary and we can change it to 8. Let me do some testing and then I'll post the patch upstream soon-ish.

[jpoimboe]

The reason it's aligned at 6 bytes, is because there's a packed array of 6-byte structs immediately after. So maybe 6-byte alignment should be allowed for this type of purpose? I don't really care much anyway, though it will waste up to two bytes for me to change it :-)

[nickdesaulniers]

@GeorgiiR @rui314 I assume lld just errors as a precaution when the ALIGNment is a NPOT, but the underlying machinery would handle the case just fine without the warning?

[GeorgiiR]

@GeorgiiR @rui314 I assume lld just errors as a precaution when the ALIGNment is a NPOT, but the underlying machinery would handle the case just fine without the warning?

If you ignore the error somehow, LLD would align to a dummy POT value then, which is 1 currently:

    if (!isPowerOf2_64(Alignment)) {
      error(Loc + ": alignment must be power of 2");
      return (uint64_t)1; // Return a dummy value.
    }

See #1
See #2

[nickdesaulniers]

right, so if that condition was removed (ex. number 2), then would LLD proceed correctly? I think it's being too strict, but maybe there's a good reason like something not being implemented that's not immediately obvious?

[GeorgiiR]

right, so if that condition was removed (ex. number 2), then would LLD proceed correctly?

This was never tested, but I know/see no obvious reasons for failure atm. Perhaps it will just work.

[dileks]

With the patch of @tpimh from here and "x86/build: Mark per-CPU symbols as absolute explicitly for LLD" from @nickdesaulniers I could build with LLD but cannot boot in QEMU (see attached log-file).

[ run_qemu.sh ]
KPATH=$(pwd)

qemu-system-x86_64 -enable-kvm -M pc -kernel $KPATH/bzImage -initrd $KPATH/initrd.img -m 512 -net none -serial stdio -append "root=/dev/ram0 console=ttyS0 hung_task_panic=1 earlyprintk=ttyS0,115200"

I tried to turn off acpi / pci / pti etc. None really helped.

qemu-log.txt

[dileks]

@E5ten Building/Linking looks good (no alignment must be power of 2 breakage):

$ ./scripts/diffconfig /boot/config-5.0.0-rc8-2-amd64-cbl-asmgoto .config
 BUILD_SALT "5.0.0-rc8-2-amd64-cbl-asmgoto" -> "5.0.0-rc8-3-amd64-cbl-asmgoto"
 CRYPTO_SHA512 m -> y
 CRYPTO_SHA512_SSSE3 n -> y

[E5ten]

@dileks ld.lld: error: <internal>: section sh_addralign is not a power of 2 I still get this when I enable CRYPTO_SHA512_SSSE3.
EDIT: and when I disable it and successfully link the kernel it fails to boot for me as well.

[GeorgiiR]

It is illegal for sh_addralign to be NPOT I think.
(https://docs.oracle.com/cd/E19455-01/806-3773/elf-2/index.html):

If a section has an address alignment constraint, the value in this field is the modulus, in byte units, by which the value of sh_addr must be congruent to 0; i.e., sh_addr = 0 (mod sh_addralign). For example, if a section contains a long (32 bits), the entire section must be ensured long alignment, so sh_addralign has the value 4. Only 0 and positive integral powers of 2 are currently allowed as values for this field. A value of 0 or 1 indicates no address alignment constraints.

[dileks]

@E5ten can you join #clangbuiltlinux @ freenode?

[GeorgiiR]

This should make an LLD error message to be a bit more meaningful:
https://reviews.llvm.org/D58670

[dileks]

@GeorgiiR How can I run a single test like test/ELF/invalid/section-alignment2.s to check (with) the new generated lld binary?

[GeorgiiR]

@dileks
llvm-lit
e.g.
~/LLVM/build_lldb/bin/llvm-lit /home/umb/LLVM/llvm/tools/lldb/lit/Breakpoint/case-sensitive.test

[dileks]

@GeorgiiR

$ cd $BUILD_DIR
$ ./bin/llvm-lit ../llvm/tools/lld/test/ELF/invalid/section-alignment2.s
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:337: note: using ld.lld: /home/sdi/src/llvm-toolchain/build/bin/ld.lld
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:337: note: using lld-link: /home/sdi/src/llvm-toolchain/build/bin/lld-link
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:337: note: using ld64.lld: /home/sdi/src/llvm-toolchain/build/bin/ld64.lld
llvm-lit: /home/sdi/src/llvm-toolchain/llvm/utils/lit/lit/llvm/config.py:337: note: using wasm-ld: /home/sdi/src/llvm-toolchain/build/bin/wasm-ld
-- Testing: 1 tests, single process --
PASS: lld :: ELF/invalid/section-alignment2.s (1 of 1)
Testing Time: 0.02s
  Expected Passes    : 1

[dileks]

@E5ten

#!/bin/sh

export LANG=C
export LC_ALL=C

cd build

# CBL issue #218
# https://github.com/ClangBuiltLinux/linux/issues/218
# https://reviews.llvm.org/D58670 ("[LLD][ELF] - Improve "sh_addralign is not a power of 2" diagnostics.")
LLD_TESTS="ELF/invalid/section-alignment2.s"

for t in $LLD_TESTS ; do ./bin/llvm-lit -v ./tools/lld/test/$t ; done

[dileks]

With @GeorgiiR D58670 diff 188357 patch I see the following:

  LD      vmlinux.o
+ modpost_link vmlinux.o
+ local objects
+ objects='--whole-archive                                              built-in.a                                                      --no-whole-archive                                              --start-group                        lib/lib.a  arch/x86/lib/lib.a                                           --end-group'
+ ld.lld -m elf_x86_64 -z max-page-size=0x200000 -O2 -r -o vmlinux.o --whole-archive built-in.a --no-whole-archive --start-group lib/lib.a arch/x86/lib/lib.a --end-group
ld.lld: error: <internal>:(.rodata.cst640.K512): section sh_addralign is not a power of 2
+ on_exit

[dileks]

@E5ten CONFIG_CRYPTO_SHA512_SSSE3 (as module or builtin) is still show-stopper.
This works:

$ ./scripts/diffconfig /boot/config-5.0.0-rc8-2-amd64-cbl-asmgoto .config
 BUILD_SALT "5.0.0-rc8-2-amd64-cbl-asmgoto" -> "5.0.0-rc8-3-amd64-cbl-asmgoto"
 CRYPTO_SHA512 m -> y

[dileks]

@GeorgiiR Patch now in LLD upstream Git.

" [LLD][ELF] - Improve "sh_addralign is not a power of 2" diagnostics."

[1] https://github.com/llvm-mirror/lld/commit/4b56aa1945588edb213c3e5caa8dab52ea966e04

[dileks]

@jpoimboe Any news (patch) on the 6 vs. 8 alignment?

[jpoimboe]

@dileks can you (or someone else) confirm that there are no plans to support 6-byte alignment? I think my use case is valid but if you guys don't want to support it then I'm fine changing to 8 bytes.

[dileks]

@jpoimboe I can confirm that 8-byte-alignment works with both BFD (boots in QEMU) and LLD (doesnot boot in QEMU, might have other causes like NPOT). Better get an opinion of @GeorgiiR?

[jpoimboe]

Hm, on second thought, I don't think 6-byte alignment is actually a thing. 2-byte alignment is probably what I'm looking for. I will work up a kernel patch to change it. Sorry for the noise.

[dileks]

@jpoimboe
Thanks for the patch "x86/unwind/orc: Fix ORC unwind table alignment".

Can you add...
Link: https://github.com/ClangBuiltLinux/linux/issues/218
and
CC: [email protected]
and some Reported-by#s?

I will test it.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/commit/?h=orc-alignment&id=b730db28e454131cf0ec7f456121a0a5e41e5995

[jpoimboe]

Sure, who shall I say it was reported by?

[dileks]

I just looked in the CI open issues, so the issues with LLD as linker are 3 months known.

I see what I fixed locally by...

"x86/realmode: Add elf_i386 linker emulation and emit relocations when CONFIG_X86_NEED_RELOCS=y"

...seems to be identical what @tpimh has.

I can only speak for myself as I jumped 14 days ago into substituting BFD by LLD.

Independently Reported-by: Sedat Dilek [e-mail]

[1] https://github.com/ClangBuiltLinux/continuous-integration/issues/24

[nickdesaulniers]

https://github.com/ClangBuiltLinux/linux/issues/218#issue-369974844 is the earliest report. :smirk:

[jpoimboe]

I have added Nick and Sedat. Anybody else who wants to be credited as a reporter, let me know.

[dileks]

Tested-by: Sedat Dilek sedat.[email protected]

[tpimh]

Oh, no. I'm wrong, original reporter is @nickdesaulniers.

[jpoimboe]

@tpimh No worries, there can be multiple reporters. I added you as well. Sending the patch out shortly.

[jpoimboe]

Patch posted upstream:
https://lkml.kernel.org/r/[e-mail]

[nickdesaulniers]

Thanks again @jpoimboe . Looks like tglx picked it up: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/commit/?h=x86/urgent&id=f76a16adc485699f95bb71fce114f97c832fe664

[nathanchance]

Merged into mainline: https://git.kernel.org/torvalds/c/f76a16adc485699f95bb71fce114f97c832fe664