Linux: missing symbol for section .text, .init.text, .ref.text

[tpimh]

I'm trying to compile x86_64 kernel using AS=clang (integrated-as in being used). A lot of warnings like these are produced:

warning: objtool: missing symbol for section .text
warning: objtool: missing symbol for section .init.text
warning: objtool: missing symbol for section .ref.text

The kernel compiles successfully, but doesn't boot.

[nickdesaulniers]

cc @jpoimboe

Does the warning mean that a symbol is undefined, or that that section is missing? I can see both being problematic.

Might be interesting to try $ make CC=gcc AS=clang.

[jpoimboe]

@nickdesaulniers I think it just means that the section symbols associated with those sections are missing. It's not necessarily a bug, I'd need to see the .o file to know more.

[jcai19]

Sharing some of my discoveries while fiddling with one of the instances, arch/x86/ia32/audit.c.

When built with gcc, llvm-readelf -s returned the following symbols,

Symbol table '.symtab' contains 13 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS audit.c
2: 0000000000000000 0 SECTION LOCAL DEFAULT 1 .text
3: 0000000000000000 0 SECTION LOCAL DEFAULT 2 .data
4: 0000000000000000 0 SECTION LOCAL DEFAULT 3 .bss
5: 0000000000000000 0 SECTION LOCAL DEFAULT 5 .note.GNU-stack
6: 0000000000000000 0 SECTION LOCAL DEFAULT 4 .comment
7: 0000000000000000 71 FUNC GLOBAL DEFAULT 1 ia32_classify_syscall
8: 0000000000000000 16 OBJECT GLOBAL DEFAULT 2 ia32_signal_class
9: 0000000000000020 40 OBJECT GLOBAL DEFAULT 2 ia32_read_class
10: 0000000000000060 100 OBJECT GLOBAL DEFAULT 2 ia32_write_class
11: 00000000000000e0 76 OBJECT GLOBAL DEFAULT 2 ia32_chattr_class
12: 0000000000000140 64 OBJECT GLOBAL DEFAULT 2 ia32_dir_class

And objdump -h returns

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000047 0000000000000000 0000000000000000 00000040 24
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000180 0000000000000000 0000000000000000 000000a0 25
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 0000000000000000 0000000000000000 00000220 20
ALLOC
3 .comment 00000027 0000000000000000 0000000000000000 00000220 20
CONTENTS, READONLY
4 .note.GNU-stack 00000000 0000000000000000 0000000000000000 00000247 20
CONTENTS, READONLY
5 .orc_unwind_ip 00000008 0000000000000000 0000000000000000 00000467 20
CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
6 .orc_unwind 0000000c 0000000000000000 0000000000000000 000004a0 2**0

With clang it became

Symbol table '.symtab' contains 8 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS audit.c
2: 0000000000000040 76 OBJECT GLOBAL DEFAULT 3 ia32_chattr_class
3: 0000000000000000 65 FUNC GLOBAL DEFAULT 2 ia32_classify_syscall
4: 0000000000000000 64 OBJECT GLOBAL DEFAULT 3 ia32_dir_class
5: 0000000000000100 40 OBJECT GLOBAL DEFAULT 3 ia32_read_class
6: 0000000000000130 16 OBJECT GLOBAL DEFAULT 3 ia32_signal_class
7: 0000000000000090 100 OBJECT GLOBAL DEFAULT 3 ia32_write_class

and

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000041 0000000000000000 0000000000000000 00000040 24
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000140 0000000000000000 0000000000000000 00000090 24
CONTENTS, ALLOC, LOAD, DATA
2 .comment 0000006a 0000000000000000 0000000000000000 000001d0 20
CONTENTS, READONLY
3 .note.GNU-stack 00000000 0000000000000000 0000000000000000 0000023a 20
CONTENTS, READONLY
4 .llvm_addrsig 00000000 0000000000000000 0000000000000000 00000300 2**0
CONTENTS, READONLY, EXCLUDE

So there are two differences: (1) the object files from the same source file built with clang and gcc respectively have different sections, (2) clang does not include symbols of some sections.

[ihalip]

LLVM no longer emits section symbols after this commit.

[jcai19]

Thank you for the pointers! They are really helpful.

[nickdesaulniers]

So a small update here, for x86, if I fix #913 , on mainline I can assemble an x86 kernel with clang, and boot despite these warnings.

For -next, these warnings become errors, so the build breaks. Since the merge window for 5.7 will likely open soon, this build will not work for long.

[jcai19]

I will see if we can disable these warnings in Clang.

[nickdesaulniers]

@jcai19 these warnings aren't coming from Clang. They're coming from objtool, which is custom object file validator host utility written and maintained by @jpoimboe .

Attached is a .o file; you can see it's missing section symbols as noted by @jcai19 above
kernel_freezer.o.txt

Two ways I can work around this:

1. allow NULL symbols in ORC entries:

diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c
index cc4601c879ce..67e525fc19e8 100644
--- a/tools/objtool/elf.c
+++ b/tools/objtool/elf.c
@@ -625,7 +625,9 @@ int elf_rebuild_rela_section(struct section *sec)
        list_for_each_entry(rela, &sec->rela_list, list) {
                relas[idx].r_offset = rela->offset;
                relas[idx].r_addend = rela->addend;
-               relas[idx].r_info = GELF_R_INFO(rela->sym->idx, rela->type);
+               if (rela->sym) {
+                       relas[idx].r_info = GELF_R_INFO(rela->sym->idx, rela->type);
+               }
                idx++;
        }

diff --git a/tools/objtool/orc_gen.c b/tools/objtool/orc_gen.c
index 27a4112848c2..2f84dca5b8b0 100644
--- a/tools/objtool/orc_gen.c
+++ b/tools/objtool/orc_gen.c
@@ -88,11 +88,6 @@ static int create_orc_entry(struct section *u_sec, struct section *ip_relasec,
        struct orc_entry *orc;
        struct rela *rela;

-       if (!insn_sec->sym) {
-               WARN("missing symbol for section %s", insn_sec->name);
-               return -1;
-       }
-
        /* populate ORC data */
        orc = (struct orc_entry *)u_sec->data->d_buf + idx;
        memcpy(orc, o, sizeof(*orc));

Kernel builds and boots, seems to unwind when panic'ed. or:

1. don't create orc entries for NULL symbols.

diff --git a/tools/objtool/orc_gen.c b/tools/objtool/orc_gen.c
index 27a4112848c2..c7b497199608 100644
--- a/tools/objtool/orc_gen.c
+++ b/tools/objtool/orc_gen.c
@@ -89,8 +89,7 @@ static int create_orc_entry(struct section *u_sec, struct section *ip_relasec,
        struct rela *rela;

        if (!insn_sec->sym) {
-               WARN("missing symbol for section %s", insn_sec->name);
-               return -1;
+               return 0;
        }

        /* populate ORC data */

builds, boots, unwinds.

Though maybe this is not creating entries in .orc_unwind for symbols defined in assembly, which I guess would mean would could unwind through C code, but would not be able to unwind through out of line assembly code?

$ make CC=clang LD=ld.lld -j71 kernel/freezer.o
$ llvm-readelf -S kernel/freezer.o
...
  [21] .orc_unwind_ip    PROGBITS        0000000000000000 0012d8 00009c 04   A  0   0  1
  [22] .rela.orc_unwind_ip RELA          0000000000000000 001378 0003a8 18   I 18  21  8
  [23] .orc_unwind       PROGBITS        0000000000000000 001720 0000ea 06   A  0   0  1
# compared to `return 0;` case 2 above
$ make CC=clang AS=clang LD=ld.lld -j71 kernel/freezer.o
$ llvm-readelf -S kernel/freezer.o
...
  [20] .orc_unwind_ip    PROGBITS        0000000000000000 001210 00009c 04   A  0   0  1
  [21] .rela.orc_unwind_ip RELA          0000000000000000 0012b0 000000 18   I 19  20  8
  [22] .orc_unwind       PROGBITS        0000000000000000 0012b0 0000ea 06   A  0   0  1

so didn't seem to make a difference in the sizes of .orc_unwind_ip or .orc_unwind_ip, but it seems that .rela.orc_unwind was empty. I don't really understand the implications of .rela.orc_unwind_ip being empty, but I suspect something may break. Thoughts @jpoimboe ?

[jcai19]

@nickdesaulniers thanks for pointing that out! I worked on this a while ago and my memory apparently failed me.

[jpoimboe]

@nickdesaulniers actually 1 will corrupt the ORC table and 2 will just leave it empty.

Instead of trying to remove the relocations (or ORC entries), we should teach objtool how to create relocations for code which doesn't have a section symbol for its section. Does this fix it?

https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/commit/?h=clang-orc-no-section-symbol&id=2aab45ff4714855406682d3e673cacd1af43dd16

@tpimh if you give me your email address I can give you a Reported-by credit in the kernel commit message.

[nathanchance]

Does this fix it?

Yes:

$ git log --oneline origin/master^.. | cat
8f436c524b78 objtool: Support missing section symbols in ORC generation
43627088f16e elfnote: mark all .note sections SHF_ALLOC
9959d8cda884 Makefile.llvm: simplify LLVM build
89295c59c1f0 Add linux-next specific files for 20200326

$ make -j$(nproc) -s LLVM=y O=out/x86_64 distclean defconfig all
...

$ ~/cbl/git/boot-utils/boot-qemu.sh -a x86_64 -k out/x86_64
...
[    0.000000] Linux version 5.6.0-rc7-next-20200326+ (nathan@ubuntu-m2-xlarge-x86) (ClangBuiltLinux clang version 11.0.0 (git://github.com/llvm/llvm-project d264f02c6f502960e2bcdd332f250efc702d09f2)) #1 SMP Thu Mar 26 21:36:54 MST 2020
...

Tested-by: Nathan Chancellor <[email protected]>

[tpimh]

@jpoimboe Reported-by: Dmitry Golovin <[email protected]>

[nickdesaulniers]

Looks like @jpoimboe has an updated patch with commit message: https://git.kernel.org/pub/scm/linux/kernel/git/jpoimboe/linux.git/commit/?h=objtool-clang-orc-section-symbol

[nickdesaulniers]

2 patches 0 1

[nathanchance]

Merged into mainline:

https://git.kernel.org/linus/8782e7cab51b6bf01a5a86471dd82228af1ac185

https://git.kernel.org/linus/e81e0724432542af8d8c702c31e9d82f57b1ff31

[nickdesaulniers]

Requested backports in https://lore.kernel.org/stable/CAKwvOdn78WAUiRtyPxW7oEhUz8GN6MkL=[e-mail]/T/#u