Linux: MIPS malta_defconfig kernel does not boot when built with clang

[nathanchance]

With the pushed branch (link), a GCC built kernel loads and boots just fine (before panicking because of no rootfs):

$ make -s -j$(nproc) ARCH=mips CROSS_COMPILE=mipsel-linux-gnu- O=out-gcc distclean malta_defconfig all
...
$ timeout 2m unbuffer qemu-system-mipsel -kernel out-gcc/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio
Linux version 5.2.0-11506-g8e8061fc0141 (root@4d223cab2337) (gcc version 8.3.0 (Debian 8.3.0-2)) #1 SMP Thu Jul 18 06:16:57 UTC 2019
...

However, a clang built kernel appears to hang and shows no output.

$ make -s -j$(nproc) ARCH=mips CC=clang CROSS_COMPILE=mipsel-linux-gnu- O=out-clang distclean malta_defconfig all
...
$ timeout 2m unbuffer qemu-system-mipsel -kernel out-clang/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio
$ echo $?
124

[tpimh]

Did you apply patches in order to get mips kernel built? If yes, please share them.

[nathanchance]

Yes, I linked to them in that post: https://github.com/ClangBuiltLinux/linux/commits/mips

[nickdesaulniers]

Where is it going?
https://nickdesaulniers.github.io/blog/2018/10/24/booting-a-custom-linux-kernel-in-qemu-and-debugging-it-with-gdb/

[nathanchance]

I am attempting to debug this with GDB but I feel like I am doing something wrong...

In one tmux window:

$ qemu-system-mipsel -kernel /out-clang/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio -s -S

In another:

$ gdb-multiarch /out-clang/vmlinux
GNU gdb (Debian 8.2.1-2) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /out-clang/vmlinux...done.
(gdb) set arch mips
The target architecture is assumed to be mips
(gdb) set endian little
The target is assumed to be little endian
(gdb) target remote :1234
Remote debugging using :1234
0xbfc00000 in ?? ()
(gdb) hbreak start_kernel
Hardware assisted breakpoint 1 at 0x808f43a0: file /linux/init/main.c, line 580.
(gdb) c
Continuing.

Breakpoint 1, start_kernel () at /linux/init/main.c:580
580             set_task_stack_end_magic(&init_task);
(gdb) c
Continuing.

Breakpoint 1, start_kernel () at /linux/init/main.c:580
580             set_task_stack_end_magic(&init_task);
(gdb) lx-version
Linux version 5.2.0-12302-g9c4754dbce85-dirty (root@4d223cab2337) (ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 544315b4197034a3be8acd12cba56a75fb1f08dc) (based on LLVM 9.0.0svn)) #1 SMP Sat Jul 20 00:00:17 UTC 2019

Is the fact that it is not continuing the bug? Trying to run lx-dmesg does nothing (presumably because there is nothing to log just yet).

[gwelymernans]

Did you try stepping through the code to see how far it gets?

[tpimh]

Can it possibly be an infinite loop at the very start of the kernel?

[nathanchance]

Seems like it might be but I'll need to look further into it.

[ihalip]

Boots for me up until the "Cannot open root device" error message: https://gist.github.com/ihalip/495c9773eb9ce9eadec7b8fa71b0d004

Built with clang-8.

[nickdesaulniers]

@ihalip what defconfig did you use? Looks like you specified an initrd, so you shouldn't need root= kernel command line (not sure if a kernel config needs to be set to use a ramdisk though).

[nathanchance]

I can confirm that clang-8 + https://github.com/llvm/llvm-project/commit/7f308af5eeea2d1b24aee0361d39dc43bac4cfe5 works.

Time to bisect :)

[nathanchance]

Hmmm... https://github.com/llvm/llvm-project/commit/d748689c7f718f531871bb44b5da05888c3c0301

d748689c7f718f531871bb44b5da05888c3c0301 is the first bad commit
commit d748689c7f718f531871bb44b5da05888c3c0301
Author: Philip Reames <[email protected]>
Date:   Mon Apr 22 20:28:19 2019 +0000

    [InstCombine] Eliminate stores to constant memory

    If we have a store to a piece of memory which is known constant, then we know the store must be storing back the same value. As a result, the store (or memset, or memmove) must either be down a dead path, or a noop. In either case, it is valid to simply remove the store.

    The motivating case for this involves a memmove to a buffer which is constant down a path which is dynamically dead.

    Note that I'm choosing to implement the less aggressive of two possible semantics here. We could simply say that the store *is undefined*, and prune the path. Consensus in the review was that the more aggressive form might be a good follow on change at a later date.

    Differential Revision: https://reviews.llvm.org/D60659

    llvm-svn: 358919

:040000 040000 89bcbd37a5e3abf0006787c85c7043c6268e529e 67cb055e634cffaa2d17f16a7712f82cd69e4705 M      llvm

$ git bisect log
git bisect start              
# good: [7b5565418f4d6e113ba805dad40d471d23bca6f6] Fix build breakage from llvm r351317
git bisect good 7b5565418f4d6e113ba805dad40d471d23bca6f6
# bad: [7f308af5eeea2d1b24aee0361d39dc43bac4cfe5] [mips] Support the "o" inline asm constraint
git bisect bad 7f308af5eeea2d1b24aee0361d39dc43bac4cfe5
# bad: [7f308af5eeea2d1b24aee0361d39dc43bac4cfe5] [mips] Support the "o" inline asm constraint
git bisect bad 7f308af5eeea2d1b24aee0361d39dc43bac4cfe5
# bad: [bf8aacb7151cf3ed569c4907c934c45244f6914f] [SelectionDAG] move splat util functions up from x86 lowering
git bisect bad bf8aacb7151cf3ed569c4907c934c45244f6914f
# good: [25f718e9f847e90597e11da5b8b18d072696d930] Delete commented-out code.
git bisect good 25f718e9f847e90597e11da5b8b18d072696d930
# good: [9f1a7e559ceac80fc2a2956a3e005bc1d81fcd68] Rename some variables in the std-module tests
git bisect good 9f1a7e559ceac80fc2a2956a3e005bc1d81fcd68
# good: [23e23836d5d0dcb90b56eb8f479f67d29e60a5b6] Move tests in r357996 to correct location within monorepo
git bisect good 23e23836d5d0dcb90b56eb8f479f67d29e60a5b6
# good: [b068f92d94a5c42ff21bfdf29344c54e2688dac3] DebugInfo: Default to standalone debug when tuning for LLDB
git bisect good b068f92d94a5c42ff21bfdf29344c54e2688dac3
# good: [d573aa0156b6aaf0e3cff13958f80d7c227cd656] [NFC] FMF propagation for GlobalIsel
git bisect good d573aa0156b6aaf0e3cff13958f80d7c227cd656
# good: [dd0e83355583898d19357abc5f17a9a94f057d74] [llvm-symbolizer] Fix section index at the end of a section
git bisect good dd0e83355583898d19357abc5f17a9a94f057d74
# good: [bc4b159bb1121ed9e1c00df35a4f5dd5712de8de] [ELF][X86] Allow R_386_TLS_LDO_32 and R_X86_64_DTPOFF{32,64} to preemptable local-dynamic symbols
git bisect good bc4b159bb1121ed9e1c00df35a4f5dd5712de8de
# good: [55043e2336c07e790f4e879e217cb561e4f6b3f5] [sema][objc] Minor refactor to OverrideSearch. NFCI.
git bisect good 55043e2336c07e790f4e879e217cb561e4f6b3f5
# good: [d8d9b7b20e7b8d7a79a069ddb78e3d9ea48b1a45] [InstSimplify] Move masked.gather w/no active lanes handling to InstSimplify from InstCombine
git bisect good d8d9b7b20e7b8d7a79a069ddb78e3d9ea48b1a45
# bad: [0d809aa21842dfe7ef6a37d8b11cfd0accc763ad] [dsymutil] Collect parseable Swift interfaces in the .dSYM bundle.
git bisect bad 0d809aa21842dfe7ef6a37d8b11cfd0accc763ad
# good: [32176baee78d145e2a02a9fda4ce906584c6a115] [Reproducers] Fix lifetime issue
git bisect good 32176baee78d145e2a02a9fda4ce906584c6a115
# good: [2a27af82379e854ee1b0c5aa7fad6b430eab4b55] [EditLineTest] Not always TERM is available, e.g. on some bots.
git bisect good 2a27af82379e854ee1b0c5aa7fad6b430eab4b55
# bad: [d748689c7f718f531871bb44b5da05888c3c0301] [InstCombine] Eliminate stores to constant memory
git bisect bad d748689c7f718f531871bb44b5da05888c3c0301
# first bad commit: [d748689c7f718f531871bb44b5da05888c3c0301] [InstCombine] Eliminate stores to constant memory

I can confirm that reverting that commit on top of tip of tree LLVM and using the following commands to build and boot works fine with https://github.com/ClangBuiltLinux/linux/commits/mips (aside from the rootfs panic):

$ /tc-build/build-llvm.py --build-stage1-only -p "clang" -t "Mips;X86"
...
$ PATH=/tc-build/build/llvm/stage1/bin:${PATH} make -C /linux -j$(nproc) ARCH=mips CC=clang CROSS_COMPILE=mipsel-linux-gnu- HOSTCC=clang O=/out distclean malta_defconfig vmlinux
...
$ timeout 2m unbuffer qemu-system-mipsel -kernel /out/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio
Linux version 5.3.0-rc1-00006-g1cc52890c2c4 (root@abb8b1219622) (ClangBuiltLinux clang version 10.0.0 (git://github.com/llvm/llvm-project b45631090220b732e614b5530bbd1d230eb9d38e) (based on LLVM 10.0.0svn)) #1 SMP Thu Jul 25 20:15:38 UTC 2019
...

[nickdesaulniers]

Filed: https://bugs.llvm.org/show_bug.cgi?id=42763
Thanks for bisecting.

[nickdesaulniers]

r358919 commit d748689c7f71 ("[InstCombine] Eliminate stores to constant memory")

adds three specific cases. I'm wondering if you could bisect which of the three introduce the issue?

ie. revert d748689c7f71
then manually add back the cases individually, rebuild clang, re-boot test. See if it's one, two, or all three that are bad.

[nathanchance]

Sure, I’ll do it now.

[nathanchance]

@nickdesaulniers it looks like the change in llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp is the problematic one.

[nathanchance]

Okay...

With:

• The llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp part of https://github.com/llvm/llvm-project/commit/d748689c7f718f531871bb44b5da05888c3c0301 reverted

• The following tc-build command:

$ ./build-llvm.py -p "clang" -t "Mips;X86"

• These commits: https://github.com/ClangBuiltLinux/linux/commits/mips

• This rootfs image: https://github.com/nathanchance/continuous-integration/blob/dc70ff37bdf41b4c145c1c7e569c5008c52b113e/images/mips/rootfs.ext4

I can build and boot a malta_defconfig kernel!

$ timeout 2m unbuffer qemu-system-mipsel -kernel /out/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio -append "root=/dev/sda" -drive file=rootfs.ext4,format=raw,if=ide
Linux version 5.3.0-rc1-00006-g1cc52890c2c4 (root@abb8b1219622) (ClangBuiltLinux clang version 10.0.0 (git://github.com/llvm/llvm-project b45631090220b732e614b5530bbd1d230eb9d38e) (based on LLVM 10.0.0svn)) #1 SMP Thu Jul 25 22:15:28 UTC 2019
earlycon: uart8250 at I/O port 0x3f8 (options '38400n8')
printk: bootconsole [uart8250] enabled
Config serial console: console=ttyS0,38400n8r
CPU0 revision is: 00019300 (MIPS 24Kc)
FPU revision is: 00739300
OF: fdt: No chosen node found, continuing without
MIPS: machine is mti,malta
Software DMA cache coherency enabled
Determined physical RAM map:
 memory: 08000000 @ 00000000 (usable)
Primary instruction cache 2kB, VIPT, 2-way, linesize 16 bytes.
Primary data cache 2kB, 2-way, VIPT, no aliases, linesize 16 bytes
Zone ranges:
  DMA      [mem 0x0000000000000000-0x0000000000ffffff]
  Normal   [mem 0x0000000001000000-0x0000000007ffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x0000000007ffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x0000000007ffffff]
percpu: Embedded 4 pages/cpu s33552 r8192 d23792 u65536
Built 1 zonelists, mobility grouping on.  Total pages: 8176
Kernel command line: root=/dev/sda console=ttyS0,38400n8r
Dentry cache hash table entries: 16384 (order: 2, 65536 bytes, linear)
Inode-cache hash table entries: 8192 (order: 1, 32768 bytes, linear)
Writing ErrCtl register=00000000
Readback ErrCtl register=00000000
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 120816K/131072K available (6826K kernel code, 245K rwdata, 1064K rodata, 368K init, 246K bss, 10256K reserved, 0K cma-reserved)
rcu: Hierarchical RCU implementation.
rcu:    RCU event tracing is enabled.
rcu:    RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 256
random: get_random_bytes called from start_kernel+0x250/0x454 with crng_init=0
CPU frequency 200.00 MHz
clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112567771 ns
sched_clock: 32 bits at 100MHz, resolution 9ns, wraps every 21474795515ns
Console: colour dummy device 80x25
Calibrating delay loop... 996.14 BogoMIPS (lpj=4980736)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 4096 (order: 0, 16384 bytes, linear)
Mountpoint-cache hash table entries: 4096 (order: 0, 16384 bytes, linear)
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: 0, 16384 bytes, linear)
NET: Registered protocol family 16
clocksource: pit: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1601818034827 ns
vgaarb: loaded
SCSI subsystem initialized
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x10000000-0x17ffffff]
pci_bus 0000:00: root bus resource [io  0x1000-0x1fffff]
pci_bus 0000:00: root bus resource [??? 0x00000000 flags 0x0]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:00.0: [11ab:4620] type 00 class 0x060000
pci 0000:00:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size)
pci 0000:00:0a.0: [8086:7110] type 00 class 0x060100
pci 0000:00:0a.1: [8086:7111] type 00 class 0x010180
pci 0000:00:0a.1: reg 0x20: [io  0x0000-0x000f]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
pci 0000:00:0a.2: [8086:7112] type 00 class 0x0c0300
pci 0000:00:0a.2: reg 0x20: [io  0x0000-0x001f]
pci 0000:00:0a.3: [8086:7113] type 00 class 0x068000
pci 0000:00:0a.3: quirk: [io  0x1000-0x103f] claimed by PIIX4 ACPI
pci 0000:00:0a.3: quirk: [io  0x1100-0x110f] claimed by PIIX4 SMB
pci 0000:00:0b.0: [1022:2000] type 00 class 0x020000
pci 0000:00:0b.0: reg 0x10: [io  0x0000-0x001f]
pci 0000:00:0b.0: reg 0x14: [mem 0x00000000-0x0000001f]
pci 0000:00:0b.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
pci 0000:00:12.0: [1013:00b8] type 00 class 0x030000
pci 0000:00:12.0: reg 0x10: [mem 0x00000000-0x01ffffff pref]
pci 0000:00:12.0: reg 0x14: [mem 0x00000000-0x00000fff]
pci 0000:00:12.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
pci 0000:00:12.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
pci 0000:00:12.0: BAR 0: assigned [mem 0x10000000-0x11ffffff pref]
pci 0000:00:0b.0: BAR 6: assigned [mem 0x12000000-0x1203ffff pref]
pci 0000:00:12.0: BAR 6: assigned [mem 0x12040000-0x1204ffff pref]
pci 0000:00:12.0: BAR 1: assigned [mem 0x12050000-0x12050fff]
pci 0000:00:0a.2: BAR 4: assigned [io  0x1040-0x105f]
pci 0000:00:0b.0: BAR 0: assigned [io  0x1060-0x107f]
pci 0000:00:0b.0: BAR 1: assigned [mem 0x12051000-0x1205101f]
pci 0000:00:0a.1: BAR 4: assigned [io  0x1080-0x108f]
clocksource: Switched to clocksource MIPS
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 4096 (order 0, 16384 bytes)
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 2048 (order: 0, 24576 bytes, linear)
TCP established hash table entries: 4096 (order: 0, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 1, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 512 (order: 0, 16384 bytes, linear)
UDP-Lite hash table entries: 512 (order: 0, 16384 bytes, linear)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:00:0a.2: enabling device (0000 -> 0001)
PCI: CLS 0 bytes, default 16
Initialise system trusted keyrings
workingset: timestamp_bits=30 max_order=13 bucket_order=0
Installing knfsd (copyright (C) 1996 [email protected]).
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
io scheduler mq-deadline registered
io scheduler kyber registered
cirrusfb 0000:00:12.0: enabling device (0000 -> 0002)
cirrusfb 0000:00:12.0: Cirrus Logic chipset on PCI bus, RAM (4096 kB) at 0x10000000
Console: switching to colour frame buffer device 80x30
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
printk: console [ttyS0] disabled
serial8250.0: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
printk: console [ttyS0] enabled
printk: console [ttyS0] enabled
printk: bootconsole [uart8250] disabled
printk: bootconsole [uart8250] disabled
serial8250.0: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
serial8250.0: ttyS2 at MMIO 0x1f000900 (irq = 20, base_baud = 230400) is a 16550A
cacheinfo: Failed to find cpu0 device node
cacheinfo: Unable to detect cache hierarchy for CPU 0
brd: module loaded
ata_piix 0000:00:0a.1: enabling device (0000 -> 0001)
scsi host0: ata_piix
scsi host1: ata_piix
ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0x1080 irq 14
ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
pcnet32: pcnet32.c:v1.35 21.Apr.2008 [email protected]
pcnet32 0000:00:0b.0: enabling device (0000 -> 0003)
pcnet32: PCnet/PCI II 79C970A at 0x1060, 52:54:00:12:34:56 assigned IRQ 10
pcnet32: eth0: registered as PCnet/PCI II 79C970A
pcnet32: 1 cards_found
mousedev: PS/2 mouse device common for all mice
rtc_cmos 70.rtc: registered as rtc0
rtc_cmos 70.rtc: alarms up to one day, 242 bytes nvram
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
Loading compiled-in X.509 certificates
rtc_cmos 70.rtc: setting system clock to 2019-07-26T06:01:02 UTC (1564120862)
ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100
ata1.00: 40960 sectors, multi 16: LBA48 
ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 40960 512-byte logical blocks: (21.0 MB/20.0 MiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: [sda] Attached SCSI disk
sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
cdrom: Uniform CD-ROM driver Revision: 3.20
EXT4-fs (sda): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) readonly on device 8:0.
Freeing unused kernel memory: 368K
This architecture does not have kernel memory protection.
Run /sbin/init as init process
EXT4-fs (sda): re-mounted. Opts: (null)

can't open /dev/null: No such file or directory

can't open /dev/null: No such file or directory

can't open /dev/null: No such file or directory

can't open /dev/null: No such file or directory
Starting syslogd: OK
Starting klogd: OK
Starting network: OK
Linux version 5.3.0-rc1-00006-g1cc52890c2c4 (root@abb8b1219622) (ClangBuiltLinux clang version 10.0.0 (git://github.com/llvm/llvm-project b45631090220b732e614b5530bbd1d230eb9d38e) (based on LLVM 10.0.0svn)) #1 SMP Thu Jul 25 22:15:28 UTC 2019
random: fast init done
Linux version 5.3.0-rc1-00006-g1cc52890c2c4 (root@abb8b1219622) (ClangBuiltLinux clang version 10.0.0 (git://github.com/llvm/llvm-project b45631090220b732e614b5530bbd1d230eb9d38e) (based on LLVM 10.0.0svn)) #1 SMP Thu Jul 25 22:15:28 UTC 2019
Stopping network: OK
Stopping klogd: OK
Stopping syslogd: OK
EXT4-fs (sda): re-mounted. Opts: (null)

The system is going down NOW!

Sent SIGTERM to all processes

Sent SIGKILL to all processes

Requesting system poweroff
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Stopping disk
reboot: Power down

I will be sending the patches on the mips branch linked above tomorrow.

[nickdesaulniers]

So I'd be curious to see if we can find the test case that's problematic, then the llvm devs can use that as a test case.

I think within the conditional identified, if you did:

if (AA->pointsToConstantMemory(Ptr)) {
    errs() << *SI->getParent()->getParent() << "\n";
    return eraseInstFromFunction(SI);
}

that would print the function for which the problematic removal occurred. Then we could take a look at the C code that produced this.

[nathanchance]

Alright I will try that in an hour or so!

[nathanchance]

$ git diff HEAD
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp b/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
index 62acebf74b2..9a82d5648fd 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp
@@ -1396,6 +1396,14 @@ Instruction *InstCombiner::visitStoreInst(StoreInst &SI) {
     }
   }

+  // If we have a store to a location which is known constant, we can conclude
+  // that the store must be storing the constant value (else the memory
+  // wouldn't be constant), and this must be a noop.
+  if (AA->pointsToConstantMemory(Ptr)) {
+    errs() << "Function being removed: " << SI.getParent()->getParent() << "\n";
+    return eraseInstFromFunction(SI);
+  }
+
   // Do really simple DSE, to catch cases where there are several consecutive
   // stores to the same location, separated by a few arithmetic operations. This
   // situation often occurs with bitfield accesses.

Then I see: Function being removed: 0x6135c38

[gwelymernans]

Make the error message this instead:

errs() << "Function being removed: " << SI.getParent()->getParent()->getName() << "\n";

or

errs() << "Function being removed:\n";
SI.getParent()->getParent()->dump();

[nathanchance]

Thanks, looks better:

Function being removed: set_io_port_base

Hmmmmm... https://github.com/torvalds/linux/blob/v5.3-rc1/arch/mips/include/asm/io.h#L58-L81

/*
 * On MIPS I/O ports are memory mapped, so we access them using normal
 * load/store instructions. mips_io_port_base is the virtual address to
 * which all ports are being mapped.  For sake of efficiency some code
 * assumes that this is an address that can be loaded with a single lui
 * instruction, so the lower 16 bits must be zero.  Should be true on
 * on any sane architecture; generic code does not use this assumption.
 */
extern const unsigned long mips_io_port_base;

/*
 * Gcc will generate code to load the value of mips_io_port_base after each
 * function call which may be fairly wasteful in some cases.  So we don't
 * play quite by the book.  We tell gcc mips_io_port_base is a long variable
 * which solves the code generation issue.  Now we need to violate the
 * aliasing rules a little to make initialization possible and finally we
 * will need the barrier() to fight side effects of the aliasing chat.
 * This trickery will eventually collapse under gcc's optimizer.  Oh well.
 */
static inline void set_io_port_base(unsigned long base)
{
    * (unsigned long *) &mips_io_port_base = base;
    barrier();
}

[nickdesaulniers]

Note the * was very much intentional as getParent method returns a pointer, so you must dereference it to print it as a Function, then the operator<< will be invoked on it.

[nathanchance]

Without that:

/tc-build/llvm-project/llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp:1403:18: error: member reference type 'llvm::StoreInst' is not a pointer; did you mean to use '.'?
    errs() << *SI->getParent()->getParent() << "\n";
               ~~^~

1 error generated.

[nickdesaulniers]

Right, it's a reference, not a pointer, my bad.

*SI.getParent()->getParent();

[nickdesaulniers]

@nathanchance can you please boot test vanilla clang (revert any changes you have) with the patch I just attached to https://bugs.llvm.org/show_bug.cgi?id=42763 applied?

[nickdesaulniers]

worst case we could always wrap it in an ifdef __clang__

[nathanchance]

Yes, I will do so now.

[nathanchance]

That patch works fine with LLVM at https://github.com/llvm/llvm-project/commit/b45631090220b732e614b5530bbd1d230eb9d38e and the MIPS branch.

[nickdesaulniers]

Do you have a mips gcc toolchain to boot test, too? If it's fine with both compilers, I can send the patch monday w/ your tested by.

[nathanchance]

Yes, I will test now.

It'd be worth sending that patch + the ones I have in the MIPS branch together since malta_defconfig won't build without the first four due to -Werror in arch/mips: https://github.com/torvalds/linux/blob/v5.3-rc1/arch/mips/Kbuild#L6

[nathanchance]

$ make -s -C /linux -j$(nproc) ARCH=mips CROSS_COMPILE=mipsel-linux-gnu- O=/out distclean malta_defconfig vmlinux && timeout 2m unbuffer qemu-system-mipsel -kernel /out/vmlinux -M malta -cpu 24Kf -no-reboot -display none -serial mon:stdio -append "root=/dev/sda" -drive file=/tmp/rootfs.ext4,format=raw,if=ide
...
Run /sbin/init as init process
EXT4-fs (sda): re-mounted. Opts: (null)
can't open /dev/null: No such file or directory
can't open /dev/null: No such file or directory
can't open /dev/null: No such file or directory
can't open /dev/null: No such file or directory
Starting syslogd: OK
Starting klogd: OK
Starting network: OK
Linux version 5.3.0-rc1-cbl-00141-g429923fa7d2c-dirty (root@abb8b1219622) (gcc version 8.3.0 (Debian 8.3.0-2)) #1 SMP Fri Jul 26 23:20:32 UTC 2019
random: fast init done
Linux version 5.3.0-rc1-cbl-00141-g429923fa7d2c-dirty (root@abb8b1219622) (gcc version 8.3.0 (Debian 8.3.0-2)) #1 SMP Fri Jul 26 23:20:32 UTC 2019
Stopping network: OK
Stopping klogd: OK
Stopping syslogd: OK
EXT4-fs (sda): re-mounted. Opts: (null)
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system poweroff
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Stopping disk
reboot: Power down

[nickdesaulniers]

Can you add the mips images in a PR to the CI? We'll do everything but enable it in .travis.yml.

[nathanchance]

Done: https://github.com/ClangBuiltLinux/continuous-integration/pull/192

[nickdesaulniers]

https://lkml.org/lkml/2019/7/29/1636

[nickdesaulniers]

https://github.com/xtreamerdev/linux-xtr/commit/c94e57dcd61d661749d53ee876ab265883b0a103

[nickdesaulniers]

https://github.com/llvm/llvm-project/commit/e39e79358fcdd5d8ad809defaa821f0bbfa809a5

[nathanchance]

Patch accepted: https://git.kernel.org/mips/c/12051b318bc3ce5b42d6d786191008284b067d83

[nathanchance]

Merged into mainline: https://git.kernel.org/linus/12051b318bc3ce5b42d6d786191008284b067d83