Crystal: LLVM4 crash: Cannot select ... & other errors in codegen

[bew]

First, version:

$ crystal -v
Crystal 0.22.0 (2017-04-22) LLVM 4.0.0

The faulty code:

Kilobyte = 1024u32
Megabyte = 1024u32 * Kilobyte
Gigabyte = 1024u32 * Megabyte

struct BigStruct
  @some_array : UInt8[Megabyte]?
end

BigStruct.new

sleep 3

Carc-in is broken on this one: https://carc.in/#/r/22nn

I get the output:

Parse:                             00:00:00.0000907 (   0.19MB)
Semantic (top level):              00:00:00.0820709 (  27.11MB)
Semantic (new):                    00:00:00.0006868 (  27.11MB)
Semantic (type declarations):      00:00:00.0100532 (  35.11MB)
Semantic (abstract def check):     00:00:00.0006654 (  35.11MB)
Semantic (ivars initializers):     00:00:00.0007388 (  35.11MB)
Semantic (cvars initializers):     00:00:00.0052621 (  35.11MB)
Semantic (main):                   00:00:00.0713333 (  43.17MB)
Semantic (cleanup):                00:00:00.0005363 (  51.17MB)
Semantic (recursive struct check): 00:00:00.0004676 (  51.17MB)
Codegen (crystal):                 00:00:00.1255013 (  51.29MB)
Codegen (bc+obj):                 LLVM ERROR: Cannot select: 0x7ecc960: ch,glue = callseq_end 0x7ecd388, TargetConstant:i64<0>, TargetConstant:i64<0>, 0x7ecd388:1
  0x7edc950: i64 = TargetConstant<0>
  0x7edc950: i64 = TargetConstant<0>
  0x7ecd388: ch,glue = X86ISD::CALL 0x7edd310, TargetGlobalAddress:i64<%BigStruct ()* @"*BigStruct::new:BigStruct"> 0 [TF=6], Register:i64 %RDI, RegisterMask:Untyped, 0x7edd310:1
    0x7edcc28: i64 = TargetGlobalAddress<%BigStruct ()* @"*BigStruct::new:BigStruct"> 0 [TF=6]
    0x7eded18: i64 = Register %RDI
    0x7eccdd8: Untyped = RegisterMask
    0x7edd310: ch,glue = CopyToReg 0x7ecd3f0, Register:i64 %RDI, FrameIndex:i64<7>
      0x7eded18: i64 = Register %RDI
      0x7ecc7c0: i64 = FrameIndex<7>
In function: __crystal_main

^C

There where an infinite loop at the end, I needed to Ctrl-C it..

• If I remove the sleep 3, the LLVM error goes away, but not the infinite loop (the sleep's time doesn't change anything)
• If I remove the BigStruct.new line, no error (but the program does nothing obviously!)

• If I use numbers instead of constants (in UInt8[1048576]?) doesn't change anything

• If I change Megabyte to Gigabyte in UInt8[Gigabyte]?, I get a completely different error (still in codegen though):

Parse:                             00:00:00.0001064 (   0.19MB)
Semantic (top level):              00:00:00.0830904 (  27.10MB)
Semantic (new):                    00:00:00.0006906 (  27.10MB)
Semantic (type declarations):      00:00:00.0096108 (  35.10MB)
Semantic (abstract def check):     00:00:00.0006449 (  35.10MB)
Semantic (ivars initializers):     00:00:00.0007385 (  35.10MB)
Semantic (cvars initializers):     00:00:00.0053483 (  35.10MB)
Semantic (main):                   00:00:00.0730416 (  51.16MB)
Semantic (cleanup):                00:00:00.0005108 (  51.16MB)
Semantic (recursive struct check): 00:00:00.0004811 (  51.16MB)
Codegen (crystal):                 00:00:00.1198496 (  51.29MB)
Codegen (bc+obj):                 Invalid memory access (signal 11) at address 0x1dffffd0
[0x43cebf] __crystal_sigfault_handler +63
[0x7f8d68cfdfe0] ???
[0x7f8d6a5b319f] _ZN4llvm13GetReturnInfoEPNS_4TypeENS_12AttributeSetERNS_15SmallVectorImplINS_3ISD9OutputArgEEERKNS_14TargetLoweringERKNS_10DataLayoutE +735
[0x7f8d6a648dcd] _ZN4llvm20FunctionLoweringInfo3setERKNS_8FunctionERNS_15MachineFunctionEPNS_12SelectionDAGE +269
[0x7f8d6a76ac2e] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +926
[0x7f8d6b9a1e74] ???
[0x7f8d6a446841] _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE +145
[0x7f8d6a2c6b32] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +642
[0x7f8d6a2c6bd3] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +51
[0x7f8d6a2c7584] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[0x7f8d6b164c45] ???
[0x7f8d6b164f64] LLVMTargetMachineEmitToFile +244
[0x13d26ca] ???
[0x13d6d53] ???
[0x411668] ???
[0x0] ???
Invalid memory access (signal 11) at address 0x3bffffd0
[0x43cebf] __crystal_sigfault_handler +63
[0x7f8d68cfdfe0] ???
[0x7f8d6a5b319f] _ZN4llvm13GetReturnInfoEPNS_4TypeENS_12AttributeSetERNS_15SmallVectorImplINS_3ISD9OutputArgEEERKNS_14TargetLoweringERKNS_10DataLayoutE +735
[0x7f8d6a63b6eb] _ZN4llvm8FastISel11lowerCallToERNS0_16CallLoweringInfoE +363
[0x7f8d6a63d14f] _ZN4llvm8FastISel9lowerCallEPKNS_8CallInstE +1391
[0x7f8d6a640b84] _ZN4llvm8FastISel10selectCallEPKNS_4UserE +100
[0x7f8d6a63df8d] _ZN4llvm8FastISel14selectOperatorEPKNS_4UserEj +413
[0x7f8d6a643333] _ZN4llvm8FastISel17selectInstructionEPKNS_11InstructionE +883
[0x7f8d6a768e68] _ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE +696
[0x7f8d6a76ad49] _ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE +1209
[0x7f8d6b9a1e74] ???
[0x7f8d6a446841] _ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE +145
[0x7f8d6a2c6b32] _ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE +642
[0x7f8d6a2c6bd3] _ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE +51
[0x7f8d6a2c7584] _ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE +772
[0x7f8d6b164c45] ???
[0x7f8d6b164f64] LLVMTargetMachineEmitToFile +244
[0x13d26ca] ???
[0x13d6d53] ???
[0x411668] ???
[0x0] ???
 00:00:04.0648193 (  51.29MB)
Codegen (linking):                _main.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc -o "/home/lesell_b/tests/crystal/fork_on_oom/fork_and_stack" "${@}"  -rdynamic  -lpcre -lgc -lpthread /usr/lib/crystal/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib`

This last error seams to be LLVM that violently crash, I don't know how I could report this to LLVM team..

Note: Is it just me ? (carcin can't comfirm, and I have only one computer to test right now)

[Papierkorb]

Is it just me ?

I can confirm this on Crystal 0.22.0 (2017-04-22) LLVM 4.0.0, I'm also getting the same error as you are

[faustinoaq]

If you use --release flag error disappear.

crystal build -s --release g4452.cr
Parse:                             00:00:00.0001905 (   0.19MB)
Semantic (top level):              00:00:00.2001953 (  27.20MB)
Semantic (new):                    00:00:00.0013384 (  27.20MB)
Semantic (type declarations):      00:00:00.0172932 (  27.20MB)
Semantic (abstract def check):     00:00:00.0014872 (  27.20MB)
Semantic (ivars initializers):     00:00:00.0011554 (  27.20MB)
Semantic (cvars initializers):     00:00:00.0124931 (  35.20MB)
Semantic (main):                   00:00:00.1674670 (  43.26MB)
Semantic (cleanup):                00:00:00.0009177 (  43.26MB)
Semantic (recursive struct check): 00:00:00.0011832 (  43.26MB)
Codegen (crystal):                 00:00:00.1593567 (  51.26MB)
Codegen (bc+obj):                  00:00:09.0689940 (  51.26MB)
Codegen (linking):                 00:00:00.0604327 (  51.26MB)

Codegen (bc+obj):
 - no previous .o files were reused

[asterite]

As far as I know, LLVM doesn't handle well big arrays (array being an LLVM type that we use to represent static arrays). And the bigger they are, the longer it takes to compile a program, and much more to optimize it.

I think the compiler should disallow static arrays bigger than, say, 10K or 20K. There's no reason to use them instead of a regular Array.

[ysbaddaden]

LLVM must have a hard time dealing with a huge gap on the stack. The program may crash randomly, too, since the stack has a limited size. I agree we should limit the maximum size of structs to an acceptable limit. If I recall correctly, HTTP/2 uses a 16KB frame size by default. It would be nice to keep a 16KB buffer on the stack (or maybe twice that, 32KB), if it's acceptable.

[konovod]

StaticArray(UInt8, 32000).new crashes in carc.in, and 16000 too, so the limit is smaller, like a 10000.

[bew]

Should we keep this issue open or can I close it?

[asterite]

Let's leave this open for now

[vlazar]

Tried this in

$ crystal --version
Crystal 0.31.1 (2019-10-02)

LLVM: 8.0.1
Default target: x86_64-apple-macosx

Maybe I didn't wait long enough but this is the only thing I get after quite a while:

$ crystal test.cr
^C