Crystal: ICE: Module validation failed

[Sija]

Module validation failed: Function return type does not match operand type of return inst!
  ret %"(Array(AnyHash::JSONTypes::Value) | Bool | Float32 | Float64 | Hash(String | Symbol, AnyHash::JSONTypes::Value)+ | Int128 | Int16 | Int32 | Int64 | Int8 | JSON::Any | Set(AnyHash::JSONTypes::Value) | String | Symbol | Time | UInt128 | UInt16 | UInt32 | UInt64 | UInt8 | Nil)" %1, !dbg !28
 %"(Array(AnyHash::JSONTypes::Value) | Bool | Float32 | Float64 | Hash(String | Symbol, AnyHash::JSONTypes::Value) | Int128 | Int16 | Int32 | Int64 | Int8 | JSON::Any | Set(AnyHash::JSONTypes::Value) | String | Symbol | Time | UInt128 | UInt16 | UInt32 | UInt64 | UInt8 | Nil)" = type { i32, [3 x i64] } (Exception)
  from Crystal::CodeGenVisitor#finish:Nil
  from Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Tuple(Array(Crystal::Compiler::CompilationUnit), Array(String)) | Nil)
  from Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result
  from Crystal::Command#run:(Bool | Crystal::Compiler::Result | Nil)
  from main

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

I have a hard time isolating the code responsible for the issue.
Happens with Crystal 0.26.0, Amber 0.8 and raven.cr (develop branch) on macOS.

[faustinoaq]

@Sija Interesting, Does this error show any specific file/line?

[Sija]

@faustinoaq none I can find, with --debug flag it fails with:

$ shards build --debug

Dependencies are satisfied
Building: pet-tracker
Error target pet-tracker failed to compile:
Module validation failed: conflicting debug info for argument
  call void @llvm.dbg.declare(metadata i32 %__temp_1797, metadata !1566, metadata !366), !dbg !1528
!1529 = !DILocalVariable(name: "self", arg: 1, scope: !1513, file: !322, line: 61, type: !1530)
!1566 = !DILocalVariable(name: "__temp_1797", arg: 1, scope: !1513, file: !322, line: 61, type: !14)

Crystal::CodeGenVisitor#finish:Nil
Crystal::Compiler#codegen<Crystal::Program, Crystal::ASTNode+, Array(Crystal::Compiler::Source), String>:(Tuple(Array(Crystal::Compiler::CompilationUnit), Array(String)) | Nil)
Crystal::Compiler#compile<Array(Crystal::Compiler::Source), String>:Crystal::Compiler::Result
Crystal::Command#run:(Bool | Crystal::Compiler::Result | Nil)
main

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

[Sija]

Ahh, finally with master (props to @RX14 for #6261!) we have some proper backtrace:

$ ../crystal/bin/crystal build src/pet-tracker.cr --debug

Using compiled compiler at `.build/crystal'
Module validation failed: conflicting debug info for argument
  call void @llvm.dbg.declare(metadata i32 %__temp_1797, metadata !1566, metadata !366), !dbg !1528
!1529 = !DILocalVariable(name: "self", arg: 1, scope: !1513, file: !322, line: 61, type: !1530)
!1566 = !DILocalVariable(name: "__temp_1797", arg: 1, scope: !1513, file: !322, line: 61, type: !14)

/Users/sija/Code/crystal/src/llvm/module.cr:78:9 in 'verify'
/Users/sija/Code/crystal/src/compiler/crystal/codegen/codegen.cr:343:9 in 'finish'
/Users/sija/Code/crystal/src/compiler/crystal/codegen/codegen.cr:67:7 in 'codegen'
/Users/sija/Code/crystal/src/compiler/crystal/codegen/codegen.cr:63:5 in 'codegen:debug:single_module'
/Users/sija/Code/crystal/src/compiler/crystal/compiler.cr:22:7 in 'codegen'
/Users/sija/Code/crystal/src/compiler/crystal/compiler.cr:145:16 in 'compile'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:264:7 in 'compile'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:262:5 in 'compile'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:164:5 in 'build'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:69:7 in 'run'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:48:5 in 'run'
/Users/sija/Code/crystal/src/compiler/crystal/command.cr:47:3 in 'run'
/Users/sija/Code/crystal/src/compiler/crystal.cr:8:1 in '__crystal_main'
/Users/sija/Code/crystal/src/crystal/main.cr:104:5 in 'main_user_code'
/Users/sija/Code/crystal/src/crystal/main.cr:93:7 in 'main'
/Users/sija/Code/crystal/src/crystal/main.cr:133:3 in 'main'

Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues

[bew]

Funny, here is another undocumented thing to debug this kind of error:
https://github.com/crystal-lang/crystal/blob/master/src/compiler/crystal/codegen/codegen.cr#L324-L338

One can do DUMP=1 crystal build <file> to dump the llvm-ir before the ir validation (that triggers a module validation error as OP).
You can also do DUMP=some_regex crystal build <file> to dump only the llvm module that matches some_regex.

[Sija]

@bew Nice find! I'll dig into that and report back.

[Sija]

Interestingly, the latter exception relates to https://github.com/amberframework/amber/blob/v0.8.0/src/amber/server/server.cr#L63, if I comment that line we're back to square/exception one.

Mentioned self is of a type Amber::Server and __temp_1797 is Signal.

[Sija]

Seems like the second error has been fixed by #6346 and/or possibly others.

[Sija]

Interestingly now it fails with:

Using compiled compiler at `.build/crystal'
Error in src/pet-tracker.cr:3: instantiating 'Amber::Server:Class#start()'

Amber::Server.start
              ^~~~~

in lib/amber/src/amber/server/server.cr:17: instantiating 'Amber::Server#run()'

      instance.run
               ^~~

in lib/amber/src/amber/server/server.cr:47: instantiating 'Amber::Cluster:Class#fork()'

        thread_count.times { Cluster.fork }
                                     ^~~~

in lib/amber/src/amber/server/cluster.cr:15: instantiating 'Process:Class#fork()'

      Process.fork { Process.run(PROGRAM_NAME, nil, env_hash, true, false, input: Process::Redirect::Inherit, output: Process::Redirect::Inherit, error: Process::Redirect::Inherit) }
              ^~~~

in /Users/sija/Code/crystal/src/process.cr:80: instantiating 'fork_internal()'

    pid = fork_internal do
          ^~~~~~~~~~~~~

in /Users/sija/Code/crystal/src/process.cr:80: instantiating 'fork_internal()'

    pid = fork_internal do
          ^~~~~~~~~~~~~

in lib/amber/src/amber/server/cluster.cr:15: instantiating 'Process:Class#fork()'

      Process.fork { Process.run(PROGRAM_NAME, nil, env_hash, true, false, input: Process::Redirect::Inherit, output: Process::Redirect::Inherit, error: Process::Redirect::Inherit) }
              ^~~~

in lib/amber/src/amber/server/cluster.cr:15: instantiating 'env_hash()'

      Process.fork { Process.run(PROGRAM_NAME, nil, env_hash, true, false, input: Process::Redirect::Inherit, output: Process::Redirect::Inherit, error: Process::Redirect::Inherit) }
                                                    ^~~~~~~~

in lib/amber/src/amber/server/cluster.cr:8: no overload matches 'Hash(Char, Char)#[]=' with types String, String
Overloads are:
 - Hash(K, V)#[]=(key : K, value : V)

        env["FORKED"] = "1"
           ^

Edit: fixed in #6499

[miketheman]

codetriage

@Sija I think it might have something to do with https://github.com/crystal-lang/crystal/pull/6333

I boiled it down to a simple example:

puts typeof(ENV.to_h)

Running on 0.25.1, we get Hash(String, String) https://carc.in/#/r/4onf

Here's a console with old vs new, showing the output change.

$ crystal --version
Crystal 0.25.1 (2018-06-30)

LLVM: 5.0.2
Default target: x86_64-apple-macosx

$ crystal eval 'puts typeof(ENV.to_h)'
Hash(String, String)

$ ~/workspace/miketheman/crystal/bin/crystal --version
Using compiled compiler at `.build/crystal'
Crystal 0.25.1+109 [ff3225fad] (2018-08-06)

LLVM: 6.0.1
Default target: x86_64-apple-macosx

$ ~/workspace/miketheman/crystal/bin/crystal eval 'puts typeof(ENV.to_h)'
Using compiled compiler at `.build/crystal'
Hash(Char, Char)

The signature has between 0.25.1 and ff3225fad - and I used git bisect to try and narrow it down - but there's been a variety of toolchain changes that I can't seem to nail down to bisect from 0.25.1 onwards.

Tagging @straight-shoota and @RX14 who committed that changeset - maybe they can shed some light on this?

[straight-shoota]

Yeah, this seems to be caused by ENV.each yielding String, String instead of {String, String}. That messes up Enumerable#first which is used by Enumerable#to_h.

[RX14]

I doubt the original error has been fixed - just masked by a compile error which has now been solved.

[Sija]

I've just managed to narrow it down to extending Hash.

[straight-shoota]

I've just managed to narrow it down to extending Hash.

My example from https://github.com/crystal-lang/crystal/issues/6997#issuecomment-529147055 also includes subclasses of Hash (Shards::Dependency and Shards::Target). So it seems plausible that this is somehow causing this.

Maybe crystal-lang/shards#230 should be reconsidered?

[Sija]

@straight-shoota Well, guess what, there _was_ a PR fixing it, but @ysbaddaden decided it was a _nitpicking_... Perhaps it's not after all? :P

Edit: you've beat me by a minute, 🎩 for the good memory :)

All in all it's still avoiding the issue instead of fixing it, but I guess it's better that way until someone (@asterite - thinking of you!) will be brave enough to tackle the root cause of it.