Crystal: Compiler crash

[stronny]

I was able to narrow it down to this snippet, however it seems so random, that I'm not even sure it's relevant:

require "socket"
def handle_client(client)
end
server = TCPServer.new("localhost", 1234)
while client = server.accept?
  spawn handle_client(client)
end

$ ~/src/crystal/bin/crystal --version
Using compiled compiler at `.build/crystal'
Crystal 0.25.1+108 [9a98613c7] (2018-08-06)

LLVM: 5.0.1
Default target: i686-pc-linux-gnu
$ ~/src/crystal/bin/crystal build -d --error-trace -s -p src.cr &> buildlog
$ head -20 buildlog
Using compiled compiler at `.build/crystal'
Parse:                             00:00:00.001304499 (   0.25MB)
Semantic (top level):              00:00:01.325749329 (  24.40MB)
Semantic (new):                    00:00:00.004914384 (  24.40MB)
Semantic (type declarations):      00:00:00.097412050 (  24.40MB)
Semantic (abstract def check):     00:00:00.011379647 (  24.40MB)
Semantic (ivars initializers):     00:00:00.012753726 (  24.40MB)
Semantic (cvars initializers):     00:00:00.516127086 (  32.57MB)
Invalid memory access (signal 11) at address 0xbf4c4ddc
[0x82bf1a] *CallStack::print_backtrace:Int32 +154
[0x803cd1] __crystal_sigfault_handler +129
[0x192f7f1] sigfault_handler +37
[0xb7f49d00] ???
[0x1922309] GC_clear_stack_inner +41
[0x1922324] GC_clear_stack_inner +68 (0 times)
[0x19223ac] GC_clear_stack +92
[0x191e913] GC_generic_malloc_many +723
[0x1926d14] GC_malloc +212
[0x8676fe] *GC::malloc<UInt32>:Pointer(Void) +30
[0x7edffe] __crystal_malloc64 +110

the full buildlog is here https://gist.githubusercontent.com/stronny/c762b3e6a95425b0eadc09cf87066280/raw/e7cb3399247e5c67315c799973fe1a9519f684ad/buildlog

[stronny]

the redacted codebase is here https://github.com/stronny/bugreport

[asterite]

The code in the first snippet does not reproduce the bug.

[stronny]

@asterite look, I've spent a non-trivial amount of time to give you as much code as I can. Please understand that your comment relates to a "snippet" that I myself have labelled as "not even sure relevant".

Of course by itself it doesn't reproduce, it comes straight from the docs.

[stronny]

jfyi 0.26.0 didn't fix this

$ crystal --version
Crystal 0.26.0 [eeb53c506] (2018-08-09)

LLVM: 4.0.0
Default target: i686-unknown-linux-gnu
$ crystal build redacted.cr
Invalid memory access (signal 11) at address 0xbf220c9c
[0xb5c89533] ???
[0xb535fdb4] ???
[0xb71f74bf] ???
[0xb721fcd4] ???

[stronny]

Still crashes with a new message

$ crystal --version
Crystal 0.27.0 [c9d1eef8f] (2018-11-01)

LLVM: 4.0.0
Default target: i686-unknown-linux-gnu
$ crystal build redacted.cr
Stack overflow (e.g., infinite or very deep recursion)
[0xb5c3d3f3] ???
[0xb53106ae] ???
[0xb71ac66f] ???
[0xb71d4f2e] ???

[bew]

If you don't give us code that reproduce the problem, we won't be able to understand the issue and fix it

[RX14]

@bew if you look at the second comment, there is a repo with code which reproduces

[RX14]

Also, it's a stack overflow, which is nice info. Glad the stack overflow detector is working in the real world.

[bew]

Oh right @RX14 I didn't see he posted a link to his repo, sorry

[stronny]

Still crashes.

$ crystal --version
Crystal 0.27.1 [64137d045] (2019-01-30)

LLVM: 4.0.0
Default target: i686-unknown-linux-gnu

$ time crystal redacted.cr
Stack overflow (e.g., infinite or very deep recursion)
[0xb5bc4a03] ???
[0xb5101f67] ???
[0xb7133e6f] ???
[0xb71595a9] ???

real    0m22.497s
user    0m33.642s
sys     0m1.171s

[konovod]

after some research...
overflow happens in a compiler, not a program.
so far i've reduced it to this snippet:
https://carc.in/#/r/6588

require "logger"
require "json"

# This class wraps a JSON::Any vector to deal with the tree as a whole.
#
# Provided are methods to get/set a value, as well as to merge in another JSON::Any vector.
#
class JSONTree

    alias JHash = Hash(String, JSON::Any)
    alias JArray = Array(JSON::Any)
    alias JVector = JHash | JArray
    alias JScalar = Bool | Float64 | Int64 | String | Nil
    alias Key = String | Int32
    alias SKey = String | Int32 | Symbol

    Jnil = JSON::Any.new nil

    # Wraps keys tuple and the current level as well as defines overloads for low level vector manipulations
    #
    class Index
        def self.get(id : self, h : JHash,  k : String) : JSON::Any; h[k] end
        def self.get(id : self, h : JHash,  k : Int32)  : JSON::Any; raise Exception.new("#{id}: index should be String") end
        def self.get(id : self, a : JArray, i : String) : JSON::Any; raise Exception.new("#{id}: index should be Int32") end
        def self.get(id : self, a : JArray, i : Int32)  : JSON::Any; a[i] end

        def self.mknext(id : self, v : JVector,  k : Key, nk : String) : Nil; set id, JSON::Any.new(JSONTree.empty_jhash),  v, k end
        def self.mknext(id : self, v : JVector,  k : Key, nk : Int32 ) : Nil; set id, JSON::Any.new(JSONTree.empty_jarray), v, k end

        def self.skey_to_key(k : Key) : Key; k end
        def self.skey_to_key(k : Symbol) : Key; k.to_s end

        @last_id : Int32
        @keys : Array(Key)
        def initialize(keys : Array(SKey) = [] of SKey)
            @keys = keys.map { |k| self.class.skey_to_key k }
            @last_id = @keys.size - 1
            @current = @keys.empty? ? -1 : 0
        end
        def root? : Bool; @current < 0 end
        def last? : Bool; @current == @last_id end
        def bump! : Nil
            raise IndexError.new("can't bump, this is the last key") if @current >= @last_id
            @current += 1
        end
        def key : Key;
            raise IndexError.new("no current key, at the root") if root?
            @keys[@current]
        end

        def get(vector : JVector) : JSON::Any
            return JSON::Any.new(vector) if root?
            self.class.get self, vector, key
        end

        def get_bump!(vector : JVector) : JSON::Any
            res = get(vector)
            bump!
            res
        end
    end

    # Used for cast errors.
    # For example you want an integer at "level1"."level2"."value", but there is an Array.
    #
    class TypeError < Exception
        def initialize(@value : JSON::Any::Type, @expected : Object.class)
            super "Got #{@value.class} instead of #{@expected}"
        end
    end


    # Raises: scalars can't be indexed
    #
    def self.get(id : Index, scalar : JScalar) : JSON::Any::Type
        raise ""
    end
    # Recursively fetches a value from `vector` at `id`. Raises on missing levels.
    #
    def self.get(id : Index, vector : JVector) : JSON::Any::Type
        return id.get(vector).raw if id.last?
        get id, id.get_bump!(vector).raw
    end


    # Transforms an SKey tuple to an SKey array
    #
    def self.keyarr(*keys : SKey) : Array(SKey)
        res = [] of SKey
        keys.each { |k| res.push k }
        res
    end


    def initialize(@root : JHash = self.class.empty_jhash); end
    def initialize(@root : JArray); end
    def initialize(scalar : JScalar)
        initialize
        raise ""
    end
    def initialize(root : JSON::Any); initialize root.raw end

    # Gets a value at `keys` index
    #
    def get(keys : Array(SKey) = [] of SKey) : JSON::Any::Type; self.class.get Index.new(keys), @root end
    def get(*keys : SKey) : JSON::Any::Type; get self.class.keyarr(*keys) end

    def string(keys : Array(SKey) = [] of SKey) : String
        value = self.class.get Index.new(keys), @root
        raise TypeError.new(value, String) unless value.is_a? String
        value
    end
    def string(*keys : SKey) : String; string self.class.keyarr(*keys) end

end

msgs = {} of String => JSONTree
msg = msgs[""]
puts Logger::Severity.parse(msg.string(:level))

[konovod]

aaand finally reduced to
https://carc.in/#/r/65az

require "logger"

class B
  def initialize(@expected : Object.class)
    puts "#{@expected}"
  end
end

class A
  def self.foo
    value = ""
    B.new(String) unless value.is_a? String
    value
  end
end

puts Logger::Severity.parse(A.foo)

no idea what is happening though.

[stronny]

Hey, very nice job there! Stripping down even more:

enum X; Nil end

class E
# def initialize(@klass : Nil.class); end  # this works
  def initialize(@klass : Object.class); end  # this doesn't
  def to_s(io : IO); io << @klass.to_s end
end

a = X.parse E.new(Nil).to_s
puts a  # works without puts

[asterite]

I'm pretty sure it happens because of Object.class. Please don't use it. If we fix this it will give a compiler error. There's simply no way right now to hold Object.class inside an instance var.

[stronny]

Please fix the inconsistency. If this is an error, it should be clearly indicated as such.

So, is there a way around? How do I get rid of Object.class? class E(T)?

[stronny]

still crashes

$ cat test.cr
#!/usr/bin/env crystal

enum X; Nil end

class E
# def initialize(@klass : Nil.class); end  # this works
  def initialize(@klass : Object.class); end  # this doesn't
  def to_s(io : IO); io << @klass.to_s end
end

a = X.parse E.new(Nil).to_s
puts a  # works without puts

$ ./test.cr
Invalid memory access (signal 11) at address 0xbf370f8c
[0xb5bcf9f3] ???
[0xb5aeeb67] ???
[0xb713ee6f] ???
[0xb71645fc] ???

$ crystal --version
Crystal 0.28.0 [639e4765f] (2019-04-18)

LLVM: 4.0.0
Default target: i386-unknown-linux-gnu

[vlazar]

@stronny 's example above gives me

Stack overflow (e.g., infinite or very deep recursion)

Crystal 0.31.1 on MacOS

$ crystal --version
Crystal 0.31.1 (2019-10-02)

LLVM: 8.0.1
Default target: x86_64-apple-macosx

[rdp]

Some kind of recursion going on?