Truffleruby: Stack level exhaustion causes segmentation fault instead of SystemStackError

[skunkworker]

When handling an endless loop RC5 is throwing a segmentation fault and quits instead of throwing a SystemStackError.

The code is a simple loop to quickly exhaust the stack.

a = -> {a.call}
a.call

ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-darwin15]

2.4.3 :001 > a = -> {a.call}
 => #<Proc:0x00007f90968cc188@(irb):1 (lambda)>
2.4.3 :002 > a.call
SystemStackError: stack level too deep
    from (irb):1:in `block in irb_binding' (5153 levels)
    from (irb):2
    from /Users/john/.rvm/rubies/ruby-2.4.3/bin/irb:11:in `<main>'
2.4.3 :003 >

truffleruby 1.0.0-rc5, like ruby 2.4.4, GraalVM CE Native [x86_64-darwin]
irb truffleruby-1.0.0-rc5 :001 > a = -> {a.call} => #<Proc:0x2ae@(irb):1 (lambda)> truffleruby-1.0.0-rc5 :002 > a.call [1] 71380 segmentation fault irb

[eregon]

Thank you for the report.

This is a known issue when running on SubstrateVM (in --native mode), which is the default.
The issue is tracked internally as GR-10149 by the SubstrateVM team.
I'll report here when there is progress on that issue.

[headius]

FWIW JRuby has taken the position that stack errors are always considered fatal and we have stopped trying to re-raise a new exception (for obvious reasons, like being unable to construct a new exception at the end of the stack).

Segfault is obviously not great but I would strongly advise not relying on SystemStackError to bubble out on non-MRI runtimes when there's a stack overflow.

[eregon]

@headius Thanks for the comment. I agree, the VM is often in a very fragile state when there is a stack overflow and I think too deep recursion should basically be considered a bug in user code.

One thing we do in TruffleRuby is immediately print [ruby] WARNING StackOverflowError on stderr, and this is typically printed 2 times, because creating the SystemStackError triggers another stack overflow.

[eregon]

This has been fixed in SubstrateVM, and should be available in the next GraalVM release.
I verified the original report and get:

$ bin/ruby ex.rb
[ruby] WARNING StackOverflowError
ex.rb:1:in `call': stack level too deep (SystemStackError)
    from ex.rb:1:in `block in <main>'
    from ex.rb:1:in `call'
    from ex.rb:1:in `block in <main>'
    from ex.rb:1:in `call'
    from ex.rb:1:in `block in <main>'
    from ex.rb:1:in `call'
...
    from ex.rb:1:in `block in <main>'
    from ex.rb:2:in `call'
    from ex.rb:2:in `<main>'