Crystal: Infinite recursion when assigning an @@ variable in module body

[jgaskins]

Code

This code (reduced about as well as I can but it still uses the ysbaddaden/pool shard):

require "pool/connection"

module Queries
  @@neo4j_pool : ConnectionPool(String) = reset_pool!

  def self.reset_pool!
    @@neo4j_pool = ConnectionPool(String).new do
      "hi"
    end
  end
end

Works as expected on v0.30.1 release but causes infinite recursion on master.

Linux stack trace

test@crystal-perf-test:~/app$ ../crystal/bin/crystal foo.cr
Using compiled compiler at `.build/crystal'
Unhandled exception: Could not create pipe: Too many open files (Errno)
  from /home/test/crystal/src/crystal/system/unix/file_descriptor.cr:0:7 in 'pipe'
  from /home/test/crystal/src/io.cr:148:5 in 'pipe'
  from lib/pool/src/pool.cr:29:14 in 'initialize:capacity'
  from lib/pool/src/pool.cr:28:3 in 'new:capacity'
  from foo.cr:7:20 in 'reset_pool!'
  from /home/test/crystal/src/signal.cr:338:3 in '~Queries::neo4j_pool:init'
  from /home/test/crystal/src/crystal/once.cr:255:3 in '__crystal_once'
  from foo.cr:7:62 in '~Queries::neo4j_pool:read'
  from foo.cr:7:5 in 'reset_pool!'
  from /home/test/crystal/src/signal.cr:338:3 in '~Queries::neo4j_pool:init'
  # begin infinite recursion

Truncated here for brevity, but it repeats everything between the signal.cr:338:3 lines until it runs out of FDs.

Info:

Crystal is the latest master branch (as of this writing)
Ubuntu 19.04

test@crystal-perf-test:~/app$ ../crystal/bin/crystal -v
Using compiled compiler at `.build/crystal'
Crystal 0.31.0-dev [e7e85367a] (2019-09-06)

LLVM: 8.0.0
Default target: x86_64-pc-linux-gnu

macOS Stack Trace

When I run it on macOS I got what I assume is an LLVM stack trace instead:

 CRYSTAL_PATH="/Users/jamie/Code/crystal/src:lib" ../crystal/bin/crystal run -Dpreview_mt src/app.cr
Using compiled compiler at `.build/crystal'
Unhandled exception: Could not create pipe: Too many open files (Errno)
Failed to raise an exception: END_OF_STACK
[0x1006779cb] *CallStack::print_backtrace:Int32 +107
[0x10063797b] __crystal_raise +91
[0x100637d4d] *raise<Errno>:NoReturn +189
[0x10069bdd5] *Crystal::System::File::open<String, String, File::Permissions>:Int32 +197
[0x100694e5f] *File::new<String, String, File::Permissions, Nil, Nil>:File +63
[0x10066db00] *CallStack::read_dwarf_sections:(Array(Tuple(UInt64, UInt64, String)) | Nil) +40256
[0x100663be3] *CallStack::decode_line_number<UInt64>:Tuple(String, Int32, Int32) +51
[0x100663462] *CallStack#decode_backtrace:Array(String) +290
[0x100663321] *CallStack#printable_backtrace:Array(String) +49
[0x1006de9b8] *Exception+@Exception#backtrace?:(Array(String) | Nil) +72
[0x1006de841] *Exception+@Exception#inspect_with_backtrace<IO::FileDescriptor>:Nil +113
[0x1006de5d2] *AtExitHandlers::run<Int32>:Int32 +434
[0x100810ece] *Crystal::main<Int32, Pointer(Pointer(UInt8))>:Int32 +126
[0x100642069] main +9

This is the complete stack trace, and it looks different from the Linux one, so I'm not 100% sure it's the same problem.

[ysbaddaden]

macOS stack is about failing to open the external dwarf file for printing the actual backtrace (the one you get on Linux), because of the FD exhaustion.

[asterite]

Reduced:

module Moo
  @@x = init

  def self.init
    @@x = [1, 2, 3]
  end

  def self.x
    @@x
  end
end

p Moo.x

In the stack trace we see:

Stack overflow (e.g., infinite or very deep recursion)
[0x104ce96db] *CallStack::print_backtrace:Int32 +107
[0x104cd1165] __crystal_sigfault_handler +181
[0x104d47bf3] sigfault_handler +35
[0x7fff5ce8cb5d] _sigtramp +29
[0x104d478ef] GC_malloc_kind +56
[0x104ceaaf9] *GC::malloc_atomic<UInt64>:Pointer(Void) +9
[0x104cc23b1] __crystal_malloc_atomic64 +17
[0x104d2bcb8] *Pointer(Int32)@Pointer(T)::malloc<Int32>:Pointer(Int32) +72
[0x104d325f9] *Array(Int32)@Array(T)#initialize<Int32>:Pointer(Int32) +137
[0x104d32551] *Array(Int32)@Array(T)::new<Int32>:Array(Int32) +81
[0x104d3245c] *Moo::init:Array(Int32) +28
[0x104cd1196] ~Moo::x:init +6
[0x104cc2357] __crystal_once +103
[0x104cd11be] ~Moo::x:read +30
[0x104d324d4] *Moo::init:Array(Int32) +148
[0x104cd1196] ~Moo::x:init +6
[0x104cc2357] __crystal_once +103
[0x104cd11be] ~Moo::x:read +30
[0x104d324d4] *Moo::init:Array(Int32) +148
[0x104cd1196] ~Moo::x:init +6

This is because with parallelism the class variables must be initialized just once and a specialized code exists for that. I don't have time to debug it or understand why it happens, I'm sure @waj will know why, but as a workaround I would avoid assigning to a class variable inside it's initializer.

[asterite]

That is, I would rewrite you code to this:

require "pool/connection"

module Queries
  @@neo4j_pool : ConnectionPool(String) = new_pool

  def self.reset_pool!
    @@neo4j_pool = new_pool
  end

  private def self.new_pool
    ConnectionPool(String).new do
      "hi"
    end
  end
end

[waj]

I just sent a PR that hopefully will help find these errors more easily: https://github.com/crystal-lang/crystal/pull/8172

So, this issue is not a bug and should be closed.

[jgaskins]

Sure, happy to close it when that PR is merged!