Crystal: Compiler spec failing on travis-ci because of insufficient memory

Created on 19 Oct 2018 · 10Comments · Source: crystal-lang/crystal

I noticed a lot of CI builds failing on travis-ci because they can't allocate enough memory at some point.

The failure is always fork: Cannot allocate memory (Errno) in spec Semantic: macro allows declaring class with macro expression (https://github.com/crystal-lang/crystal/blob/master/spec/compiler/semantic/macro_spec.cr#L569).

This really doesn't seem to make any sense. It doesn't always happen, but it comes up from time to time.

According to travis docs the machines running with Ubuntu Trusty on GCE have 7.5 GB of memory available. That should be plenty for compiling the std and compiler specs - even in one go.

bug infrastructure

Source

straight-shoota

Most helpful comment

Maybe we somehow blow up memory sharing? That is after a fork the OS lets the child use the parents memory until it tries to write to it, at that point it's copied. if we have a 2GB compiler process forking two times that's easily 6GB, if we fail to free unneeded memory after fork.

jhass on 19 Oct 2018

👍3

All 10 comments

If it always happens in that specific spec, we can change it or remove it. I don't see the need for invoking echo there. Plus, it won't work once we port the compiler to Windows.

asterite on 19 Oct 2018

But the intriguing question is: Why does it fail at this specific point? =) I can't see any indication for what might be a cause for this.

straight-shoota on 19 Oct 2018

I think it's the only spec that compiles a program that executes a macro that ends up invoking a shell command. I don't know if that requires more memory for some reason or what...

Also, I think right now we are calling make crystal std_spec compiler_spec docs, and for example make std_spec will execute crystal spec/compiler_spec.cr which will compile that and start executing the specs while still having all the memory occupied because of the compile process. So maybe we should split that into crystal build spec/compiler_spec.cr && ./compiler_spec.

asterite on 19 Oct 2018

jhass on 19 Oct 2018

👍3

@asterite the Makefile performs build and run in a different stages already $(O)/std_spec vs std_spec: $(O)/std_spec targets