Truffleruby: Failing async specs - various issues

[ioquatix]

The core of async is now passing on TruffleRuby is pretty awesome.

However there are failures in other gems, e.g. async-http, etc: https://travis-ci.org/socketry/async/jobs/513351431

You can run this huge test suite yourself by doing the following:

git clone [email protected]:socketry/async
cd async
bundle update
bundle exec rake external

This will test several gems which depend on async.

[aardvark179]

There are a few different underlying issues here. I'll summarise the ones I have identified and have fixes for so far:

1. Incomplete socket APIs (especially round the kwargs options)
2. Incomplete handling support for MRI's C API for kwargs.

There's some other issues I don't yet have full fixes for:

1. Extra methods are exposed on IO which breaks some tests
2. There is a locking problem I'm trying to track down which prevents the threaded tests from passing and causes them to deadlock
3. We don't yet support fork
4. You can't trap USR1 because it's already being trapped by the vm.

I think 3 and 4 on that list are best handled by changes to the test code, 2 definitely needs further investigation. I think most of the extra methods on IO can be safely made private with a little work, or moved into a Truffle specific package.

[ioquatix]

We can certainly change 3, but 4 seems like it might be more of a compatibility issue with other systems. It can certainly be changed, but why is it trapped by the VM? Everything else looks good. Thanks for your time investigating these issues!

[chrisseaton]

The VM uses these signals for internal services such as threads and the JIT. JRuby also does this, and I guess Rubinius to some extent but I'm not sure maybe not.

I went looking for an exact answer to your question about SIGUSR1 specifically and couldn't easily find it in the source code. I did find this document https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/signals.html which says it's used for 'the implementation of the java.lang.Thread.interrupt method' which is perhaps something to do with unsticking native methods.

We should document which signals the VM uses, and maybe we should give better errors that explain the problem.

[aardvark179]

It's going to be a little tricky to document the signals trapped by the VM, it can differ between platforms and OSes, may change with new JDK versions, can be changed with some command line options, and third party libraries or other languages may also want to trap signals. The situation is made even harder because there is no easy way to know that a signal is already trapped without trying to add a handler for it.

Would it be possible to refactor Async IO's use of signals to a pure pipe/select approach?

[ioquatix]

How does a user send a signal to a process using pipes and select?

[chrisseaton]

Hang on we'll try to fix signals for you before trying to change any of the user code - there might be some other options.

[eregon]

Re trapping SIGUSR1, this is a non-issue on SVM (native builds). It's only problematic on JVM.

[ioquatix]

Congratulations everyone, https://travis-ci.org/socketry/async/builds/542974447 is green.

[ioquatix]

I'm going to close this issue. The external test suite probably still has failures but we can address those individually.