Truffleruby: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0

[ioquatix]

https://travis-ci.org/socketry/async-io/jobs/551120214

[eregon]

Thanks for the report!
I will try to reproduce locally.

[eregon]

I can reproduce on master:

23.54s    error: <Async::Task:0x1ca60 failed> [pid=7893] [2019-07-15 16:04:39 +0200]
               |   RuntimeError: 0 (ArrayIndexOutOfBoundsException)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper$CallbackHelperNode.prepareCallbackArguments(LLVMNativeWrapper.java:150)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper$CallbackHelperNode.doCached(LLVMNativeWrapper.java:103)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperFactory$CallbackHelperNodeGen.executeAndSpecialize(LLVMNativeWrapperFactory.java:96)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperFactory$CallbackHelperNodeGen.execute(LLVMNativeWrapperFactory.java:57)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper.execute(LLVMNativeWrapper.java:85)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperGen$InteropLibraryExports$Cached.executeAndSpecialize(LLVMNativeWrapperGen.java:97)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperGen$InteropLibraryExports$Cached.execute(LLVMNativeWrapperGen.java:84)
               |    from com.oracle.truffle.nfi.impl.LibFFIClosure$CallClosureNode.execute(LibFFIClosure.java:165)
               |    from com.oracle.truffle.nfi.impl.LibFFIClosure$ObjectRetClosureRootNode.execute(LibFFIClosure.java:226)
               |    from com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:102)
               |   Translated to internal error
               |   → /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/cext_ruby.rb:20 in `accept_nonblock'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:210 in `async_send'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:56 in `accept'
               |     /home/eregon/code/async-io/lib/async/io/ssl_socket.rb:139 in `block in accept'
               |     /home/eregon/code/async-io/vendor/bundle/truffleruby/19.2.0-dev-46466eee/gems/async-1.20.1/lib/async/task.rb:228 in `block in make_fiber'
23.56s    error: <Async::Task:0x1ca68 RSpec::ExampleGroups::AsyncIOSSLServer::MultipleHosts failed> [pid=7893] [2019-07-15 16:04:39 +0200]
               |   OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: sslv3 alert handshake failure
               |   → /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/cext.rb:1160 in `rb_exc_raise'
               |     ruby.c:2089 in `rb_exc_raise'
               |     ossl.c:300 in `ossl_raise'
               |     ossl_ssl.c:1703 in `ossl_start_ssl'
               |     ossl_ssl.c:1755 in `ossl_ssl_connect_nonblock'
               |     /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/cext_ruby.rb:37 in `connect_nonblock'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:210 in `async_send'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:56 in `connect'
               |     /home/eregon/code/async-io/lib/async/io/ssl_socket.rb:45 in `connect'
               |     /home/eregon/code/async-io/lib/async/io/ssl_endpoint.rb:90 in `connect'
               |     /home/eregon/code/async-io/spec/async/io/ssl_server_spec.rb:97 in `block (4 levels) in <top (required)>'
               |     /home/eregon/code/async-io/vendor/bundle/truffleruby/19.2.0-dev-46466eee/gems/async-1.20.1/lib/async/task.rb:228 in `block in make_fiber'
 33.5s    error: <Async::Task:0x1ca78 timer task duration=10 failed> [pid=7893] [2019-07-15 16:04:49 +0200]
               |   Async::TimeoutError: run time exceeded duration 10s:
               |   <Async::Reactor:0x1ca70 stopped=false>
               |    <Async::Task:0x1ca68 RSpec::ExampleGroups::AsyncIOSSLServer::MultipleHosts failed>
               |        <Async::Task:0x1ca78 timer task duration=10 running>
               |        <Async::Task:0x1ca80 example runner failed>
               |            <Async::Task:0x1ca88 complete>
               |                <Async::Task:0x1ca90 accepting secure connection #<Addrinfo: 127.0.0.1:47708 TCP> running>
               |   → /home/eregon/code/async-io/vendor/bundle/truffleruby/19.2.0-dev-46466eee/gems/async-rspec-1.13.0/lib/async/rspec/reactor.rb:48 in `block (2 levels) in run_example'
               |     /home/eregon/code/async-io/vendor/bundle/truffleruby/19.2.0-dev-46466eee/gems/async-1.20.1/lib/async/task.rb:228 in `block in make_fiber'
    can select correct host
34.22s    error: <Async::Task:0x1cac0 failed> [pid=7893] [2019-07-15 16:04:50 +0200]
               |   RuntimeError: 0 (ArrayIndexOutOfBoundsException)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper$CallbackHelperNode.prepareCallbackArguments(LLVMNativeWrapper.java:150)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper$CallbackHelperNode.doCached(LLVMNativeWrapper.java:103)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperFactory$CallbackHelperNodeGen.execute(LLVMNativeWrapperFactory.java:47)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapper.execute(LLVMNativeWrapper.java:85)
               |    from com.oracle.truffle.llvm.runtime.interop.nfi.LLVMNativeWrapperGen$InteropLibraryExports$Cached.execute(LLVMNativeWrapperGen.java:81)
               |    from com.oracle.truffle.nfi.impl.LibFFIClosure$CallClosureNode.execute(LibFFIClosure.java:165)
               |    from com.oracle.truffle.nfi.impl.LibFFIClosure$ObjectRetClosureRootNode.execute(LibFFIClosure.java:226)
               |    from com.oracle.truffle.api.impl.DefaultCallTarget.call(DefaultCallTarget.java:102)
               |    from com.oracle.truffle.nfi.impl.NFIContext.executePrimitive(Native Method)
               |    from com.oracle.truffle.nfi.impl.NFIContext.executePrimitive(NFIContext.java:286)
               |   Translated to internal error
               |   → /home/eregon/code/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm/jre/languages/ruby/lib/truffle/truffle/cext_ruby.rb:20 in `accept_nonblock'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:210 in `async_send'
               |     /home/eregon/code/async-io/lib/async/io/generic.rb:56 in `accept'
               |     /home/eregon/code/async-io/lib/async/io/ssl_socket.rb:139 in `block in accept'
               |     /home/eregon/code/async-io/vendor/bundle/truffleruby/19.2.0-dev-46466eee/gems/async-1.20.1/lib/async/task.rb:228 in `block in make_fiber'
    it fails with invalid host

[eregon]

@ioquatix Minor thing: there is a missing require 'async/io/ssl_endpoint' in spec/async/io/ssl_server_spec.rb

[ioquatix]

Can you submit a PR? :)

[eregon]

PR submitted: https://github.com/socketry/async-io/pull/29

[eregon]

This was a bug in Sulong, fixed in https://github.com/oracle/graal/commit/dc40d6dc7bbc3070311a2b1809cafb861ce7cd0c, it will be in the next release.
With that, I only have one async-io spec failing:

  1) Async::IO::Stream performance (BLOCK_SIZE: 8192 MAXIMUM_READ_SIZE: 8388608) can read data quickly
     Failure/Error: super(string)

     ArgumentError:
       Result of string concatenation exceeds the system maximum string length
     # ./lib/async/io/buffer.rb:34:in `<<'
     # ./lib/async/io/buffer.rb:34:in `<<'
     # ./lib/async/io/stream.rb:244:in `fill_read_buffer'
     # ./lib/async/io/stream.rb:71:in `read'
     # ./spec/async/io/stream_spec.rb:101:in `block (4 levels) in <top (required)>'
     # ./vendor/bundle/truffleruby/19.2.0-dev-2b2a7f81/gems/async-1.20.1/lib/async/clock.rb:32:in `measure'
     # ./spec/async/io/stream_spec.rb:100:in `block (3 levels) in <top (required)>'
     # ./vendor/bundle/truffleruby/19.2.0-dev-2b2a7f81/gems/async-rspec-1.13.0/lib/async/rspec/reactor.rb:55:in `block (2 levels) in run_example'
     # ./vendor/bundle/truffleruby/19.2.0-dev-2b2a7f81/gems/async-1.20.1/lib/async/task.rb:228:in `block in make_fiber'

[eregon]

That specs does:

data = stream.read(4*1024**3)

which tries to read 4GB of data, that's just too big for TruffleRuby Strings which have a maximum length of around 2GB, i.e., a 32-bit signed integer, the same limitation as the maximum size of byte[] on JVM.
And so @read_buffer << chunk fails when it would become larger than 2GB.

Maybe we could transition to a String in native memory for that case, but I'm not sure if Ruby applications actually use Strings larger than 2GB.

[eregon]

When reading 1GB, it works fine. I made a PR: https://github.com/socketry/async-io/pull/30

[ioquatix]

Thanks for this.

I already anticipated and ran into a similar issue recently:

https://github.com/socketry/async-io/blob/master/lib/async/io/stream.rb#L236-L239

Why are we not hitting that limit? It should be around 8MB by default.

Is it because String itself cannot hold 4GB of data, even if read is getting chunks of 8MB?

[chrisseaton]

A TruffleRuby String cannot contain more than 2**31-1, or 2147483647 characters - that's the underlying limitation. It doesn't matter how the String is created.

We should perhaps list this on https://github.com/oracle/truffleruby/blob/master/doc/user/compatibility.md and make the error message more clear.

[ioquatix]

Is there anything that can make larger buffers in the JVM?

[chrisseaton]

I think @nirvdrum experimented - can you comment?

JRuby has the same restriction. @headius has this been a problem for other people in practice?

[ioquatix]

I think the main point is, if there are limitations, we document it, and additionally, we might need to figure out how this affects Ruby code in general. For example, the requirement of generating buffers in chunks by default rather than assuming one can fill 64-bit address space.

[headius]

@chrisseaton We get occasional reports, usually from people who didn't realize they were reading >2GB of data into memory. None of the potential workarounds would be compatible with regular JVM APIs, and since we have users across many different JVMs those options won't work for us.

[ioquatix]

A dedicated rope style data type which accumulated multiple strings and uses readv/writev would make a lot of sense.

[chrisseaton]

That's what we already do - a rope - and that is compatible with a standard JVM except for being able to write the whole thing out atomically. We don't use readv and writev but we could do fairly easily, and that could also be done on a standard JVM using JNI or some abstraction on top of that.

We didn't allow long strings yet as our current design permits you to flatten at any point. We'd have to disallow flattening of a string that would become too large.

[nirvdrum]

@chrisseaton's summary is basically correct. A couple years back I had worked on extending our String representation to permit long length strings. It's doable, but it's unfortunately a viral. E.g., we'd have to extend jcodings and joni to work with them as well, since many operations currently expect a byte[]. Likewise, anywhere we might want to convert to a Java String (e.g., Truffle interop) or make a JDK call, we'd have to accept that long strings wouldn't be permissible and throw a runtime exception.

A smaller concern was around memory consumption. Our ropes track the byte and character lengths, so each of those fields would now take up twice as much space.

In the end, I abandoned the work because the scope was just creeping up too much and we didn't have a use case in hand that really warranted the additional complexity.

[ioquatix]

That makes total sense, and encourages me that the correct solution is some kind of binary-only data buffer specifically for efficient I/O.

[eregon]

This is now documented in b4dbbb345586e851a4a9df540aa05248d55992b5.
async-io specs should all pass with the next release, so I'll close this.
@ioquatix Thank you for the bug report!

[ioquatix]

Thanks for fixing it so quickly!

[nirvdrum]

@ioquatix I'm a strong proponent of having a proper byte array type in Ruby. I'm not sure it buys us anything in terms of max array length, but it would allow for cleaner solutions for the IO use case. Please comment on https://bugs.ruby-lang.org/issues/13166 if you'd like to see something for Ruby 3.