Deno: Panicking on opening too many files

[kevinkassimo]

Deno panics and hard crash on opening too many files, aka fildes count exceeding ulimit -n. Ideally it should throw a user space error instead.

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: "Too many open files" }', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread 'main' panicked at 'internal error: entered unreachable code', ../../third_party/rust_crates/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.19/src/runtime/threadpool/mod.rs:293:26
thread '<unnamed>' panicked at 'unexpected state while aborting task: Complete', ../../third_party/rust_crates/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:216:21

(Might have been a Tokio problem though)

[Fenzland]

I have the same problem with Deno.run.

Deno version: 0.4.0
OS: Ubuntu 18.04

(async ()=> {
    try{
        const process= Deno.run( 'my command' );
        const status= process.status();
        const output= process.output();
        await process.status();
        if( (await status).success )
            return process.output();
        else
            throw new Error();
    }finally{
        process.close();
    }
})();

The processes should be closed finally.

But after several request, deno crashes with:

Uncaught Other: Too many open files (os error 24)
    at DenoError (js/errors.ts:22:5)
    at maybeError (js/errors.ts:33:12)
    at maybeThrowError (js/errors.ts:39:15)
    at sendSync (js/dispatch.ts:114:5)
    at run (js/process.ts:189:19)
    at my file where Deno.run() called 

[kevinkassimo]

@Fenzland I think this is a different problem (since in my case the process crashed under panic). How many times do you run before this error occurs? (Usually ulimit -n should return a value 4000+)

[Fenzland]

I haven't accouted this. Maybe several hundreds requests.

[lyptt]

I've got this from trying to import a module that has a ton of files, my ulimit -n is 256 on macOS 10.14.5. For example:

import Three from 'https://unpkg.com/[email protected]/src/Three.js?module'

console.log(Three)

[kt3k]

Is this issue still remaining?

I tried with a small ulimit number and it seems throwing js errors.

$ ulimit -n 50
$ cat test.ts 
window.onload = function () {
  for (let i = 0; i < 100; i++) {
    Deno.open(`foo${i}`, 'w')
  }
}
$ deno test.ts -A
error: Uncaught Other: Too many open files (os error 24)
► js/errors.ts:33:12
    at DenoError (js/errors.ts:22:5)
    at maybeError (js/errors.ts:33:12)
    at handleAsyncMsgFromRust (js/dispatch.ts:41:17)
    at handleAsyncMsgFromRust (shared_queue.js:157:9)

[7k8m]

🙁

shell [tomohito.nakayama@shorthair deno (master=)]$ uname Darwin [tomohito.nakayama@shorthair deno (master=)]$ cat test.ts window.onload = function () { for (let i = 0; i < 100; i++) { Deno.open(`foo${i}`, 'w') } } [tomohito.nakayama@shorthair deno (master=)]$ ./target/debug/deno test.ts -A Compile file:///Users/tomohito.nakayama/dev/deno/test.ts thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: "Too many open files" }', src/libcore/result.rs:999:5 note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace. thread 'tokio-runtime-worker-0' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:999:5 fatal runtime error: failed to initiate panic, error 5 Abort trap: 6 [tomohito.nakayama@shorthair deno (master=)]$

[7k8m]

bash tmnk@calico:~/dev/deno$ RUST_BACKTRACE=1 ./target/debug/deno test.ts Compile file:///home/tmnk/dev/deno/test.ts thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Other, message: "Too many open files" }', src/libcore/result.rs:999:5 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. stack backtrace: 0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39 1: std::sys_common::backtrace::_print at src/libstd/sys_common/backtrace.rs:71 2: std::panicking::default_hook::{{closure}} at src/libstd/sys_common/backtrace.rs:59 at src/libstd/panicking.rs:197 3: std::panicking::default_hook at src/libstd/panicking.rs:211 4: std::panicking::rust_panic_with_hook at src/libstd/panicking.rs:474 5: std::panicking::continue_panic_fmt at src/libstd/panicking.rs:381 6: rust_begin_unwind at src/libstd/panicking.rs:308 7: core::panicking::panic_fmt at src/libcore/panicking.rs:85 8: core::result::unwrap_failed at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/macros.rs:18 9: core::result::Result<T,E>::unwrap at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/result.rs:800 10: deno_bin::tokio_util::create_threadpool_runtime at ../../cli/tokio_util.rs:14 11: deno_bin::tokio_util::block_on::{{closure}} at ../../cli/tokio_util.rs:53 thread 'tokio-runtime-worker-0' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', src/libcore/result.rs:999:5 stack backtrace: 0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39 1: std::sys_common::backtrace::_print at src/libstd/sys_common/backtrace.rs:71 2: std::panicking::default_hook::{{closure}} at src/libstd/sys_common/backtrace.rs:59 at src/libstd/panicking.rs:197 3: std::panicking::default_hook at src/libstd/panicking.rs:211 4: std::panicking::rust_panic_with_hook at src/libstd/panicking.rs:474 5: std::panicking::continue_panic_fmt at src/libstd/panicking.rs:381 6: rust_begin_unwind at src/libstd/panicking.rs:308 7: core::panicking::panic_fmt at src/libcore/panicking.rs:85 8: core::result::unwrap_failed at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/macros.rs:18 9: core::result::Result<T,E>::unwrap at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/libcore/result.rs:800 10: deno_bin::tokio_util::block_on at ../../cli/tokio_util.rs:57 11: deno_bin::ops::op_fetch_source_file at ../../cli/ops.rs:534 12: deno_bin::ops::dispatch_all_legacy at ../../cli/ops.rs:125 13: deno_bin::ops::dispatch_all at ../../cli/ops.rs:93 14: deno_bin::state::ThreadSafeState::dispatch at ../../cli/state.rs:113 15: deno_bin::worker::Worker::new::{{closure}} at ../../cli/worker.rs:37 16: deno::isolate::Isolate::pre_dispatch at ../../core/isolate.rs:332 17: _ZN4deno4SendERKN2v820FunctionCallbackInfoINS0_5ValueEEE at ./../../core/libdeno/binding.cc:248 18: _ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE at ../../third_party/v8/src/api/api-arguments-inl.h:158 19: _ZN2v88internal12_GLOBAL__N_119HandleApiCallHelperILb0EEENS0_11MaybeHandleINS0_6ObjectEEEPNS0_7IsolateENS0_6HandleINS0_10HeapObjectEEESA_NS8_INS0_20FunctionTemplateInfoEEENS8_IS4_EENS0_16BuiltinArgumentsE at ./../../third_party/v8/src/builtins/builtins-api.cc:111 20: _ZN2v88internalL26Builtin_Impl_HandleApiCallENS0_16BuiltinArgumentsEPNS0_7IsolateE at ./../../third_party/v8/src/builtins/builtins-api.cc:141 21: _ZN2v88internal21Builtin_HandleApiCallEiPmPNS0_7IsolateE at ./../../third_party/v8/src/builtins/builtins-api.cc:129 22: <unknown> fatal runtime error: failed to initiate panic, error 5 Aborted

shell tmnk@calico:~/dev/deno$ cat /proc/version Linux version 4.19.57-microsoft-standard (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP Wed Jul 3 20:47:25 UTC 2019

[7k8m]

Seeing stacktrace of Rust layer, core::result::Result<T,E>::unwrap in libcore of Rust seems to be failed.
https://github.com/rust-lang/rust/blob/master/src/libcore/result.rs

Not a problem of deno, but a problem of Rust ? 🤔

[ry]

@7k8m The crash is here
https://github.com/denoland/deno/blob/58f0e9b9b1b53ca486ef38ae662b98cbde839248/cli/tokio_util.rs#L57
unwrap() is the rust way of saying "if we get a runtime error, panic"...
(I can't say more without digging into this -- which I don't want to do right now).

[bartlomieju]

I'll be tackling this issue this week

[ry]

It seems v0.17.0 doesn't panic anymore.

~/src/deno> ulimit -n 50

~/src/deno> deno -v
deno: 0.17.0
v8: 7.7.200
typescript: 3.5.1

~/src/deno> cat t.js
window.onload = function () {
  for (let i = 0; i < 100; i++) {
    Deno.open(`foo${i}`, 'w')
  }
}

~/src/deno> deno -A t.js
error: Uncaught Other: Too many open files (os error 24)
► file:///Users/travis/build/denoland/deno/js/dispatch_json.ts:40:11
    at DenoError (file:///Users/travis/build/denoland/deno/js/errors.ts:20:5)
    at unwrapResponse (file:///Users/travis/build/denoland/deno/js/dispatch_json.ts:40:11)
    at sendAsync (file:///Users/travis/build/denoland/deno/js/dispatch_json.ts:85:10)

[bartlomieju]

Yes, I think issues described here are now resolved :tada:

[7k8m]

🎉
[tomohito.nakayama@shorthair deno (master=)]$ ./target/debug/deno test_2243.ts Compile file:///Users/tomohito.nakayama/dev/deno/test_2243.ts error: Uncaught Other: Too many open files (os error 24) ► file:///Users/tomohito.nakayama/dev/deno/js/dispatch_json.ts:40:11 at DenoError (file:///Users/tomohito.nakayama/dev/deno/js/errors.ts:20:5) at unwrapResponse (file:///Users/tomohito.nakayama/dev/deno/js/dispatch_json.ts:40:11) at sendSync (file:///Users/tomohito.nakayama/dev/deno/js/dispatch_json.ts:67:10) at fetchSourceFile (file:///Users/tomohito.nakayama/dev/deno/js/compiler.ts:143:15) at _resolveModule (file:///Users/tomohito.nakayama/dev/deno/js/compiler.ts:244:24) at getSourceFile (file:///Users/tomohito.nakayama/dev/deno/js/compiler.ts:350:16) at findSourceFile (typescript.js:90560:29) at typescript.js:90459:85 at getSourceFileFromReferenceWorker (typescript.js:90426:34) at processSourceFile (typescript.js:90459:13)