Rust: Rustc endless loop out-of-memory and consequent SIGKILL in generic new type

Created on 29 Dec 2019 · 9Comments · Source: rust-lang/rust

Hi, i have made some changes for a crate stubborn-io to support a generic type that implements ToSocketAddrs from Tokio, but suddenly the code can't compile because it runs out of memory with 6, 8, 10 GB.

This is the code i'm trying to compile:

use super::io::{StubbornIo, UnderlyingIo};
use std::future::Future;
use std::io;
use tokio::net::ToSocketAddrs;
use std::pin::Pin;
use tokio::net::TcpStream;

impl<A> UnderlyingIo<A> for TcpStream
where A: ToSocketAddrs + Sync + Send + Clone + Unpin + 'static {
    fn establish(addr: A) -> Pin<Box<dyn Future<Output = io::Result<Self>> + Send>> {
        Box::pin(TcpStream::connect(addr))
    }
}

pub type StubbornTcpStream<A: ToSocketAddrs + Sync + Send + Clone + Unpin + 'static> = StubbornIo<TcpStream, A>;

if changed the type A for SocketAddr or String it then compiles and runs fine.

Source: https://github.com/dangerousplay/stubborn-io/blob/64bad09fbb398ceca0add972399cddd04751e378/src/tokio/tcp.rs#L1-L29

It fails to compile on x86_64-unknown-linux-gnu with stable, nightly.

I'm attaching the Heaptrack files for further analysis.
https://gofile.io/?c=Vnar2K

C-bug I-crash T-compiler

Source

dangerousplay

👍1

All 9 comments

It seems a problem with regex utilized in the rustc, unfortunately heaptrack memory report is strange, the execution time tracked is wrong besides the graph shows up the memory grow.

I'm thinking ways to better troubleshoot, but the side effects of the issue is hard.

dangerousplay on 29 Dec 2019

It'd be helpful to try and simplify the code, ideally to something that doesn't have dependencies (beyond std or so), or at least as few dependencies as possible.

As a start putting up a github repository with a Cargo.toml and as simple a crate as possible would be helpful for reproducing on our side (i.e., extracting the file you've linked from that crate -- or, if the crate is really necessary, then indicating as such).

The heaptrack profile looks pretty useless unfortunately -- rustc on linux is shipped with jemalloc as the allocator which likely makes heaptrack tracking not really work too well.

Mark-Simulacrum on 29 Dec 2019

👍1

Thank you, sorry for the late response (working) but i get it working on playground.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=4512361986a733a75f43705ca512a736

I'm now trying to minimize them to focus on the issue.

dangerousplay on 31 Dec 2019

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=562a19ca1324c9c57b06149d0e0dd19a

Got it down to 18 lines. Still produces the same error. I have not tested if this goes out-of-memory. But receiving a SIGKILL would _signal_ that.

EDIT:
After taking some more time, I got rid of Tokio and reduced it to 6 lines:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=110e3925bb9e920ccca499b059cf2cc3

Additionally, I advise to rename this issue to something like out-of-memory and consequent SIGKILL in generic new type to make it very clear and concise.

Byter09 on 31 Dec 2019

One liner: pub type T<A: Send + Sync + Clone> = A;.

Strangely, this no longer crashes if you remove any of pub, Send, Sync, or Clone.
There's nothing special about Send or Sync, this also crashes:

#![no_std]
mod a {
    pub trait A {}
}
pub trait B {}
pub trait C {}
pub type T<P: a::A + B + C> = P;

Even weirder, this does _not_ crash: pub type T<P: B + C + a::A> = P;

(I didn't come up with the one-liner, that was @pythondude325)

jyn514 on 31 Dec 2019

👀1 😄1

I stopped rustc (SIGSTOP) and got a backtrace with gdb, it looks like it's trying to generate infinite errors:

backtrace

#0  0x00007ff429b08329 in rustc_errors::styled_buffer::StyledBuffer::putc ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#1  0x00007ff429b21fb7 in <rustc_errors::emitter::EmitterWriter as rustc_errors::emitter::Emitter>::emit_diagnostic ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#2  0x00007ff429b372a9 in rustc_errors::HandlerInner::emit_diagnostic ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#3  0x00007ff429b0662b in rustc_errors::diagnostic_builder::DiagnosticBuilder::emit
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#4  0x00007ff427eb00ec in <rustc_lint::builtin::TypeAliasBounds as rustc::lint::LateLintPass>::check_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#5  0x00007ff427e40ca6 in <rustc_lint::BuiltinCombinedModuleLateLintPass as rustc::lint::LateLintPass>::check_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#6  0x00007ff427e6652e in rustc::hir::intravisit::Visitor::visit_nested_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#7  0x00007ff427e6850d in rustc::lint::context::late_lint_mod ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#8  0x00007ff427df5c8a in rustc::ty::query::__query_compute::lint_mod ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#9  0x00007ff427e1263b in rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::lint_mod>::compute ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#10 0x00007ff427dfa04d in rustc::dep_graph::graph::DepGraph::with_task_impl ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#11 0x00007ff427e25174 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin---Type <return> to continue, or q <return> to quit---
/../lib/librustc_driver-63848c3330cc991c.so
#12 0x00007ff427e12c25 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::ensure_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#13 0x00007ff427d57876 in rustc::lint::context::check_crate::{{closure}}::{{closure}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#14 0x00007ff427d582f6 in rustc::util::common::time ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#15 0x00007ff427d58956 in rustc::util::common::time ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#16 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#17 0x00007ff427de3f73 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#18 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#19 0x00007ff427d5e4e2 in rustc_interface::passes::analysis::{{closure}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#20 0x00007ff427d5df45 in rustc_interface::passes::analysis ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#21 0x00007ff427ce8481 in rustc::ty::query::__query_compute::analysis ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#22 0x00007ff427ceac7d in rustc::dep_graph::graph::DepGraph::with_task_impl ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#23 0x00007ff427c633c7 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#24 0x00007ff427ceda3a in rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}} ()
---Type <return> to continue, or q <return> to quit---
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#25 0x00007ff427daf92a in rustc_interface::passes::create_global_ctxt::{{closure}}
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#26 0x00007ff427ced78e in rustc_interface::passes::BoxedGlobalCtxt::enter ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#27 0x00007ff427cbd7c7 in rustc_interface::interface::run_compiler_in_existing_thread_pool ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#28 0x00007ff427c9e112 in std::thread::local::LocalKey<T>::with ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#29 0x00007ff427c9bbce in scoped_tls::ScopedKey<T>::set ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#30 0x00007ff427cca264 in syntax::with_globals ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#31 0x00007ff427ce28d0 in std::sys_common::backtrace::__rust_begin_short_backtrace
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#32 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#33 0x00007ff427ce3809 in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#34 0x00007ff42777ae2f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once ()
    at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
#35 0x00007ff4277a8e70 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once ()
    at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
#36 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#37 std::sys::unix::thread::Thread::new::thread_start ()
    at src/libstd/sys/unix/thread.rs:79
#38 0x00007ff4275156db in start_thread (arg=0x7ff425bff700) at pthread_create.c:463
---Type <return> to continue, or q <return> to quit---
#39 0x00007ff426e3288f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

jyn514 on 31 Dec 2019

It looks like this is a problem with the error reporter itself: it's affected by the length of the name (!!)

OOM:

pub trait AAAAAAAAAAAAAA {}
pub trait B {}
pub type T<P: AAAAAAAAAAAAAA + B> = P;

No OOM:

pub trait AAAAAAAAAAAAA {}
pub trait B {}
pub type T<P: AAAAAAAAAAAAA + B> = P;

jyn514 on 31 Dec 2019

👀1 😕1

I figured out the issue has to do with how the error printer places the annotations, adding a newline after the trait names and before the plus solves the issue.

pub trait AAAA {}
pub trait B {}
pub trait C {}
pub type T<P: AAAA
+ B + C> = P;

pythondude325 on 31 Dec 2019

Probably caused by some code in here: https://github.com/rust-lang/rust/blob/master/src/librustc_errors/emitter.rs#L792-L849

pythondude325 on 31 Dec 2019

Was this page helpful?

0 / 5 - 0 ratings

Related issues

core::marker::Send and raw pointers

drewcrawford · 3Comments

Callback parameter names are missing from rustdoc

dtolnay · 3Comments

Inherited trait bounds are not completely checked

behnam · 3Comments

decl_macro: imports don't work in cross-crate invocations

tikue · 3Comments

Tracking issue for the `iterator_for_each` library feature

cuviper · 3Comments