Rust: internal compiler error: our vector went away? in proc macro expansion

[95th]

Hi, I have been trying to follow @jonhoo's stream on implementation of David Tolnay's proc-macro-workshop.

While implementing the seq with nested repetitions (source code: https://github.com/95th/proc-macro-workshop/tree/ICE) (commit: https://github.com/95th/proc-macro-workshop/commit/c2c7af72b7027381d24a7d8a95b7e0d47873a41b)

seq!(N in 1..100 {
    #(
        fn f#N () -> u64 {
            #( N + )* 2
        }
    )*
});

I got following ICE:

error: internal compiler error: our vector went away? - found Was(Some((Token(Token { kind: Ident("fn", false), span: Span { lo: BytePos(195), hi: BytePos(197), ctxt: #0 } }), NonJoint)))
  --> main.rs:13:9
   |
13 |         fn f#N () -> u64 {
   |         ^^

thread 'rustc' panicked at 'no errors encountered even though `delay_span_bug` issued', src/librustc_errors/lib.rs:347:17
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:84
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:61
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1030
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1412
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:65
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:50
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:188
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:205
  10: rustc_driver::report_ice
  11: <alloc::boxed::Box<F> as core::ops::function::Fn<A>>::call
             at /rustc/412f43ac5b4ae8c3599e71c6972112e9be4758fa/src/liballoc/boxed.rs:956
  12: proc_macro::bridge::client::<impl proc_macro::bridge::Bridge>::enter::{{closure}}::{{closure}}
             at /rustc/412f43ac5b4ae8c3599e71c6972112e9be4758fa/src/libproc_macro/bridge/client.rs:305
  13: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:468
  14: std::panicking::begin_panic
  15: <rustc_errors::HandlerInner as core::ops::drop::Drop>::drop
  16: core::ptr::real_drop_in_place
  17: <alloc::rc::Rc<T> as core::ops::drop::Drop>::drop
  18: core::ptr::real_drop_in_place
  19: rustc_interface::interface::run_compiler_in_existing_thread_pool
  20: std::thread::local::LocalKey<T>::with
  21: scoped_tls::ScopedKey<T>::set
  22: syntax::with_globals

rustc --version --verbose:

rustc 1.41.0-nightly (412f43ac5 2019-11-24)
binary: rustc
commit-hash: 412f43ac5b4ae8c3599e71c6972112e9be4758fa
commit-date: 2019-11-24
host: x86_64-unknown-linux-gnu
release: 1.41.0-nightly
LLVM version: 9.0

[Centril]

Blame for our vector went away?: @estebank

https://github.com/rust-lang/rust/blob/6b604a91b7667106495a60291a74ce66923c9a8a/src/librustc_parse/parser/mod.rs#L1313

cc @petrochenkov @matklad

[pnkfelix]

triage: @estebank has been cc'ed (tho' not yet assigned). P-high, removing nomination.

[estebank]

I've recently modified that code but didn't introduce it, so I'm not as familiar with this, I believe @petrochenkov might be a better person to be point on this. Having said that, I have some time to kill, I'll take a look at it.

[Aaron1011]

This will be fixed by https://github.com/rust-lang/rust/pull/72287

[Aaron1011]

This compiles on the latest nightly, and now just needs a test. I'm removing P-high under the assumption that writing a regression test for a fixed issue is not high-priority.