Rust: Compiler unexpectedly panics occasionally on Windows

[igmanthony]

Within all of my projects, when I run "cargo run" while in the directory of the project, the compiler occasionally panics and outputs:

warning: error copying object file `C:\Users\Ian_Anthony\svd2\target\debug\deps\svd2-6e54b28804aea753.3s4d4kekc4aqhd.rcgu.o` to incremental directory as `\\?\C:\Users\Ian_Anthony\svd2\target\debug\incremental\svd2-3ho7dmdfxg9p9\s-eytrqlmgkk-1skg6ha-working\3s4d4kekc4aqhd.o`: Access is denied. (os error 5)

warning: file-system error deleting outdated file `\\?\C:\Users\Ian_Anthony\svd2\target\debug\incremental\svd2-3ho7dmdfxg9p9\s-eytrqlmgkk-1skg6ha-working\3s4d4kekc4aqhd.o`: The system cannot find the file specified. (os error 2)

error: failed to remove C:\Users\Ian_Anthony\svd2\target\debug\deps\svd2-6e54b28804aea753.49lx1q7cxvpykyv0.rcgu.o: The process cannot access the file because it is being used by another process. (os error 32)

warning: Error finalizing incremental compilation session directory `\\?\C:\Users\Ian_Anthony\svd2\target\debug\incremental\svd2-3ho7dmdfxg9p9\s-eytrqlmgkk-1skg6ha-working`: The system cannot find the file specified. (os error 2)

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.24.0 (4d90ac38c 2018-02-12) running on x86_64-pc-windows-msvc

thread 'rustc' panicked at 'src\librustc\session\mod.rs:650: Trying to invalidate IncrCompSession `InvalidBecauseOfErrors { session_directory: "\\\\?\\C:\\Users\\Ian_Anthony\\svd2\\target\\debug\\incremental\\svd2-3ho7dmdfxg9p9\\s-eytrqlmgkk-1skg6ha-working" }`', src\librustc\session\mod.rs:1141:26
note: Run with `RUST_BACKTRACE=1` for a backtrace.

error: Could not compile `svd2`.

Previous to 1.24 I did not experience this. Generally if I re-run the code without changing anything (i.e., I see the error/panic, retype "cargo run" and hit enter in the console) it compiles properly (so the compilation error / panic is annoying, but not critical in my case). I use Windows at work and Linux (Xubuntu 17.10) at home. If I work on the same code at home (on my linux box) the compiler error doesn't ever occur. My thoughts are that it might be something due to incremental compilation on Windows, but I'm new to Rust (and compiled code in general) so I'm not sure.

`rustc --version --verbose`:

rustc 1.24.0 (4d90ac38c 2018-02-12)
binary: rustc
commit-hash: 4d90ac38c0b61bb69470b61ea2cccea0df48d9e5
commit-date: 2018-02-12
host: x86_64-pc-windows-msvc
release: 1.24.0
LLVM version: 4.0

[Michael0x2a]

I also ran into what seems to be a nearly identical issue just now, also on Windows.

I suspect based on similar errors I run into from time to time when working on other programs, that this may be due to conflicts between 3rd party file indexing programs and the compilation process.

In particular, if you're using any kind of antivirus software or any kind of automatic backup service like Dropbox, what can happen is that when you create or modify a new file, the external indexing program temporarily locks the file to scan it. This can cause issues if you then try deleting or moving the file soon after -- it's locked, so the operation fails. This problem is particularly acute if you're generating and deleting a bunch of files in rapid succession. (So basically, there's a race condition somewhere.)

I vaguely recall doing some research into fixing this, and found the easiest solution was to just retry the deleting operation two or three times with a short delay then give up if that doesn't work. And now that I'm re-thinking about this, a second solution might also be to just write-lock the file until the compilation process is completely done -- I think that would prevent some other process stealing the lock in the middle of compilation.

But I'm no Windows expert, and am certainly no Rust expert, so please take everything I'm saying with a grain of salt. (I figured I might as well just leave this info here to give anybody who decides to tackle this issue some leads to work with.)

[kobenauf]

I am getting a similar but different error when I run cargo run on a simple test project. rustc is the line giving the warning:

rustc --edition=2018 --crate-name grep_lite src/main.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=0456ed87980f8088 -C extra-filename=-0456ed87980f8088 --out-dir /home/kobenauf/code/grep-lite/target/debug/deps -C incremental=/home/kobenauf/code/grep-lite/target/debug/incremental -L dependency=/home/kobenauf/code/grep-lite/target/debug/deps --extern regex=/home/kobenauf/code/grep-lite/target/debug/deps/libregex-988e30d4a642ebea.rlib

warning: Error finalizing incremental compilation session directory `/home/kobenauf/code/grep-lite/target/debug/incremental/grep_lite-14tfocywywcnj/s-fdpofyvsd6-1dgd059-working`: Permission denied (os error 13)

I'm running this in WSL, Windows 10 1903.

ls -l of that incremental directory shows a bunch of .o and .bin files, and they are are all owned by me, as is the directory itself, and permissions are rw.

It's also worth noting, warning aside, the program seems to actually be generated and work.

[MoAlyousef]

I'm also having this problem recently after updating to 1.36 on wsl. It's annoying because it explodes compile times.

``cargo build Compiling hem v0.1.0 (/mnt/e/Med/dev/rst/hem) warning: Error finalizing incremental compilation session directory/mnt/e/Med/dev/rst/hem/target/debug/incremental/hem-3i2so4g3tkaqp/s-fe52czk8pr-bxxvut-working`: Permission denied (os error 13)

Finished dev [unoptimized + debuginfo] target(s) in 5.96s

That's 6 seconds for a hello world.

[andrew-w-ross]

@MoAlyousef I had the same issue, if your using vscode it's because of the file watcher.
Ignoring the target folder fixed it for me, add this to your settings.

{
    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/*/**": true,
        "**/target/**": true
    }
}

[MoAlyousef]

@MoAlyousef I had the same issue, if your using vscode it's because of the file watcher.
Ignoring the target folder fixed it for me, add this to your settings.

{
    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/*/**": true,
        "**/target/**": true
    }
}

Thank you...that fixed things.

[Barronli]

I am not using VScode, but I had something similar in my build of https://github.com/rust-osdev/bootloader in release mode. I reported it to bootloader project, but after some exploration, I believe it is a problem in rustc.

Symptoms:
When I build bootloader in WSL (windows subsystem Linux, ubuntu X84_64) with cargo xbuild --release --features binary, it reports error complaining that it cannot remove certain intermediate object file(s). The problem is reproducible every time.

I managed to generate the stack trace as shown below:

````error: failed to remove /mnt/d/devs/Rust/bootloader/target/x86_64-bootloader/release/deps/bootloader-589b595378e59014.bootloader.3pdwijpa-cgu.0.rcgu.o: Input/output error (os error 5)

thread 'rustc' panicked at 'aborting due to -Z treat-err-as-bug=1', src/librustc_errors/lib.rs:540:13
stack backtrace:
0: backtrace::backtrace::libunwind::trace
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
1: backtrace::backtrace::trace_unsynchronized
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
2: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:47
3: std::sys_common::backtrace::print
at src/libstd/sys_common/backtrace.rs:36
4: std::panicking::default_hook::{{closure}}
at src/libstd/panicking.rs:200
5: std::panicking::default_hook
at src/libstd/panicking.rs:214
6: rustc::util::common::panic_hook
7: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:481
8: std::panicking::begin_panic
9: rustc_errors::Handler::emit_db
10: rustc_errors::Handler::err
11: rustc_codegen_ssa::back::link::remove
12: rustc_codegen_ssa::back::link::link_binary
13: rustc::util::common::time
14: ::join_codegen_and_link
15: rustc_interface::queries::Query::compute
16: rustc_interface::queries::::link
17: rustc_interface::interface::run_compiler_in_existing_thread_pool
18: std::thread::local::LocalKey::with
19: syntax::with_globals
note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.39.0-nightly (760226733 2019-08-22) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z treat-err-as-bug -C opt-level=3 -C panic=abort -C debuginfo=2 --crate-type bin

note: some of the compiler flags provided by cargo are hidden
````

Line 12 and 11 above show that rustc tries to remove the intermediate objects after linking, then fails.

1. The problem only happens so far when I build the bootloader bin crate, not with other projects.
2. The error is benign (if compiled without -Z treat-err-as-bug). Application can run as expected.
3. The error does not happen if I build debug mode with cargo xbuild --feature binary.
4. I didn't find any process locking the object file(s) in problem. (I used ProcessExplorer, Lockhunter in windows side, and lsof in WSL side. I also disabled my antivirus file scanning.) Perhaps the file(s) are locked in a very short period.
5. I can remove the said object file(s) manually, and all of their rw permissions are set.

[Barronli]

I found if I move the bootloader project directory to be under the wsl filesystem (e.g., the home ~/ in my case), the bootloader just builds fine. The error only happens when the project is in a mounted windows filesystem (e.g., the disk D: or /mnt/d in my case).

I suspected the error was caused by some mismatch in the file permission handling between rustc and wsl for the mounted filesystem. But I cannot produce a small test case that generates the same error. I produced a binary (with std::fs::remove_file(), which is used by rustc linking) under ~/ that removes files under /mnt/d. It works fine. So it is probably not that simple.

[redstrike]

This issue is happening for me even without any vs-code instance running.

warning: Error finalizing incremental compilation session directory `/home/95th/btrs/target/debug/incremental/btrs-311gr3ac8npws/s-fjgujplfts-i503kt-working`: Permission denied (os error 13)

delete the target folder, exclude future target folders from being watched with files.watcherExclude (e.g. "/target/": true), rebuild and you won't get the message anymore.

Source: https://github.com/rust-lang/rust/issues/62031#issuecomment-604078882

That helped me fix this compiler's warning. I hope it could fix yours too.