Rust: Spurious "Command exited with code 259" on AppVeyor

[pietroalbini]

Some builds on AppVeyor are "failing" with an exit code of 259. That exit code is not a failure though, but it's the way Windows marks processes that didn't fully finish yet.

We contacted AppVeyor support about this.

cc @rust-lang/infra

[pietroalbini]

We catched a spurious failure after we merged some debug code in https://github.com/rust-lang/rust/pull/57975, and the output is (full log):

Build completed successfully in 2:59:39
script exited with 0
Command exited with code 259

I'll add a bit of extra debugging code, even though I'm pretty sure this is not our fault.

[pietroalbini]

Task list when it spuriously fails:

exit code in src/ci/run.sh: 0
tasklist:
Image Name                     PID Session Name        Session#    Mem Usage
========================= ======== ================ =========== ============
System Idle Process              0 Services                   0          4 K
System                           4 Services                   0        264 K
smss.exe                       344 Services                   0      1,044 K
csrss.exe                      452 Services                   0      3,764 K
wininit.exe                    504 Services                   0      3,916 K
csrss.exe                      512 Console                    1      9,364 K
winlogon.exe                   540 Console                    1      6,460 K
services.exe                   596 Services                   0      5,716 K
lsass.exe                      604 Services                   0     10,864 K
svchost.exe                    672 Services                   0      9,728 K
svchost.exe                    712 Services                   0      6,276 K
dwm.exe                        788 Console                    1     25,064 K
svchost.exe                    800 Services                   0     13,736 K
svchost.exe                    828 Services                   0     28,932 K
svchost.exe                    888 Services                   0     10,296 K
svchost.exe                    960 Services                   0     17,620 K
svchost.exe                    500 Services                   0     12,832 K
svchost.exe                    724 Services                   0      7,992 K
svchost.exe                   1040 Services                   0      9,992 K
svchost.exe                   1228 Services                   0     15,668 K
svchost.exe                   1436 Services                   0      9,284 K
svchost.exe                   1612 Services                   0      7,892 K
VSSVC.exe                     1824 Services                   0      6,120 K
explorer.exe                  2400 Console                    1     69,540 K
svchost.exe                   2552 Services                   0      3,984 K
dllhost.exe                   2688 Console                    1      5,016 K
Appveyor.BuildAgent.Inter     1968 Console                    1     92,236 K
msdtc.exe                     3272 Services                   0      7,172 K
cmd.exe                       5080 Console                    1      2,588 K
conhost.exe                    732 Console                    1      5,280 K
sh.exe                        3868 Console                    1      6,860 K
sccache.exe                   5108 Console                    1     23,980 K
msiexec.exe                   4324 Services                   0      6,144 K
sh.exe                        3560 Console                    1      4,572 K
tasklist.exe                  2208 Console                    1      5,912 K
WmiPrvSE.exe                  4512 Services                   0      6,396 K
location of sh: C:\msys64\usr\bin\sh.exe
C:\Program Files\Git\usr\bin\sh.exe

Full log

[Disasm]

At least 3 of these builds finished in >3 hours. Maybe this is the reason.
The same timeout problem happens with Travis:
https://travis-ci.com/rust-lang/rust/builds/101031463
https://travis-ci.com/rust-lang/rust/builds/101019099
https://travis-ci.com/rust-lang/rust/builds/100972043
https://travis-ci.com/rust-lang/rust/builds/100867455
https://travis-ci.com/rust-lang/rust/builds/100469492

[TimDiekmann]

https://github.com/rust-lang/rust/pull/58750 failed two times in a row because of this. Both failures on jobs with builds > 3hours.

[pietroalbini]

AppVeyor said they fixed the underlying cause of this failure :tada:

[Centril]

Happened again in https://github.com/rust-lang/rust/pull/59522#issuecomment-478097014.

[pietroalbini]

That was a spurious spurious failure, AppVeyor said it shouldn't happen again. We also added some debug code to better investigate it if it happens again.