Rust: A big performance regression in benchmarks

[ljedrz]

rustc performance data shows a pretty big regression between 74bd074e and 6050e52. I don't know if this is expected, but I'm putting this out there in case others see the dent in perf charts and wonder what had happened.

---

EDIT(@eddyb): we how have a single-PR perf comparison showing the regression for #69718.

70156, the other PR in the range, is much more of a mixed bag (but only for a small subset of runs).

[jonas-schievink]

cc @arlosi @eddyb

[eddyb]

The range is https://github.com/rust-lang/rust/compare/74bd074eefcf4915c73d1ab91bc90859664729e6...6050e523bae6de61de4e060facc43dc512adaccd so the candidates are:

• #70156 "Make the rustc respect the -C codegen-units flag in incremental mode."
• #69718 "Add hash of source files in debug info"

Neither PR looks like it was tested for perf individually.
EDIT: oops, missed https://github.com/rust-lang/rust/pull/70156#issuecomment-601855964 - so it's likely #69718. Slow MD5 implementation?

@Mark-Simulacrum I'm curious, why does 9e55101bb681010c82c3c827305e2665fc8f2aa0 (merge commit for #70156) not have its own perf run? Is there a way to trigger one now?

[Mark-Simulacrum]

9e55101 is in the queue: https://perf.rust-lang.org/status.html.

[arlosi]

It looks like this was caused by #69718. I'll start investigating why.

I timed several compilations of ripgrep locally in debug with and without #69718.

master (sec): 41 40 39 40 40
revert (sec): 34 34 34 36 34

[eddyb]

For the record, this is the single-PR perf comparison showing the regression from #69718.
I'll update the issue description to minimize potential confusion.

@arlosi Two things I'm seeing:

1. there are way worse regressions than ripgrep, you can focus on the worse ones to get better contrast in measurements (e.g. tokio-webpush-simple-debug even has "patched incremental" runs slowed down a lot, unlike many others, clap-rs-debug being another example I guess)
2. the detailed profile (if you click the percentage for a run) shows a lot of LLVM differences, e.g. for tokio-webpush-simple-debug "clean", and most of the regressions are -debug, with effectively no -check ones, so I think we might be able to get away without a revert, just adding a flag controlling whether LLVM sees these hashes (assuming that's the source of the slowdown)

[eddyb]

We have perf data for the fix (#70803), which at a glance looks right.

Before the regression to after the fix looks like noise, and so does after the regression to before the fix, so I believe this confirms the fix is complete.

For the cost of MD5 hashing compared to SipHash, based on the changes to -check runs in the original regression, it looks insignificant.