Rxjs: Record perf results vs RxJS 4, then drop RxJS 4 from perf testing

[benlesh]

It's time to move on. Future versions of this library need to be compared against previous versions of itself, not Rx4.

We need to

1. Record and keep the perf results vs RxJS 4, if for no other reason than documentation purposes.
2. Fix the perf tests to target a previous version of RxJS 5.

[benlesh]

cc @kwonoj @jayphelps

[manekinekko]

Hi @blesh are you still planning to use Benchpress for future perf tests? I'm actually building a tool called Perfy, on top of Benchpress which allows to keep track of previous tests campaign, which is exactly what you need. The tool lives here :https://github.com/manekinekko/perfy.

If this is something you wanna try out, I can prepare a POC for RxJS.

[benlesh]

@manekinekko that sounds awesome. I'd love to see it in action. I have a feeling our benchpress tests have probably fallen into disrepair.

Do note, though, that we have both micro perf and macro perf tests.

[manekinekko]

alright. I'll begin working on that just right after christmas 🎅

[manekinekko]

@blesh here is a quick POC for the macro perfs... tell if this is what you had in mind?

[kwonoj]

@manekinekko I'm curious if we could create github webhooks to display some critical info per each PR (let's not think about ergonomics like where to run server, etcs for now)

[manekinekko]

@kwonoj yep I think it's doable.

Here are some facts about Perfy:

• It runs after protractor/benchpress has ended executing specs tests.
• It overrides benchpress config in order to make benchpress outputs JSON files.
• It reads those JSON files and merges them into a single JSON file which serves as a database.
• It has a CLI interface.

Here is the complete workflow:

npm run tests --> protractor --> benchpress --> perfy --> charts

One thing I'm missing here is how we'll be able to provide a subset of spec tests to benchpress. Typically, we will need to:
1) run only the tests that are affected by the specific PR that is being created.
2) Or do we need to run the whole tests?

I think 2) would be a good approach 🤔

[claudiordgz]

Is benchpress still being supported? I don't see it in the latest modules in angular, it's supposed to be @angular/benchpress, but it's missing after angular 2.4. Seems like we need to overhaul all protractor and benchpress test code too if we want to move forward with this.

[benlesh]

This is just old now.

[manekinekko]

Haha! I totally forgot about this experiment 😁