Docs: CockroachDB performance page is out of date

[jseldess]

https://www.cockroachlabs.com/docs/v20.1/performance
https://www.cockroachlabs.com/docs/v20.2/performance

The 20.1 and 20.2 versions of the page have an out-dated note under "See also":

Also note that CockroachDB creates a yearly cloud report focused on evaluating hardware performance. In November 2019, we will provide metrics on AWS, GCP, and Azure. In the meantime, you can read the [2018 Cloud Report] that focuses on AWS and GCP.

Changes:

• We need to update that note to point to reference the 2020 cloud report.
• We need to check with @awoods187 (or another appropriate PM) to make sure the metrics on the page are still correct.

[awoods187]

I think we should make a quick fix here to point at the correct report and make a new issue to vet the other perf info. We will be testing out larger scales of tpcc among other things in the stability period.

[nvanbenschoten]

@jseldess Would this be a good time to explore improving the reporting of performance numbers and other graph visuals in our documentation? As is, the process is very ad hoc and this shows in the final result. https://www.cockroachlabs.com/docs/stable/performance.html contains graphs from three separate Google sheets generated years apart using completely different styling and labeling. This all looks a little off. It also makes the process of updating these results much more of a burden and infeasible to ever automate.

We can compare this against the performance graphs in the SIGMOD paper. There, performance numbers were collected in .csv assets and graphs were generated from these. This meant that we had consistent styling across all graphs and that it was easy to update these results.

[johnrk]

cc @rmloveland

[rmloveland]

@nvanbenschoten I completely agree re: improving perf graphs. I looked at the CSV file you linked and it looks great! I also found the R script used to generate the charts and think we should do something similar for docs.

I don't have an R environment set up (yet) but I can either use/tweak that or write something new in Python that lives in the docs repo's scripts dir and emits Gnuplot or similar depending on what output we are happy with.

[rmloveland]

I think we should make a quick fix here to point at the correct report and make a new issue to vet the other perf info. We will be testing out larger scales of tpcc among other things in the stability period.

Agree - @johnrk and I just chatted online and he is creating a different issue re: updating all the various perf things in docs for 20.2 so we can discuss details of what all needs to happen