Vscode: Improve build perf

[joaomoreno]

Several perf problems with the build that need to be addressed:

• All agents do the same compilation/optimization/minification
• Each extension get compiled/optimized/webpacked 6 (!) times
• node.exe gets downloaded 2 times
• macOS agents seem slower than before, though they end up doing more than others:
  
  
  
  • Sourcemaps
  
  
  • Bundle stats (Submitted bundle stats)
  
  

DONE :heavy_check_mark:

• [x] Remove VSCODE_WEB hack, create vscode-web-* tasks
• [x] De-duplicate compilation tasks
  
  
  
  • [x] Remove old extensions compilation
  
  
  • [x] Dedupe extensions build
  
  
  • [x] Cleanup Node.JS downloading
  
  
  • [x] Dedupe Node.JS downloading
  
  
  • [x] Dedupe vscode build
  
  
  • [x] Alpine
  
  
  • [x] ARM
  
  
• [x] Verify Webpack catches TS type errors
• [x] Use the Yarn cache task
• [x] Run common steps only once in a separate Linux build agent
  
  
  
  • [x] Hygiene
  
  
  • [x] Compilation
  
  
  • [x] Optimization & minification
  
  
  • [x] Cache result in VSTS artifacts, such that other agents and successive build resubmissions can simply pick that up
  
  
• [x] Have VSCODE_COMPILE_ONLY flag
• [x] Get template for arm/alpine
• [x] Remove node build/lib/builtInExtensions.js
• [x] Move quality/*/package.json dependencies into actual package.json dependencies
• [x] Deduplicate extension compilation https://github.com/microsoft/vscode/pull/77058
• [x] Cleanup VSCODE_ARCH usages in Linux build https://github.com/microsoft/vscode/pull/73119/commits/7b0cd3259167fb4351c841e24c13898d13419fc5
• [x] Set sensible values for task timeout
• [x] Lower agent parallel jobs
• [x] Move sourcemaps to common agent
• Move bundle stats to common agent
• Move upload-configuration to common agent

---

Task | Step | Windows | Linux | macOS | Unit
-- | -- | -- | -- | -- | --
Prepare build |   | 11 | 6.5 | 7.5 | min
Build |   | 25 | 21 | 45 | min
  |   |   |   |   |  
VS Code | compile-client-build | 170 | 160 | 215 | sec
  | optimize-vscode | 31 | 31 | 86 | sec
  | minify-vscode | 152 | 139 | 290 | sec
  | vscode-win32-x64-min | 259 | 191 | 436 | sec
  |   |   |   |   |  
Server | compile-client-build | 146 | 134 | 198 | sec
  | optimize-vscode-reh | 5 | 5 | 16 | sec
  | minify-vscode-reh | 27 | 23 | 79 | sec
  | vscode-reh-win32-x64-min | 169 | 113 | 312 | sec
  |   |   |   |   |  
Web | compile-client-build | 148 | 136 | 188 | sec
  | optimize-vscode-reh | 27 | 28 | 64 | sec
  | minify-vscode-reh (web) | 134 | 120 | 273 | sec
  | vscode-reh-win32-x64-min (web) | 167 | 116 | 329 | sec
  |   |   |   |   |  
Unit |   | 1.5 | 1 | 1.5 | min
Integration |   | 9 | 0 | 8 | min
Publish |   | 6 | 4 | 0.5 | min

[joaomoreno]

Removing old extensions build and deduping extension webpacking across all three targets brought the build time down from 1114s to 379s: 3x speedup.

[joaomoreno]

Deduping TypeScript compilation brought it further down to 264s: 4x speedup.

[joaomoreno]

Merged deduplication effort. Here are the improvements for the build step, given two sample builds:

Step | Before | After | Speedup
-- | -- | -- | --
Windows | 23m | 15m | 1.5x
Windows32 | 27m | 15m | 1.8x
Linux | 19m | 10m | 1.9x
LinuxArmHf | 6m | 6m | 1x
LinuxAlpine | 14m | 8m | 1.75x
macOS | 41m | 16m | 2.5x

[joaomoreno]

Adding Yarn cache:

Step | Before | After | Speedup
-- | -- | -- | --
Windows (x64, ia32) | 9m | 4m | 2.3x
Linux (x64, arm, alpine) | 4m | 2m | 2x
macOS | 4m | 3m | 1.3x

[joaomoreno]

Here are some really exciting numbers for the shared agent which runs hygiene checks, telemetry gathering and core & extensions compilation:

Type of Build | Duration
-- | --
No Cache | 15m
Cached Dependencies (99% of builds) | 10m
Rebuild Same Commit | 3m

Platform specific agents can just pick up this agent's work and proceed with testing, packaging, signing and publishing.

[joaomoreno]

Some more preliminary numbers, for total build times:

Step | Before | After | Speedup
-- | -- | -- | --
Windows | 58m | 44m | 1.3x
Windows32 | 59m | 46m | 1.3x
Linux | 38m | 23m | 1.7x
LinuxArmHf | 22m | 21m | 1x
LinuxAlpine | 23m | 26m | 0.9x
macOS | 68m | 42m | 1.6x

There's still a bit more to go through. We're still compiling extensions twice unnecessarily, for example.