Terminal: Port Windows Terminal to be a Microsoft Edge web browser as an extension and a PWA

[MOrlassino]

Description of the new feature/enhancement

Browsers have tabs and the browser and tab functionality build into Edge and other browser have all the proper key bindings for duplication, tab navigation/selection (e.g. ctrl+tab, ctrl+shift+tab), rearrangement, zoom/full-screen, moving the tab to a new window, web fonts, caret browsing, etc. Local and remote development and sysadmin functions are rapidly moving towards web rendered interfaces (e.g. VS Code uses Electron, Windows Admin Center, Azure Cloud Shell, etc.). That said, it might make sense to make the Windows Terminal operate embedded in the Edge web browser with an extension. Similarly, Windows Terminal, using the same technology underpinnings, could still be deployed as a stand-alone app (as a PWA) and distributed part of Microsoft Windows store, for those non-browser usage scenarios.

Proposed technical implementation details (optional)

Ingredient Technologies:

• Progressive Web Apps (PWA)
  
  
  
  • https://web.dev/progressive-web-apps/
  
  
  • https://developer.microsoft.com/en-us/windows/pwa/
  
  
  • https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps
  
  
• Edge Browser Extension
• WASM
• SSH, ConPTY, Named Pipes

[zadjii-msft]

You know, this was something I investigated at the onset of this project, and explicitly decided against. We've already got the excellent xterm.js that we help maintain, why not use that?

The conclusion we came to centered around one key set of facts:

• xterm.js is _currently_ a more complete terminal emulator than we currently had on hand.
• We plan on supporting this project for a while, not just a quick 1 year sprint and then wash our hands of this project.
• For any JS-based terminal, we need an HTML engine. At the time, we were considering WebViews in WPF, UWP, etc. An HTML engine incurs a pretty _substantial_ memory impact.
• No matter how long we'd work on a web-based terminal, we'd _never_ be able to optimize out the footprint of an HTML engine.
• If we supported our own terminal implementation for long enough, we'd pretty easily be able to reach feature parity with xterm.js.

These are some raw memory numbers from my original investigation in 2018.

Stack | Idle, Empty Buffer | Idle, full buffer | Scrolling | Worst-case
-- | -- | -- | -- | --
VT#(WPF+no renderer) | 34MB | 93.7MB | Peaks at ~100, mean 95MB |  
VT#(WPF+GlyphRun) | 35.9-38MB | 84.6-95MB | see above |  
VT# buffer+UWP | unavailable |   |   |  
xtermjs+UWP | 46.3MB | 69-73.1MB | 102-117, mean 110 MB |  
xtermjs+WPF | 28(+29)=57MB[1] | 37(+48.4)=86MB | 33.4-34(+101)=135MB |  
Prototype benchmarks |   |   |   |  
WPF+DX+native Buffer [4] |   |   |   | 22.4MB
VT#+optimal buffer [5] | 17.1MB | 19.6MB |   | 29.2MB
Other benchmarks |   |   |   |  
conhost+gdi | 6.8 MB | 6.8 MB | 6.8 MB | 22.5 MB
C++winrt CoreWindow [3] | 2.0MB |   |   |  
C++winrt Xaml app | 11.5MB->9.8 |   |   |  
CX UWP with nothing | 10.0MB |   |   |  
C# UWP with nothing | 10.8MB |   |   |  
UWP with webview | 21.6MB |   |   |  
WPF with nothing | 24.3MB->14.9 [2] |   |   |  
WPF with webview | 25.2+16.8=42MB |   |   |  

Now that we're past the hypothetical, in a world with an _actual_ Terminal application, we can look at how the _actual_ terminal is doing. Right now I've got the terminal running with 3 tabs, and 3 panes in one of the tabs, for 5 total terminal instances. WT is taking 30MB of memory by itself, and with conpty's and attached processes, the entire tree is sitting at 41MB. _Does even the simplest electron application take less than 41MB of memory to boot?_

So that's a bit of background on why we made these decisions in the first place.

Now, I don't know the most (anything) about browser extensions.

• Is there a way to embed arbitrary WinRT types in an edgeium browser tab?
  
  
  
  • We'd almost certainly be unable to use our current UWP TermControl in a browser, but maybe someone could build a WebTermControl that creates a HTML renderer for the existing Terminal implementation we have
  
  
  • Though, why would someone go through the trouble when they could just embed a xterm.js terminal instance and be done with it?
  
  
• How would launching external processes work? I'm pretty sure there's an enormous amount of work that's been done to _prevent_ browsers from running arbitrary executables on the user's machine. Do PWA's let you ignore that restriction? Because _launching processes_ is a pretty fundamental use of a terminal, so it wouldn't make sense to have a terminal running in a context where that's not possible, or heavily restricted 😆

I'm gonna leave this open for discussion, because I'd love to hear a compelling argument for why we'd _want_ Terminal in the browser, other than just "it's what everyone else is doing".

[WSLUser]

I honestly envision a time in which Windows Terminal became embedded in games, possibly web-hosted games. I say this because there are plenty of games that created their personal console for use in-game with their own syntax language for doing stuff in-game. The Elder Scrolls games in particular come to mind but so does the Quake series and they are both from the same vendor. I think once the few PRs pending for getting us VT compliant are merged in conhost (which almost certainly these in-game consoles were using in the backend for Windows), copying the VT levels over to Terminal should be mostly a simple breeze (just copy from the console dispatch to the terminal dispatch and do alittle optimizing). Then the "core" part of Terminal which Visual Studio even now uses could be re-used anywhere so long as a supporting UI accompanied it. I don't personally see a PWA being created or this project having a stand-alone web extension for browsers. It would be nice if we could browse tabs of everything in the Windows shell like was originally planned but then discarded but it's not necessary.

[jdhitsolutions]

Personally, I would not want to see this implemented as a browser-based solution. Certainly not one that would force me to use Edge.

[DHowett-MSFT]

This is a fascinating discussion; thanks for starting it! The terminal team is pretty deeply invested in our native code, so we're not really likely to do this.

Please feel free to continue the discussion, but I'm going to close out this issue as it's not really actionable for us without _years_ of work (to undock the Windows APIs, and the console subsystem, and all that jazz.)

[MOrlassino]

@jdhitsolutions, @WSLUser the idea was to refactor the base code so it could be delivered as a Windows store app but implemented using browser technologies behind the scenes. The Microsoft store app would continue to be the primary deployment mechanism that we all use and love. My mention of Windows Terminal running in a Edge as an extension and PWA was just a matter of convenience for developers that develop web apps and use web-based interfaces rendered in a web browser. It was not an either/or scenario.
@DHowett-MSFT, @zadjii-msft I now completely understand the native code issue, it didn't dawn on me that even the code was mostly C++. In any case, thank you for the clarification. In the long run, it probably makes more sense to focus on xterm.js with SSH as a PWA (and Store App). Once WASM, WASI and complier/transpiler ecosystem matures it will be a much easier endeavor.

Thank you for everyone's comments and taking the time to write detailed responses. Stay safe and healthy.