PowerShell Core Servicing model

[iSazonov]

The Issue is open to collect ideas about PowerShell Core Servicing model.

By Servicing model we mean releasing new versions, feature and security patches.

We'd rather not discuss small details here, but collect common scenarios with pros and cons to help PG team make a weighted decision.

[iSazonov]

From #5458 @SteveL-MSFT:

PSCore6.0.0 will be what we call modern lifecycle which effectively means fix forward only. Barring any critical security issue, customers are expected to install the latest release version (currently planned at 6 month intervals).

[markekraus]

I have concerns about several cmdlets being tied to the rest of the project. 6 months is a pretty rapid cycle, however, for some cmdlets there is a need to possibly be even more rapid. Specifically the web cmdlets, but there may be others. These cmdlets are susceptible to the same security issues as web browsers which are on a more rapid release cycle to address the ever changing security landscape of the internet. Even curl is in a very rapid release cycle.

I don't want to rush the project for these kinds of fixes, and I don't want the rest of the project to slow them down.

If anything, if there is some decent way to decouple the release cycle of the individual modules so they could be updated independently (provided those updates don't break functionality in other modules), that would be great. Most users may not take advantage of that, but some may find it critical.

I realize that users always have the option to build from source to get new features and such out of band with the release cycle. But that's not ideal, especially for windows admins who are not used to that method (where as *nix admins are probably more familiar with building from source). I also realize that they could always grab the nightly build and install, but, sometimes that means broader changes than warranted from the issue the update is being done to address.

[iSazonov]

VS Code - publish new stable release every month.

• Pros - Users get bug fixes, security patches and new features very fast.
• Cons - Without fully automating the build process, this will require a noticeable effort from the team.

VS Code has a manager for extentions.

• Pros - Users can easily install minimal PowerShell and install additional modules as need.
• Cons - The need to install core components (normally present in one package) as additional can annoy users.
• Cons - Deploying from a common distributive to a single step (click) is very convenient in most scenarios and saves time.
• Cons - Not all systems have access to Internet.
• Cons - it's overkill for PowerShell - the need for multiple packages (unbundling some features in seperate modules) to get fast updates disappears if we release new versions fast.

VS Code has insiders releases which contains new, experimental and unstable features.

• Pros - We can freely create any patches without fear of disrupting the work of users or their (public) services.
• Cons - Supporting two major branches (public and insiders) requires additional effort from the team.

Exchange Server team releases new version every 3 months (quarter) and the build expires 6 months later. Also security patches can be released for previous non-expired builds. Release is full (cumulative) discributive. Exchange Server team provides insider builds for TAPs to test before public release.

• Pros - Build expiration idea is a good complement to the modern lifecycle concept.
• Cons - 6 months expiration looks very short for PowerShell Core which is platform for many products.
• Cons - Releases every 3 months is very good for a heavy class product like Exchange Server but very long for a fast-growing product like PowerShell Core.

Conclusions

Servicing model:

• follow modern servicing model like Windows and other MSFT products - no releases with patch holes, to get a patch users should install new version with al new patches.
• have two branches - public and insiders.
• add new patches in insider branch and move it in public branch every month before build public.
• build daily insider releases 6.0.0-insider-daily every day.
• build main public releases 6.x.0 every six months.
• build minor public releases 6.0.y every one month.

Distributives:

• minimal PowerShell Core (no PSReadLine, Help, remoting, Pester, PowerShellGet) - for local automation.
• minimal PowerShell Core with remoting - for Nano Server.
• full PowerShell Core (PSReadline, Help, remoting, Pester, PowerShellGet) - for common scenarios.

[iSazonov]

Based On SymVer 2.0 and modern life cycle we could give up on "beta" and "preview" and use 6.1.x on a daily basis.
I'd expect the scheme:

• 6.1.x-daily
  then for next release:
• 6.1.x-preview1
• 6.1.x-RC1
• 6.1.x-GA

[iSazonov]

Interesting question is about working on long time features which is breaking changes - in theory we should make its in 7.0.0 branch but if we follow modern servicing model we are limited to one branch 6.1.x.

[markekraus]

@iSazonov I had the same question. If we can't put breaking changes into 6.x.x per semver, then we would only be able to accept breaking changes to a 7.0.0 branch. I have one breaking change in the pipe with regards to legacy -Credential usage on the web cmdlets which couldn't get approved before RC. I have no clue how to manage that change now. Do I have to sit on in until we get close to a 7.0.0? can e into breaking changes into 6.x?

[SteveL-MSFT]

@markekraus at some future time, it may make sense to pull the webcmdlets out of the Utility module into a separate module that is not tied to a specific PSCore6 release. At that time, it would have it's own versioning and can be v8.1 while PSCore6 is only at v6.3, for example.

[joeyaiello]

@iSazonov I have a non-binding, very WIP document in PR at https://github.com/PowerShell/PowerShell-Docs/pull/1994

Would love some feedback there if you (or anyone else) has some.

[iSazonov]

@joeyaiello Thanks! I thought about it in the last month and share my thoughts tomorrow here.

[iSazonov]

Looking at the .Net Core and the .Net Framework, we see a convergence based on .Net Standard 2.0. Already 70% of nuget packages support this standard. However, many APIs are still not ported. This year we expect significant further progress.

Here we see three main things that we should consider when considering the service model:

1. Intensive development of .Net Core (and therefore PowerShell Core) in the absence of commercial projects with long life time.
2. Still a significant number of APIs are not ported, not all critical issues are identified and corrected.
3. Convergence of .Net Core and .Net Framework

The main criterion is minimization of efforts. This means minimizing the number of branches. By limiting the service model we reduce the number of branches.

Below I am trying to show that the modern service model should be limited in the next year or two, because it creates significant difficulties, the overcoming of which requires unreasonable expenses.

An interesting question arises with the .Net Core service updates. These updates can be critical and security updates. So we have to support them. But this is a serious problem - they can break something, we need to make a new distro, we do not have traditional tools (Windows Update) to force users to update. The release of the new service version will require considerable effort from us - .Net Core can release service updates every month. Microsoft may have sufficient internal resources to release a new service version of PowerShell Core, but taking into account the intensive development of .Net Core and PowerShell Core, these efforts will not be in demand today - new versions will be much more functional and attractive for consumers. We should limit the service model in this area to a year or two as described above until there are commercial projects based on PowerShell Core with a long life time. This should also be reflected in the documentation - support or not support.
If we support, the new release model every 6 months loses its meaning - we will have to just release the new version every month (6.0.1-Jun, 6.0.2-Feb,...). Support for previous versions also loses its meaning - we are forced to reject all previous versions in support. We can return to longer-term support if commercial projects request it. Only most likely they will also use the modern model.
The question is how we will develop on the versions of .Net Core that have not yet been released. This implies that we need two branches - release and Dev.
All this convinces me that it is better to postpone for a year or two.

Almost all PowerShell Core subsystems can be improved and enhanced. PowerShell Core also implements only a subset of the capabilities of Windows PowerShell (40-60%). This means that the PowerShell Core code base will be changed intensively, including breaking changes. We have them already done a lot. Most likely there will be more. Implies making such changes to the master branch. Banning this makes no sense while we are at the beginning of the way. We could ignore this and indicate in the documentation that the service model in the next year or two admits that minor versions will contain breaking changes as inevitability.

Convergence brings other thoughts, but they're less related to servicing model.

[bergmeister]

With regards to the security updates of .Net Core: I suggest to keep all the branches of releases that are supported (6.0, 6.1, etc) and monitor for .Net Core patches. Should a patch arrive, then we update the .Net Core version in the release branch, recompile and ship it e.g. 6.0.1. By definition, patches are not breaking so I do not see a problem here and should there be a breaking change then we can always ping back to corefx and request another patch or a custom build of the patch that suits our needs. We only have to make sure that when someone runs sudo apt-get update that it fetches only patches but not minor versions (unless the current version has become unsupported). For Windows, we could add a patch-checker into the product or as a separate tool.
With regards to the question of .Net Core updates: In the first phase when we do not have an LTS version yet, I do not see the need of having to use an officially released/supported version of .Net Core. As long as the version of .Net Core to be shipped with PowerShell has been used for more than one month and had RC testing then the risk is negligible IMHO. There will always known and unknown bugs in new versions of software anyway and the modern lifecycle means to me that the customer should roll back the PowerShell upgrade if it causes a blocking problem, report it and wait for the next release with the fix. Therefore I think we should support the current and previous version.
What would be nice for the user if one could upgrade/change the version of .Net Core when using it directly via the [Namespace]::Method() syntax. I use this feature of calling .Net directly in PowerShell a lot when developing/testing .Net code like e.g. regular expressions because it is so much faster compared to having to run a unit test or dummy project.

[dragonwolf83]

On Windows, is it feasible to only submit security patches to _Microsoft Update_ much like Office, .NET, and SQL Server do today? That fits into the existing workflow for most companies making it the best option for us.

Major and Minor updates can follow their own distribution methods.

[Jaykul]

For what it's worth, I think it would be very useful to pull the web cmdlets out -- if only because they should work on older versions of PowerShell. I've actually been thinking about doing that even if you don't...

Being able to make breaking changes to the cmdlets would be a mixed blessing.

If you extract a Microsoft.PowerShell.Commands.Web and distribute it via the gallery, what would happen to the built-in ones? What about security fixes?

Assume you've got an incompatible new version of the web cmdlets, and you find a security problem. You need to patch it in the external module, but do you also need to patch the old version and release a new PowerShell release to ship that fix?

You're going to have to keep all of that in mind when you're talking about servicing models...

[joeyaiello]

Given that we have a servicing model now at https://aka.ms/pslifecycle, I'm closing this one out