Powershell: Move to .Net Core 2.1 RC1-Final

[iSazonov]

.Net Core 2.1 Preview2 is released.

We should move to the new version.

1. Roll-forward in RTM and Self-contained Application Servicing
   This can affect the creation of security patches.
2. Elimination of platform dependencies on libcurl
   We have some workarounds for libcurl. We should review this. "On Linux, you need to deploy libcurl yourself if you want to use the old HttpClient implementation."

[markekraus]

hmm.. Should we really use a preview release? It's one thing to test it in a branch or something, prep for changes, and test for regressions, but we shouldn't commit a Preview release of .NET Core into PowerShell's Master, IMO.

[iSazonov]

@markekraus Plan is to move .Net Core 2.1 Preview2 before PowerShell Core 6.1 Preview2.

.Net Core 2.1 policy is "no breaking changes in APIs".
My main concerns about the new HttpClient client but .Net Core allow use old implementation based on libcurl.

[SteveL-MSFT]

The PSCore6.1 release is essentially tied to the .Net Core 2.1 release because of some dependencies on 2.1. For the 6.1 Previews, I think it's fine to move to a dotnetcore Preview so we can find issues in dotnetcore and get those fixed before they move out of Preview.

[markekraus]

@SteveL-MSFT Ok, do we have a contract that states that we will not ship 6.1 until 2.1 is GA? What if 2.1 is delayed several months beyond the planned 6.1 GA? Will we revert and ship or will we delay release until 2.1 is final?

@iSazonov Right, no one is looking forward to getting rid of that libcurl dependency as much as I am. That doesn't mean we should merge a preview framework into master.

[iSazonov]

@markekraus I had the same fears when we moved from 1.0 to 2.0 but @daxian-dbw did this job perfectly!
It should be a lot easier now - it is 2.0 -> 2.1 not 2.0 -> 3.0. Initially it will be a PR that you can check locally before merging. We should do this as quickly as possible to identify all possible problems before GA.

[SteveL-MSFT]

@markekraus We plan to have a new minor GA approx every 6 months, but we're not bound to that timeline. So if 2.1 slips, 6.1 will slip with it. Note that PSCore6 will not always have a hard dependency on latest dotnetcore.

[jherby2k]

Oh please oh please do it. Right now my binary module doesn't work under PSCore because it uses some of the Span APIs in System.Memory, which seem to conflict with the library in the 2.0 SDK.

[iSazonov]

@SteveL-MSFT

Note that PSCore6 will not always have a hard dependency on latest dotnetcore.

From https://blogs.msdn.microsoft.com/dotnet/2018/04/11/announcing-net-core-2-1-preview-2/ :

dotnet publish now publishes self-contained applications with a serviced runtime version. When you publish a self-contained application with the new SDK, your application will include the latest serviced runtime version known by that SDK. When you upgrade to the latest SDK, you’ll publish with the latest .NET Core runtime version. This applies for .NET Core 1.0 runtimes and later.

[SteveL-MSFT]

@iSazonov I believe what that means is that when we publish with 2.1, we would automatically get a 2.1.1 servicing release if available, not that it would automatically move to 2.2-preview1 (for example) which is what I meant by "latest dotnetcore". Perhaps I should have said "latest dotnetcore preview".

[iSazonov]

Preview and RTM has different policy.
Previews "do not roll forward from one preview to another or from preview to RTM."
For RTM - "When you upgrade to the latest SDK, you’ll publish with the latest .NET Core runtime version. "

[SteveL-MSFT]

Correct, that means we need to explicitly move to latest Preview

[markekraus]

Has anyone successfully made this work? I tried updating all of the references and installing the latest SDK but TypeGen ends up doing nothing and then the build dies with a sea of red.

[iSazonov]

Seems TypeGen depend cyclically on dotnet restore.

[iSazonov]

@markekraus I tried 2.1 preview2 sdk and I see the issue. We dynamically create a temporary proj file to generate powershell.inc file but the file doesn't created. I don't discover a root cause. Could you push PR from your branch? Then I'd confirm that we catch the same issue.

@SteveL-MSFT Seems April 25 is a Preview2 milestone - is someone from MSFT working on the Issue tomove to .Net Core 2.1 Preview2?

[SteveL-MSFT]

@iSazonov I'm going to try to work on this today, otherwise we'll have to wait until after PSCore6.1-Preview2 to pick up dotnetcore-2.1-Preview2 (or whatever version they have at that time)

[iSazonov]

I see rc1 in their repo.

[markekraus]

@iSazonov I had it on a local branch and ended up deleting it when I couldn't get it to work. But that's the same issue I saw. The command to generate the powershell.inc completes without output or error btu the file is not created. If you let me know what branch in your repo I can skim the changes to see if it was the same. But it was basically changing all the version references to the new SDK and RuntimeFrameworkVersion references. I also updated all the netcoreapp2.0 to netcoreapp2.1.. not sure if that was the right thing or not.

[SteveL-MSFT]

@daxian-dbw TypeCatalogGen on this line runs, but it doesn't generate a dependency file if we build with dotnetcore 2.1-preview2. Can you take a look?

[iSazonov]

@markekraus See #6718

[markekraus]

@iSazonov and @SteveL-MSFT have you had any success getting VSCode/Omnisharp to play nice with the RC preview SDK? I'm trying to work on redoing the logic for -PreserveAuthorizationOnRedirect but I'm having 0 luck getting a working development environment up and running. if you have some tips or tricks.. I'd appreciate it.

I'm seeing this in my Omnisharp output

[fail]: OmniSharp.MSBuild.ProjectLoader
        The "ResolvePackageAssets" task failed unexpectedly.
System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: value
   at System.IO.FileStream.set_Position(Int64 value)
   at Microsoft.NET.Build.Tasks.ResolvePackageAssets.CacheReader.ReadMetadataStringTable()
   at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ReadItemGroups()
   at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ExecuteCore()
   at Microsoft.NET.Build.Tasks.TaskBase.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
[warn]: OmniSharp.MSBuild.ProjectManager
        Failed to load project file 'c:\Users\MarkLocal\Documents\github\PowerShell\src\System.Management.Automation\System.Management.Automation.csproj'.

[iSazonov]

@markekraus I had to remove all previous .Net Core SDKs before I could able to compile.

[markekraus]

@iSazonov I can build fine. but I can't get VSCode and Omnisharp to work. I have deleted all the previous SDKs.

[markekraus]

Hmm.. moving the repo from a deeper path to c:gitPowerShell seems ro have fixed the problem. I saw some unlreated issue in dotnet/sdk about nupkg path lengths.. and decided to try a shorter repo path. *shrugs

[iSazonov]

:smile: DOS is still alive!

[markekraus]

So now that the official 2.1 RC1 is out, do we need to make updates? https://blogs.msdn.microsoft.com/dotnet/2018/05/07/announcing-net-core-2-1-rc-1/

[SteveL-MSFT]

We should move to RC1-Final before Preview.3 ships. Should just involve updating global.json as we've made changes in tests/product necessary to move to 2.1

[daxian-dbw]

@markekraus @SteveL-MSFT
The 2.1-rc1 SDK downloaded from the link given in the blog shows version 2.1.300-rc1-008673 with 2.1.0-rc1 Host and 2.1.0-rc1-final .NET Core rutnimes (see below). However, running Install-Dotnet -Channel release -Version 2.1.300-rc1-008673 with the same verison installs the same version of SDK but different Host/rutnimes versions.

I'm not sure what went wrong, but it seems to me on the dotnet core side ...

dotnet SDK download form the blog

C:\>C:\Users\dongbow\Downloads\dotnet\dotnet.exe --info
.NET Core SDK (reflecting any global.json):
 Version:   2.1.300-rc1-008673
 Commit:    f5e3ddbe73

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Users\dongbow\Downloads\dotnet\sdk\2.1.300-rc1-008673\

Host (useful for support):
  Version: 2.1.0-rc1
  Commit:  eb9bc92051

.NET Core SDKs installed:
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.300-rc1-008673 [C:\Users\dongbow\Downloads\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0-rc1-final [C:\Users\dongbow\Downloads\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0-rc1-final [C:\Users\dongbow\Downloads\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0-rc1 [C:\Users\dongbow\Downloads\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

dotnet SDK downloaded with Install-Dotnet -Channel release -Version 2.1.300-rc1-008673

PS:8> C:\Users\dongbow\AppData\Local\Microsoft\dotnet\dotnet.exe --info
.NET Core SDK (reflecting any global.json):
 Version:   2.1.300-rc1-008673
 Commit:    f5e3ddbe73

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.16299
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Users\dongbow\AppData\Local\Microsoft\dotnet\sdk\2.1.300-rc1-008673\

Host (useful for support):
  Version: 2.1.0-rc1-26428-03
  Commit:  2c36b039fd

.NET Core SDKs installed:
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.300-rc1-008673 [C:\Users\dongbow\AppData\Local\Microsoft\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0-rc1-30682 [C:\Users\dongbow\AppData\Local\Microsoft\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0-rc1-30682 [C:\Users\dongbow\AppData\Local\Microsoft\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0-rc1-26428-03 [C:\Users\dongbow\AppData\Local\Microsoft\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[markekraus]

@daxian-dbw weird. I get the 2.1.0-rc1 [sic] host when I do Install-Dotnet -Channel release -Version 2.1.300-rc1-008673

[SteveL-MSFT]

I see the same as @daxian-dbw on macOS. I'll reach out to the dotnet team on this.

[iSazonov]

I see rtm builds on MyGet.
https://dotnet.myget.org/feed/dotnet-core/package/nuget/Microsoft.NET.Sdk/2.1.300-rtm-62909-02

[SteveL-MSFT]

Dotnet team must be busy with Build as they aren't responding. Perhaps for now we should just wait for RTM if they are starting producing those builds.

[iSazonov]

While we could release 6.0.3 with latest .Net Core security updates. :-)

[markekraus]

what ever happened to the 6.0.0 branch? I don't see it anymore. was it made private or moved somewhere?

[SteveL-MSFT]

@markekraus 6.0.x servicing updates are done on a private branch since they contain security updates and we don't want to disclose the changes prematurely before the bits being publicly available

[SteveL-MSFT]

Got a response now that Build is over. They had an issue with CDN not having correct version, but that has since been fixed. If you still see wrong info with dotnet --info, try cleaning out ~/.dotnet folder.