Powershell: Publish PowerShell NuGet packages for Linux/OSX platforms

[daxian-dbw]

PowerShell Core builds separately for Windows and Unix, so the assemblies are different between Windows and Unix platforms. All PowerShell Core packages we published so far contain windows assemblies only.

The ideal solution is to have one package that contains assemblies for both Windows and Unix platforms. But dotnet pack doesn't seem to support this.

This is blocking hosting powershell in .NET Core applications on Linux/OSX.
The discussion of this issue originates from #3409

[mirichmo]

We need to publish the Windows version of the packages to NuGet as well.

Note: This issue seems very similar to #1152

[lzybkr]

I think we want a single set of reference assemblies for portability. If we have platform specific APIs, then hopefully there is a way to refactor those into another assembly if need be.

[daxian-dbw]

Take System.Management.Automation as an example, I think we will eventually end up with 2 set of NuGet packages:

1. A NuGet package that only contains the reference assembly. This will be the __PowerShell Standard__ reference assembly, which exposes an API surface that is supported by both Windows PowerShell and PowerShell Core. But the API surface will a subset of the actual publish APIs exposed in Windows PowerShell or PowerShell Core.
   
   
   
   • This reference assembly is an abstraction that doesn't have any runtime implementation that exactly maps to it, so there won't be any runtime DLLs in this NuGet package.
   
   
   • This NuGet package is the friend of powershell binary module authors. Binary modules targeting this NuGet package and netstandard2.0 will theoretically work on both Windows PowerShell and PowerShell Core.
   
   

1. A NuGet package that is specific for PowerShell Core. It contains the reference assembly that exposes all public APIs from the latest System.Management.Automation.dll in PowerShell Core, as well as the runtime assemblies for both Win and Unix platforms, including the native dynamic library libpsl.so that powershell depends on in Unix plats.
   
   
   
   • This NuGet package is supposed to be used by any netcoreapp2.0 application that needs to host PowerShell Core.
   
   

[iSazonov]

@daxian-dbw Can we move the Issue to #3961 ?

[daxian-dbw]

@iSazonov #3961 is tracking the task to create new release packages of powershell core for Linux. .NET Core is moving to support generic Linux RID for future Linux distros -- for example, there will be no RID for Ubuntu 17 and you need to use Linux-64 as the RID. This issue is tracking the NuGet packages which are different, so I think we should keep this one separate.

[Qowy]

Is there a way to manually circumvent this for the moment. For example manually copying files to the publish directory or something similar?

[windischb]

So for now, it's not possible to host Powershell in a Process on Linux?
Or is there a workaround?
Copy files manually, install other packages before, or something?

[iSazonov]

@Qowy @windischb Sorry, it seems all experts is busy.

@adityapatwardhan Could you help please? Is there a workaround?

[ianByrne]

@iSazonov @adityapatwardhan @daxian-dbw - any updates on this yet? Or is there a workaround? Thanks.

[iSazonov]

The Issue milestone is GA so I think we get solution soon.
Workaround from @daxian-dbw :

There is a workaround. Build your application using the existing powershell core NuGet packages, but replace the powershell assemblies with those from a Linux/OSX package. That hopefully would work.

[windischb]

Should this work now with v6.0.0-rc.2?

Edit:
I have tried the sample from the docs.
https://github.com/PowerShell/PowerShell/tree/master/docs/host-powershell
Doesn't work on Linux for me...

[daxian-dbw]

@windischb this issue will be fixed in GA build.

[ianByrne]

@daxian-dbw thanks for the update; do you have an ETA of when GA is happening?

@iSazonov also thanks for the workaround, although I'm not too sure where to get "powershell assemblies from a Linux package". Is there a stash of DLLs somewhere that I can copy across, or will I need to somehow build them myself?

[iSazonov]

@ianByrne You can download and unzip Linux package. Although it is no longer meaningful - GA will be released in three weeks!

[daxian-dbw]

@ianByrne You can click on the 6.0.0-GA milestone on the right side to see the due date, which is 1/10/2018.

[windischb]

Just tried it with the Release Version 6.0.0 and still get following error:

[ianByrne]

Looks like they're still working on that bit for 6.0.0 - this is the only remaining open issue of the milestone, so hopefully will be finished up soon.

[windischb]

Ah ok, thanks!

[ianByrne]

@daxian-dbw is there any update on the status of this issue? Is it no longer to be part of the 6.0.0 release?

[lmcdougald]

^Assumed it was complete based on the release notes and couldn't understand what I was doing wrong. Glad it wasn't a configuration error.

If I follow the steps for Windows, can I assume that at some release relatively soon (next year?) Linux will get support?

[iSazonov]

Guys, be patient some days - our team still works on publishing GA packages.

[mi-hol]

@iSazonov on MacOS brew tries to install version 6.0.0_rc2 over an working but manually installed 6.0.0, might this be related to this open issue?

[iSazonov]

@mi-hol No, You should new Issue.

[ianByrne]

Guys, be patient some days - our team still works on publishing GA packages.

@iSazonov sorry for the constant pestering, but are there any updates on this?

[iSazonov]

@ianByrne I haven't news 😕

[adityapatwardhan]

@ianByrne Thanks for your patience, I am actively working on it right now. Hopefully will be done in just a few days.

[MaximoTrinidad]

I have seen any type of tags in NuGet packages to help identify Linux (or .NET Core) ones.
NuGet need to implement a way to stop installation on Linux systems.

At least, in PowerShell Gallery you can use the -Tag parameter to identify Linux modules.

:)

[dinesarun]

I am eagerly waiting for this as my modules forcefully depends on PowerShell

[michaelrajece]

As per @daxian-dbw workaround, I have replaced the powershell assemblies with those from a Linux/OSX package. But, I am getting the following error when starting the application.
Unhandled Exception: System.TypeInitializationException: The type initializer for 'System.Management.Automation.Runspaces.RunspaceFactory' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'Microsoft.PowerShell.CoreCLR.Eventing, Version=6.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

When checking the powershell assemblies of Linux/OSX package, all are having file version 1.0.0.0. Please help me on this issue?

[daxian-dbw]

@michaelrajece Maybe it's because you are still building your application targeting a Windows System.Management.Automation ref assembly. The Eventing dll is only available on Windows platform.

@adityapatwardhan is about to finish this nuget packaging work. Hopefully, we will be able to use it soon.

[michaelrajece]

@daxian-dbw Thanks for your update. When Replacing the System.Management.Automation dll with those from a Linux/OSX package is also failed with the same assembly's manifest definition mismatch error.
Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'System.Management.Automation, Version=6.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

So, How can I overcome this issue?

[ianByrne]

I am actively working on it right now. Hopefully will be done in just a few days.

@adityapatwardhan are there any updates on when this may be available?

[adityapatwardhan]

@ianByrne Sorry for the late response. I am currently testing the nuget packages and we are targeting a release by the end of next week (2/23).

[adityapatwardhan]

@ianByrne The PR for the required script changes is out #6167. Once this is merged, I will release the nuget packages.

[ianByrne]

Thanks @adityapatwardhan and @daxian-dbw for implementing this fix. I have tried my application with these new versions, however am still having some issues - are they related to this? Or am I doing something completely wrong.

Error:

This parameter set requires WSMan, and no supported WSMan client library was found. WSMan is either not installed or unavailable for this system.: Unable to load DLL 'libpsrpclient': The specified module or one of its dependencies could not be found

Code:
```c#
WSManConnectionInfo connectionInfo = new WSManConnectionInfo(new Uri("https://outlook.office365.com/powershell-liveid"), "http://schemas.microsoft.com/powershell/Microsoft.Exchange", new PSCredential(username, password));
connectionInfo.AuthenticationMechanism = AuthenticationMechanism.Basic;
connectionInfo.MaximumConnectionRedirectionCount = 2;

using (PowerShell ps = PowerShell.Create())
{
ps.Runspace = RunspaceFactory.CreateRunspace(connectionInfo); // Exception gets thrown here
}

csproj:
```xml
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeIdentifiers>linux-x64</RuntimeIdentifiers>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.PowerShell.SDK" Version="6.0.1.1" /> 
    <PackageReference Include="Microsoft.PowerShell.Commands.Diagnostics" Version="6.0.1.1" />
    <PackageReference Include="Microsoft.WSMan.Management" Version="6.0.1.1" />
  </ItemGroup>

</Project>

[adityapatwardhan]

@ianByrne Thanks for the feedback. I am having a look.

[adityapatwardhan]

/cc @dantraMSFT

@ianByrne The libpsrpclient.so and libmi.so seem to be placed under the runtimes/linux-x64/native folder during the dotnet publish step.

A workaround for the issue is to use:
dotnet publish --runtime linux-x64

If a specific runtime is chosen at build time, then the issue with runtimes folder is mitigated.

[ianByrne]

@adityapatwardhan I ran dotnet publish -c Release --runtime linux-x64 but these two files do not seem to get created. Am I missing a reference or some other dependency?

This is the contents of my /bin/Release/netcoreapp2.0/linux-x64/publish dir:

libpsl-native.so
linuxtest.deps.json
linuxtest.dll
linuxtest.pdb
Microsoft.ApplicationInsights.dll
Microsoft.CodeAnalysis.CSharp.dll
Microsoft.CodeAnalysis.dll
Microsoft.Management.Infrastructure.dll
Microsoft.Management.Infrastructure.Native.dll
Microsoft.PowerShell.Commands.Management.dll
Microsoft.PowerShell.Commands.Utility.dll
Microsoft.PowerShell.ConsoleHost.dll
Microsoft.PowerShell.CoreCLR.Eventing.dll
Microsoft.PowerShell.SDK.dll
Microsoft.PowerShell.Security.dll
Microsoft.Win32.Registry.AccessControl.dll
Newtonsoft.Json.dll
System.Data.SqlClient.dll
System.IO.Packaging.dll
System.Management.Automation.dll
System.Net.Http.WinHttpHandler.dll
System.Private.ServiceModel.dll
System.Security.AccessControl.dll
System.Security.Cryptography.Pkcs.dll
System.Security.Permissions.dll
System.Security.Principal.Windows.dll
System.ServiceModel.dll
System.ServiceModel.Duplex.dll
System.ServiceModel.Http.dll
System.ServiceModel.NetTcp.dll
System.ServiceModel.Primitives.dll
System.ServiceModel.Security.dll
System.ServiceProcess.ServiceController.dll
System.Text.Encoding.CodePages.dll
System.Text.Encodings.Web.dll
System.Threading.AccessControl.dll

[adityapatwardhan]

@ianByrne Sorry I forgot to mention. Add the package psrp to your csproj. https://powershell.myget.org/feed/powershell-core/package/nuget/psrp

[ianByrne]

Thanks @adityapatwardhan - I am now seeing those files. However, the application continues to throw at the same point with the same error, that the DLL is missing.

[adityapatwardhan]

@ianByrne Found the root cause. Filed issue https://github.com/PowerShell/PowerShell/issues/6281