Powershell: Build powershell core in Visual Studio 2017

[daxian-dbw]

With #3398, we are now building powershell core via MSBuild with our building script. However, it seems not working properly in Visual Studio 2017. We need to make it possible to build powershell in VS2017 as well.

[powercode]

Should we think about what is a Configuration and what is a Platform?

Linux is a config today, but it seems more reasonable to treat it as a platform.

I have a branch, https://github.com/powercode/PowerShell/tree/vs2017-build, where I can build with vs2017, but it is untested on Linux/Mac and from the command line.

[daxian-dbw]

The term "Platform" sounds like the right one to use, but it seems always indicating the CPU architecture based on my limited MSBuild experience, for example, Any CPU, x64, and x86.

And we have another problem -- with the current project files, dotnet pack will generate a NuGet package that only contains assemblies built for windows, which cannot be used reliably on Linux. I wonder if it's possible that we can change the .csproj somehow to make dotnet pack produce a single NuGet package that contains both win and unix assemblies. #3417 is tracking it.

Is changing to "Platform" a possible direction? Do you have some resources on the differences between "Configuration" and "Project" in the new MSBuild from .NET Core SDK?

[powercode]

Platform may be what we choose. LinuxX64 for example.
They are just monikers to give builders an easy way to set different settings based on Config and platform.
C++ has had Win32 and x64 while .NET has used AnyCPU, x86 and x64. But I think this traditionally has been Windows specific, so instead of saying Win-x86, they just went with the shorter alternative.
I haven't had much time to dig into the changes with the new msbuild. Seems nice and clean though.

[iSazonov]

I wonder how .Net Core resolve the same for itself packages? Can we use the same?

[iSazonov]

@powercode Could you make PR with your vs2017 sln file?

[oising]

Please, please, this. I am loathe to return to building software like it's 1993. I already had that life. I just wasted three hours trying to make anything work at all in VS 2017 15.3-pre (supposedly it understands net standard 2.x) and it's just a confusing mess.

[iSazonov]

I've already started taking little steps in this direction, but @daxian-dbw too busy these days to do the PR review.

[BobbyCannon-Quantros]

It's been a few months... any progress on this?

[oising]

Ping pong poong.

I (and many others I'm sure) want it to build in VS 2017 so we can use ReSharper and other tools.

[BobbyCannon-Quantros]

Plus + 1000 @oising! You know... use Visual Studio. :)

[SteveL-MSFT]

This isn't something we're prioritizing for 6.0.0, but we can add it to the 6.1.0 queue although that itself is growing quite large...

[powercode]

I just noticed that if I add a property

<TargetFramework>$(TargetFramework)</TargetFramework>

then visual studio understands the project formats.

Sorry that I haven't been pushing this. So silly, I make it work locally, but never get around to pushing it.

[iSazonov]

@powercode If you haven't free time - put a link on the file so everybody can grab, test and push PR.

[powercode]

That seems to be needed in the csproj files. Seems like a bug in visual studio, since the property is there, but via an import.
Adding the strange no-op property seems to work around it, but maybe we should push for a fix in VS too.

I also ran into build issues from the command line where the latest VS version comes with tool version 2.0.2, and we require 2.0.0. Maybe we should be a bit more relaxed in the version checks?

[iSazonov]

Perhaps #5118 can help with version checks.

[sharpninja]

Just pulled the current master yesterday and opened in VS 2019. All goes well except that the RESX files are not detected at all by either VS or dotnet doing a build from VS.

[TobiasPSP]

I did the same, and it did not work at all in VS2019. The project folders all show in project explorer but none is loaded, and there is no reason given why.

Sooooo...what are the prerequisites for loading and compiling powershell.sln in VS2019? What are settings that need to be in place in VS2019? That's probably a 10min effort to put together. The current documentation is super old and gives no details whatsoever.

[KirkMunro]

I never build PowerShell in Visual Studio. We have automation for that. ;)

I do the following whenever I want to work in PowerShell code:

1. Open PowerShell.exe (or pwsh if you like -- I've just always done this part from PowerShell.exe).
2. Browse into the root of your local PowerShell repository git clone.
3. git remote add upstream https://github.com/PowerShell/PowerShell (if you haven't already done this).
4. If I'm in a clean state with no local uncommitted changes:
   
   
   
   1. git checkout master (switch to the master branch)
   
   
   2. git pull upstream master (get the latest PowerShell repo changes)
   
   
   3. git push origin master (push those changes into my fork)
   
   
5. If I'm about to work on a new changeset:
   
   
   
   1. git checkout -B my-branch-name (where my-branch-name is the name of the branch I am about to start working on, using an appropriate name -- e.g. splat-members for a branch where I add support for splatting members of object instances)
   
   
6. If I'm about to continue working on an existing changeset:
   
   
   
   1. git checkout my-branch-name (switch to my changeset branch)
   
   
   2. git merge master (catch my branch up to master)
   
   
7. ipmo ./build.psm1
8. Start-PSBootstrap (I only do this the first time ever on a system, and then periodically if I need to update .NET 3.x or something -- a failed build will give you a hint when this is necessary).
9. Start-PSBuild (with -Clean if I want a clean rebuild, with -ResGen if I want to regenerate resource string DLLs -- when and how I use these depends on which branch I'm switching from/to).

Once I've got a build complete, I'll invoke it using pwsh-debug, which is an alias that gets defined from this script in my profile that needs to be periodically updated when the .NET version changes:

#region Define pwsh-debug, and pwsh-release aliases.

$pwshClone = "${env:USERPROFILE}/source/repos/PowerShell"
if (Test-Path -LiteralPath $pwshClone) {
    foreach ($buildName in @('debug','release')) {
        $buildExe = "${pwshClone}/src/powershell-win-core/bin/${buildName}/netcoreapp3.1/win7-x64/publish/pwsh.exe"
        if (Test-Path -LiteralPath $buildExe) {
            if ($Host.Name -eq 'Windows PowerShell ISE Host') {
                $executionContext.SessionState.PSVariable.set("pwsh-${buildName}", $buildExe)
                Set-Content "function:pwsh-${buildName}" @"
Start-Process '$buildExe'
"@
            } else {
                New-Alias -Name pwsh-${buildName} -Value "$buildExe"
            }
        }
    }
}

#endregion

You need to prime the pump for that to be in your profile, because if you've never built PowerShell before, you won't have the alias created, so you would need to manually dot source your profile after build to get the alias. You could alternatively use Start-DevPowerShell (that's part of the build module), but I never use that myself, preferring to work from a single console window.

At that point, I'll open PowerShell.sln in Visual Studio if I don't have it open already (if you use Start-PSBuild -Clean, you'll need to close VS first to release some file locks), invoke $PID in my PowerShell window to get the process ID, and then use Debug | Attach to Process in Visual Studio to attach to that process and start debugging.

Some folks will prefer just working in Visual Studio, but given how much PowerShell is about automation/command line, I find this process suits my needs perfectly.

[TobiasPSP]

Thanks so much for your details. Very helpful and constructive information! Will try over the WE.

I really would prefer to work on PowerShell code just like I work on any other software. Building is no big deal in VS (a matter of a click), so I wouldn't really consider this a need for automation, rather something that in VS is typically absolutely transparent to me, and with the powershell sources suddenly turns into a series of crucial steps to consider ;-)

My problem starts that when I open the sln in VS2019 OOB, it won't load a thing, and I have no clue why. I can't open any source.

I'll dig through your checklist and am very grateful for it, probably things will resolve with your help.

Ultimately, though, I would expect powershell.sln to open and run without any tricks and workarounds in MSFT's premier dev environment VS.

I might reconsider once I get the tool chain working that you describe. Meanwhile, I believe the more 'extra considerations' and 'rerequisite settings' are required to simply compile the project, the worse it is. It should be a friendly and easy first encounter, and "just work" for anyone used to program and compile software in VS.

[vexx32]

@TobiasPSP some things required for building pwsh are added by the build process itself, but I'm not sure why VS would be completely unable to load any of the files.

I think some of the folx on the PowerShell team still work in Visual Studio, so perhaps they can assist better here. I tend to work in VS Code most of the time, so I haven't seen the issues you're running into. :(

[TobiasPSP]

Many thanks. VS2019 can load the project and shows the subfolders, but I cannot see any files, and VS2019 says the projects could not be loaded. It is not saying why. Yes it would be awesome if someone who is actually using VS2019 to work on powershell could chime in, and possibly update the docs.

I imagine once I wrap my head around the build process as you and most of the team got used to, I might find it equally appealing.

Most professional devs (coming from the Windows side) use VS and their personal set of dev tools, so to welcome them, IMHO it is important to either provide a seamless dev experience using VS2019 OOB, or provide sufficient up-to-date information for new devs to quickly find their way into the build process.

The more people work with this project, the better for all. I do know of quite a few Windows devs who really struggled with the way how powershell is built/how it cannot be built from VS. Do you know of someone specifically that is using VS to build powershell.sln that I could approach?

I am more than glad to help updating the documentation while I work my way into building with VS and would gladly serve as test candidate.

[iSazonov]

@powercode has an experience with VS.

[TobiasPSP]

Awesome, thx! I just contacted Staffan. Will share what I learned from him so we can update the docs.

[iSazonov]

You could look how other projects like CoreFX build. Also previously MSFT team did not want to move from Build.psm1 to csproj-s and I stopped the work.

[AndrewSav]

VS2019 can load the project and shows the subfolders, but I cannot see any files, and VS2019 says the projects could not be loaded

This is confusing. It can load the project but the project cannot be loaded? What are you trying to say?

I imagine once I wrap my head around the build process as you and most of the team got used to, I might find it equally appealing.

Well, I'm not on the team, but "the process" as described is literally a few powershell commands. I personally had not trouble following them. I do not find the process "appealing" I found it normal, documented and working.

Most professional devs (coming from the Windows side) use VS and their personal set of dev tools, so to welcome them, IMHO it is important to either provide a seamless dev experience using VS2019 OOB, or provide sufficient up-to-date information for new devs to quickly find their way into the build process.

I'm not sure about that. If a dev needs to be baby-sitted into making their dev enviroment comfortable for them, I would question how valueable their contribution might be. Don't get me wrong, I'm not against a proper VS2019 build process that is documented. It's just if the dev team does not _need_ it, it is quite understandable why this is not a priority, this is only natural.

For what it's worth: once you've build the solution with the documented Build.psm1 way, you can open the solution in Visual Studion 2019 and it loads without any problems at all, if you have the correct SDK installed. If you do not, it tells you so. After that the solution does build in VS. It does not do _everything_ that Build.psm1 does but it builds. You can set a break point and debug. You can make a code change and iterate. I found it quite straightforward.

Thank you for your desire to document what you've leared from Staffan, and good luck with that.

[TobiasPSP]

Not babysitted but just a short info whether it is possible or not, and if so, how. Babysitting has a negative notation. We all work on this volunteering our spare time, so whatever mainstream dev env someone is used to should be respected and covered properly in the docs.

Thanks for your feedback, will try the build script as a prerequisite. If that was it, then adding this one instruction to the VS section in the docs would make everyone happy.

[TobiasPSP]

This is confusing. It can load the project but the project cannot be loaded? What are you trying to say?

Sorry for any confusion: the project folders show in VS project explorer, but no content (no files) are shown. For each folder, VS claims that „the project couldn‘t be loaded“ but provides no reason.

[AndrewSav]

Babysitting has a negative notation.

Yes, sorry about that, I should have avoided this turn of speech.

[bergmeister]

Up until a few months ago, VS worked quite well, it only required to build via the command line once (import-module ./build.psm1; start-psbuild) and from then on one could use VS to make changes and get intellisense and compilation/syntax errors, which made it a useful enough text editor comparable to vs code.
A few months ago one of the csproj files stopped loading and I thought it was due to .net core 3 preview and VS being a bit behind. Still not a big issue unless you have to modify code in this area, but we should fix the project loading error. Apart from that I am happy with the level of VS support, even if it is not first class, we could do a better job at documentating what is supported in VS and what is not and known issues like the local changes in the sln file due to the way the projects are setup...
I will try to see what the current behaviour with the VS preview (because we are on the .net core 3.1 preview now) and if I can fix it, it might even be a problem in VS or .net core where a certain element is not supported.

[iSazonov]

@bergmeister Perhaps you need to remove all other Core versions from your system and install only version coming with VS last update.

[SteveL-MSFT]

https://github.com/PowerShell/PowerShell/pull/11085 should allow VS to build pwsh although you still need to exercise the psm1 to run resgen and typegen the first time (or until types or resx change).

[bergmeister]

In .Net Core 3 they re-implemented some of the resgen functionality but usage within the csproj from command line and VS is not working well for non-WinForms projects, I opened an issue with them but once that is done, we could potentially use the functionality baked into dotnet build soon

[xtqqczze]

Related: #12373

[iSazonov]

We already moved ResGen to csproj.
Only TypeGen is still out of csproj that is required for VS works well. We could do the move as only MSFT team approve this.

[kyanha]

Can TypeGen be invoked as a separate tool from the VS2019 prebuild commands configuration?

[iSazonov]

@kyanha You could run Start-PSBiuld -TypeGen there. (pwsh -command ....)