Home: VS 2017 doesn't execute install.ps1 from packages if used as PackageReference

[monkey-dsc]

Hi,

maybe a duplicate Topic, I don't know... But none of the topics here solved my Issue.

I need to install a Nuget-Package, Glass.Mapper.Sc -Version 4.4.0.199, the Package has a Dependency to Glass.Mapper.Sc.Core -Version 4.4.0.199 which evaluates with install.ps1 which Sitecore Version is present in the project for backward compatibility. If a Sitecore.Kernel package is present, e.g. Sitecore.Kernel.NoReferences -Version 8.2.170407, it will install Glass.Mapper.Sc.dll, if also Sitecore MVC Version is present in the project, eg. Sitecore.Mvc.NoReferences -Version 8.2.170407, it will install Glass.Mapper.Sc.Mvc.dll (also described in the readme.txt in the package...).

Sitecore does have its own NuGet feed so you have to add _https://sitecore.myget.org/F/sc-packages/api/v3/index.json_ to your nuget package sources.

NuGet Product used: Package Manager in VS 2017

NuGet Version is 3.5+

Visual Studio 2017

OS Version: Win10

Worked before in VS 2015

Repro:

1. Create a new class library in VS 2017 (.Net Framework)
2. Format project to the new csproj format as described here (Startover: Class Library)
3. Install Sitecore.Kernel.NoReferences -Version 8.2.170407 NuGet package
4. Install Sitecore.Mvc.NoReferences -Version 8.2.170407 NuGet package
5. Install Glass.Mapper.Sc-Version 4.4.0.199 NuGet package

If everything works as expected, Glass.Mapper.Sc.dll and Glass.Mapper.Sc.Mvc.dll should be added to the Project. But it doesn't.

The evaluation of the Sitecore version works fine in 2015, since 2017 it doesn't work anymore :(
We are very dependend on this package, because we want to build lots of nuget packages which will have a dependency on this...

Am I doing something wrong? I could also provide a Solution with a single Interface where its not working to install the package properly.

Regards
Dirk

[mishra14]

@monkey-dsc The help page for Glass.Mapper.Sc is http://www.glass.lu/Mapper/Sc/HowTo/1-Install

It seems that you also need to add a reference to System.Web.Mvs.dll. Can you please try adding reference to that before installing Glass.Mapper.Sc?

If you are still facing the same issue, please let us know.

[monkey-dsc]

Hi @mishra14 ,

I have setup a little sample project where you can see this behaviour. A reference to System.Web.Mvc is also present and you can just start adding Glass.Mapper.Sc to the project.

Regards
Dirk
Company.Foundation.PresentationSettings.Core.zip

[jainaashish]

Format project to the new csproj format as described here (Startover: Class Library)

install.ps1 doesn't execute with PackageReference format. Blogpost about PackageReference https://blog.nuget.org/20170316/NuGet-now-fully-integrated-into-MSBuild.html also talks about this.

There is already a tracking bug https://github.com/NuGet/Home/issues/5963 for this issue. And our preliminary thought is to have msbuild targets instead of install.ps1 scripts to achieve the same behavior. So once we've all the guidelines then we'll ask package authors to update their packages to remove install or uninstall ps1 scripts and add targets instead.

[monkey-dsc]

Wow hard stuff... @jainaashish that's a really really everything breaking change... 👎

[jainaashish]

It has always been like this with transitive world, be it project.json or PackageReference. It was never a good idea to have these install.ps1 and uninstall.ps1 scripts at the first place which are even more troublesome to maintain since users do all sort of weird stuff through these scripts and they provide a loophole in the system. Besides, since most of these scripts have hard dependency on dte project, so they only works inside visual studio, and couldn't be supported via command line.

So we don't want to carry the same design issues of packages.config to PackageReference and which is what we're trying to fix here. And MSBuild seems to the right approach for all these kind of stuff. We understand it's not going to be an easy process but we're gradually working on it and making progress. We also plan to work with package authors to make this transition as smooth as possible.

[RussKie]

So once we've all the guidelines then we'll ask package authors to update their packages to remove install or uninstall ps1 scripts and add targets instead.

It has been 8 month since you were going to create guidelines, still nothing...

We are migrating a large SDK-style solution which is heavily dependent on install.ps1. Because netstandards and .netcore projects use exclusively PackageReference we are essentially blocked...

[augustoproiete]

>

Because netstandards and .netcore projects use exclusively PackageReference we are essentially blocked...

Same here. Really looking forward to having a supported alternative to install.ps1 guys!

[jainaashish]

@RussKie @caioproiete can you guys please share specific details about the requirement of install.ps1 with your packages? If you guys can't share details in public then drop me an email at asja @ Microsoft .com with all the details or possibly a sample solution to depict what were you trying to achieve with these scripts.

[augustoproiete]

Hey @jainaashish here are some examples:

Configuration templates

• I need to add files to the user's project when installing a nuget package. These files are "template files" containing configuration settings (i.e. a starter template), which the user is expected to make changes to it / adapt. PackageReference dropped support for content and introduced contentFiles that are immutable which I cannot use because I need mutable files.

• I need to dynamically rename a file being added to the project. The nuget package has "template.ext" and as the package is installed, I need to rename it to something like `$(ProjectName)-v1.ext". You can see an example here.

• I need to perform XDT transformations on configuration files. This was supported in the packages.config model, but it's not supported with PackageReference

Code templates

• I need to add CSharp files to the user's project with code transformation (.pp) when installing a nuget package. Again, the user is expected to change the file, so contentFiles is not an option. See LibLog as an example.

Assembly settings

• Some assemblies that are added as reference to the project need to be marked as CopyLocal=false. I can do with PowerShell on the packages.config model. You can see an example here.

Cleanup on uninstall

• Need to do some clean-up when the user uninstalls a nuget package. That can include removing and/or renaming files from the project that we're uninstalling the package from. You can see an example here.

[augustoproiete]

@jainaashish One more scenario I've just remembered:

Changing project settings via DTS upon installation of NuGet package

• I'm providing to users a framework that allows them to develop Add-Ins for Excel. That means that when they debug the add-in, Visual Studio must run EXCEL.exe and attach to it, so that I can debug.

This means that I need to update properties in the .csproj.user file such as StartAction, StartProgram, and StartArguments which are settings for that particular user (given that different people use different versions of Excel, etc.).

e.g.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/ ...">
  <PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|AnyCPU'">
    <StartAction>Program</StartAction>
    <StartProgram>C:\Program Files\Microsoft Office\Office15\EXCEL.EXE</StartProgram>
    <StartArguments>"MyExcelAddIn-AddIn.xll"</StartArguments>
  </PropertyGroup>
</Project>

I do that via PowerShell through install.ps1

This is not something that can be added to a .targets files, first because it's a per-user setting, and not global to the team sharing the solution, and second because Visual Studio requires that these properties must be inside the .csproj or csproj.user... Defining them in a .targets file that gets imported has no effect.

[RussKie]

I am part of a team developing a Enterprise RAD framework (in many ways it is similar to .NET SDK).
Our SDK consists of numerous packages, and whenever a developer installs or updates a package in their project, the package must make necessary changes to project's files, such as:

• apply XDT or XSLT transformations to a web.config files (including Debug and Release), and/or
• create or modify configurations in web.config files (including Debug and Release), and/or
• create a project's manifest file (a vital xml file in our case) or apply XSLT to it, and/or
• apply XSLT to project's manifest file, and/or
• run code transformations (.pp), and/or
• uninstall deprecated Nuget packages, etc

All of these are install-time operations. I've looked at .targets, but they seem to be run-time operations, and thus can't meet our requirements.

[david-m-forster]

To add to @RussKie 's comments (I work with him),

We originally used the nuget built-in xdt capability to transform web.*.config as well as custom xml files when a package was added. A given consumer may install one or many nuget packages depending on their business requirements. i.e different nuget packages enabled different related capabilities. So each package may add/remove XML sections or update existing sections.

The problem with xdt is that it would end up completely corrupting the targeted resource (like web.config say) as each nuget package would end up interfering with the changes made by other packages. This lead to 100s of our engineers being confused and led to a support nightmare attempting to help them get their config sorted out.

We then innovated by using the install.ps1 script to kick off an XSLT transformation. XSLT, although somewhat out-of-vogue these days is incredibly powerful so we could selectively transform, add or remove items without interfering with other nuget installs (examples like deprecating old app settings or even renaming them). Further, the XSLT files could be unit tested to ensure they were robust under many different installation scenarios. we even have integration tests that apply multiple package transformations in one go.

So now we want to move everything over to .netcore but if we do we will be seriously regressing when it comes to transformations of config files.

[BrunoJuchli]

@jainaashish
Another example is https://github.com/Fody/Fody
It's a weaver with several plugins like https://github.com/Fody/PropertyChanged
Generally, the sequence of the weaver-plugins can matter, plus some have configuration options.
The sequence and the configuration were done in an XML, FodyWeavers.xml
For convenience, when you installed a weaver-plugin, it updated the FodyWeavers.xml and added an entry for itself.
When relevant, the user still had to edit the FodyWeavers.xml manually to fix weaver-plugin sequence and -configuration. However, it was more convenient as he didn't have to look everything up in the Wiki.

Fody has been migrated to PackageReference, and now the user has to create the xml manually and also add entries to it manually. It's possible that he'll forget to create the entry upon installation (how would he know he has to create them, in the first place?).

This is not a huge deal, and compared to other broken use cases something of little importance.
It might help with showing a pattern (or patterns), though.

[jainaashish]

Thanks everyone, keep pouring these examples. These will certainly help us figure out right strategy going forward.

We'll analyze all these examples and then decides the next course of action. For time being, I'm reopening this issue for further tracking.

@anangaur

[StingyJack]

In addition to the points above, the install.ps1 is also useful for setting specific BuildAction and CopyToOutput for files that are being added to the project.

[dfoulk]

The removal of this functionality impacted us as well. We need to be able to transform existing files and add new files to a user's project that they can modify after we're done. Not being able to do so rendered something like 50% of our internal packages useless. We committed to .NET Standard assuming it would be an upgrade to our workflow. I hope that we can see a return for this functionality as it really limits our ability to provide users with editable plugins.

[StingyJack]

users do all sort of weird stuff through these scripts and they provide a loophole in the system

Users do all sort of things that are useful and/or necessary for them because nuget or the project system provides no other way to accomplish these things. FTFY

[Kilowhisky]

From a simple compatibility sake this functionality should be restored. About 95% of the packages in every single one of my solutions is broken because of this. JQuery, JQueryUI, Knockout, WebApi.HelpPage, WebApi.HelpPage.Ex, NLog, UAParser are only a few. Basically anything that manipulates or drops files in a specific project directory is now broken. This absolutely ruins ASP.NET projects.

[nbarnwell]

This is pretty poor show. I take the arguments why arbitrary scripts seem bad, but you shouldn't take them away unless there's another option available. Backwards-compatibility only needs to last long enough for people to have a reasonable amount of time to move to the new way of doing things, but you do need to give us a chance rather than just pulling the rug out from under us with no alternatives.

[nbarnwell]

@jainaashish Might you have a few minutes to update this issue? We're struggling here because of this missing functionality. I understand the nature of open source, but it seems disingenuous to call people to your platform, let them get locked in to it, then break it so they're all screwed. I don't want the answer to be "Build your own nuget, then"; I liked yours.

[jainaashish]

@nbarnwell I'm afraid I don't have any update for this issue at this time. We have been working towards improving PackageReference eco system and bridge compatibility gaps between packages.config vs PackageReference while working on enabling lock file for PackageReference or supporting interop assemblies or allowing msbuild assets flow transitively but weren't able to look at this in last 6-8 months. There were other commitments as well like configuring trust policies, authentication plugin for dotnet.exe/ msbuild, etc... (more details at our blogpost or release notes)

But this is next item on the team's list to improve PackageReference support. This may still take some time for someone to be looking at above scenarios and see how to best support these with PackageReference but I can assure it's on the priority list.

[dfoulk]

@jainaashish well when you put it like that 🤣... Thank you for the detailed explanation!

[MauriRojas]

Are there any updates on this?

[mevans-fraedom]

Two years on.
Is there an update?

[ShaneCourtrille]

@mevans-fraedom Based on https://github.com/NuGet/Home/issues/5877 and https://github.com/NuGet/Home/issues/4942 it looks like this has been left behind.. maybe .NET 5 will resolve it?

[StingyJack]

@jainaashish - did you all inspect the current uses of the ps1 files at the time the feature was dropped?

If you did, were there any general categories? That would probably be the best source for public packages, at least better than waiting for people to comment on this thread.

Also, do you recall what percentage of packages used the feature? (Counting it now would produce skewed results unless a point in time query is available)

[mnmr]

I have not been following this thread, but think it is important to throw in some arguments to counter-balance the views above.

While performing all kinds of setup magic can be helpful in reducing first-time friction, it is equally frustrating to work with when you don’t use the package defaults. Move a file elsewhere, and it pops up again when you update. Don’t keep your settings in the config file, you get one now anyway. It’s annoying as hell.

A package manager should download packages, not run them. Running arbitrary powershell downloaded from the internet is dangerous at best, especially considering that many devs mess with the execution policy to reduce daily friction.

There are many other ways to achieve the end goal. Users can download a script and run it. Project templates can be created. Dotnet-tools could do it. Don’t abuse the package manager.

If sanity somehow does not prevail on this issue, at least enforce a rule that packages with setup logic MUST be separate, so people can add it once and then remove the dependency (while keeping the package with the actual libs).

[StingyJack]

@mnmr - I think even this rule should not apply to private repositories. Being able to run a script to complete the installation of a project component is a huge time saver. We are in control of our repository, someone uploading an intentionally poisoned package faces real-world consequences (termination, legal, criminal, etc).

Making everyone take an additional step outside of the package installation is silly when that rule is in place for repositories with unknown or untrusted packages.

[mnmr]

@StingyJack I just don't think all of this "setup magic" belongs in a package manager; whether the feed is private/trusted does not significantly impact this.

Abusing the package manager as a trigger for all sorts of "setup magic" is just not the right solution to the problem. I fully understand the desire to have certain scenarios be easy and friction free, but we're talking once-per-project here.

If your org needs something that can do more than fetch packages, consider creating project templates, global dotnet tools (e.g. to inject stuff into a project after it has been created), custom scripts (you could add pre-build actions to your custom project template to ensure stuff has been added), use chocolatey or some other app installation system, etc.. There are literally a thousand ways to do this, and none of the good solutions involve abusing the package manager.

[BrunoJuchli]

@mnmr

If your org needs something that can do more than fetch packages, consider creating project templates, global dotnet tools (e.g. to inject stuff into a project after it has been created), custom scripts (you could add pre-build actions to your custom project template to ensure stuff has been added), use chocolatey or some other app installation system, etc.. There are literally a thousand ways to do this, and none of the good solutions involve abusing the package manager.

(Emphasis added by me).

I literally challenge you to prove that your statement is true, not just a balloon of hot air.

---

On a more serious note:

(...) Don’t keep your settings in the config file, you get one now anyway. It’s annoying as hell.

But for cases where there is a single way to configure something, for example: there must be that package-specific config file (with a fixed relative path) in case you are using that package, it wouldn't be annoying, no, it would actually be convenient! How else would you offer an automatic migration? With a dotnet tool? Worse usability and not available in classic msbuild.
Otherwise: Just one more way a package author can mess up. Bad package? Nobody forces you to use it...

There are many other ways to achieve the end goal.

But in some cases they might not be as clean nor as user friendly.
So we won't go down that route because that would enable bad usability for other use cases?

Note: I'm not saying that the number of "good" use cases and the ration of "bad" and "good" implementations thereof warrant the feature being re-added.
I'm saying that it's not a simple case of "0 or 1".

[StingyJack]

@mnmr - the package managers job is to install and remove packages. Sometimes it is deficient in this role. The nuget team feels the need to make it more crippled instead of fixing those gaps.

I'm not creating other whole systems just to do things like set the correct build action for a file that a package needs - that the user wants too - when I can cover the gap the nuget team is unwilling to fix with a simple, inspect-able, powershell script. There are a hundred other gaps like this and instead of helping them get bridged, the nuget team is making the schism wider. For what? Something they can't get an antivirus to catch? Some pursuit of engineering perfection? You sound like someone from the latter camp.

The real world likes pretty things, but doesn't have time or mercy for those kind of camping trips. It demands usefulness first.

[Kilowhisky]

It’s pretty simple.

Component Installation != dropping a file in a folder and walking away. That’s a plug-in.

Component Installation involves configuration of dependencies, that sometimes involves scripts. That is it plain and simple.

By removing the install scripts all they are doing is forcing people to do it by hand.

[augustoproiete]

@mnmr:

While performing all kinds of setup magic can be helpful in reducing first-time friction, it is equally frustrating to work with when you don’t use the package defaults

I hear you and I've experienced this pain myself too at times. Still, you're describing issues caused by package authors not doing a good job in terms of experience. I also experienced the opposite, where packages install and clean-up properly and reliably every time... The NuGet install.ps1 feature is not the problem here. Just because some authors don't do a good job, doesn't mean a similar feature shouldn't exist.

Just an idea, but it seems that a command-line argument to prevent scripts from running could ease most of the pain you describe. e.g. dotnet add package mypackage --no-scripts

[augustoproiete]

@mnmr:

Abusing the package manager as a trigger for all sorts of "setup magic" is just not the right solution

I respectfully disagree. Moving the "setup magic" to global dotnet tools doesn't change any of the "setup magic" necessary to consume certain packages... It just moves "the magic" somewhere else, makes it harder for everyone by adding extra steps for both consumers and developers (that now have more documentation to write, and more things to maintain, manage, and deploy)...

The main end-goal of NuGet should be to reduce friction for developers to re-use shared components. Asking users to download project templates separately, run additional scripts manually, and run global tools, is the opposite of reducing friction.

I'd love to get to a place where I can install a NuGet package that will automatically add new project templates related to the package (available only for that solution) and perform any kind of setup needed to get me going immediately.

[augustoproiete]

@mnmr:

A package manager should download packages, not run them. Running arbitrary powershell downloaded from the internet is dangerous at best

Please stop saying that because it's just silly... So "_Running arbitrary powershell downloaded from the internet is dangerous_" but executing arbitrary dotnet global tools downloaded from the internet is not dangerous?" 😃

In fact, if you were truly concerned about security, you wouldn't be using NuGet at all... Any NuGet package today can run arbitrary code on your machine. Not having the install.ps1 doesn't make it "safe" to install NuGet packages...

If you have a minute, install this NuGet package in any project (.NET Core or .NET Framework) and see it of yourself:

---

To be clear, I don't think bringing back install.ps1 is the answer, but I would like to see a strategy that bridges the gap created over two years ago that doesn't involve asking the user to do everything by hand, or jumping through hoops.

[mnmr]

Please forgive me for not quoting you all properly; consolidating your comments into topics helps keep the discussion noise down.

Frustration-Free Packaging
I don't buy the premise that the only solution to low-friction is the ability to execute arbitrary powershell code. If your package is that complicated to use/setup, most likely people will consult your web site anyway. Provide guidance, tools or scripts to help people get started.

Consider the example of Chocolatey. You are asked to open a shell and paste a single line of code. That's hardly friction.

Manual execution also enables customization (you can ask the user for things), which allows for much richer setup experiences.

Security
I absolutely agree that NuGet is currently insecure, but purposefully adding or restoring features that are known to be insecure only ensures that it will never become secure. An insecure package manager is a threat to the whole eco-system and we should stick to requesting features that do not compromise security.

And yes, I do believe that it is more insecure to have code executing automatically (as opposed to being explicitly run). For example, I may have started VS with admin privileges and forgotten about this when later adding a package.

Possible Solutions
I'd support a declarative solution, where package authors could specify certain actions to be performed, as long as the code performing those actions is part of NuGet itself. This would limit the scope of what can be done, thus providing both convenience and security. Even a limited set of actions would probably cover most use cases, so also not a herculean task to implement.

The ability to enable/disable install scripts could work. However, any solution that also requires VS changes (in this case the package manager UI) is much less likely to be adopted (or so I presume).

Using dedicated Setup packages (e.g. MyFancyCMS.Setup.nupkg) that only have the scripts would also go a long way, by making the choice to run something more explicit (and enabling opt-out by not adding the package). However, not all users would know about such a convention and I'm therefore not sure how viable it would be.

[BrunoJuchli]

@mnmr
It could also be designed to require consent to require install/uninstall scripts.
This could also be used to perform opt-out (i'm not sure how useful that is and if there should be a distinction between

• "oh this runs scripts! Might not be safe, I'd rather not install it"
• "I'd rather not run the script but install the package anyway"

I'd support a declarative solution, where package authors could specify certain actions to be performed

Certainly safer. But will be less flexible while costing a whole lot more to implement & maintain. Right?

At any rate, a good overview of the previous use cases is necessary to decide which of those are better addressed with different designs/tools (chocolatey, Dotnet tools,...) or are really best addressed by the package installation/uninstallation process.

[DerekFoulk]

@StingyJack I just don't think all of this "setup magic" belongs in a package manager; whether the feed is private/trusted does not significantly impact this.

Abusing the package manager as a trigger for all sorts of "setup magic" is just not the right solution to the problem. I fully understand the desire to have certain scenarios be easy and friction free, but we're talking once-per-project here.

If your org needs something that can do more than fetch packages, consider creating project templates, global dotnet tools (e.g. to inject stuff into a project after it has been created), custom scripts (you could add pre-build actions to your custom project template to ensure stuff has been added), use chocolatey or some other app installation system, etc.. There are literally a thousand ways to do this, and none of the good solutions involve abusing the package manager.

I just want Fody/PropertyChanged to automatically add FodyWeavers.xml again.

Extra Credit:

• Let me add and transform files when using PackageReference- like Web.config...
• Let me add files to a project, that the user can modify, in .NET Standard

We're all adults that are using source control. If a package modifies files that you don't want it to, you'll see the change before committing them (because you thoroughly review your 'git diff' before commiting 😉).

[jlmarrntx]

Have there been any updates on this topic? I assume ps1 files are still a no go...?

[danielmeza]

@jlmarrntx this is tracked here https://github.com/NuGet/Home/issues/5963

[zkat]

Thanks for all the discussion. We're not going to do this because it's a security risk.

[StingyJack]

@zkat It's not a security risk for private repositories, which is exactly where we usually need special "enterprise" tooling.

Nobody wants to have to do a bunch of stuff manually after installing a package, especially when it can be done automatically. It lowers the value of having a package.

There are other gaps in nuget that the ps1 solves. Maybe you should be fixing them instead of painting us into a corner.

It's not even a security risk for nuget.org, because you already scan packages for malware, etc.

[augustoproiete]

@zkat Is there an alternative plan that mitigates the security risk and, at the same time, allows for library authors to set up things upon package installation (and clean-up upon uninstall)?

Also, is there a plan to also block running MSBuild scripts bundled in the packages? Blocking the execution of PowerShell scripts doesn't mean package authors can't execute arbitrary code in the user's machine.

[IanKemp]

Thanks for all the discussion. We're not going to do this because it's a security risk.

Not acceptable. You cannot simply decide to remove a well-known, well-understood, and often necessary function without providing an alternative.

At least be honest by saying "we can't be arsed to implement this for Core" instead of hiding behind the already-debunked "security risk" excuse.

[gulbanana]

This has nothing to do with .NET Core. The feature is gone if you use PackageReference in old-style .NET Framework csproj files too, and that's a good thing.

[Neoff224]

Completely agree with the people who think that remove this functionality is a BIG mistake.

As it has been said, What about private repositories? And products as Azure DevOps and private organizations with private access and private repositories?

Don't hide yourself behind the excuse about security risk. Simply say, "We don't want to make it possible". The thread would be already closed.