Home: support for bulk-updating references into csproj from commandline(s)

[rrelyea]

splitting off from #3751 - which now just covers the add side of this work

Spec draft: https://github.com/NuGet/Home/wiki/Support-dotnet-ref-add%7Cupdate%7Cdelete-packageref

nuget.exe update for package reference

[rrelyea]

since add will support "update", we'll prioritize this task as the lowest of the 4:
1) add 2) remove 3) list 4) update

[dasMulli]

Would it make sense to allow updating all package references? For a lot of dependencies, it is very quick to do sth like dotnet update pkg --all to begin with when updating a project.

[kjbetz]

I would highly suggest checking out how yarn has taken care of this with yarn upgrade and yarn upgrade-interactive

https://yarnpkg.com/lang/en/docs/cli/upgrade/

https://yarnpkg.com/lang/en/docs/cli/upgrade-interactive/

I would just use the same names as well... dotnet upgrade dotnet upgrade-interactive or possibly dotnet package upgrade dotnet package upgrade-interactive

[rrelyea]

Notice that #6193 added the additional requests to support cross-project updates, etc...

[mudrz]

Since this issue is 2 years old - are there any plans to create any basic cli commands for managing package versions?
Currently it is very inconvenient to manage packages outside of Visual Studio on Windows.

[letmaik]

As part of continuous integration I'd like to auto-update dependencies to check that nothing breaks with newer versions. However, this seems impossible without this feature if you manage your dependencies in .csproj files. Any ETA for this? It seems like a basic feature to have.

[mishra14]

@letmaik you can always run dotnet add package <package_name>. This command will find the latest version of the package and update it in your csproj.

[letmaik]

Unfortunately the VSTS .NET Core task does not support add and if you use the "custom command" feature then it doesn't support authentication, which I would need since I use a private VSTS feed. dotnet restore supports authentication of VSTS feeds. Back to square one I think.

[jjgriff93]

Any update on this? Really basic feature making dotnet core harder to work with from VS Code

[mishra14]

@jjgriff93 you can always run dotnet add package <package_name>. This command will find the latest version of the package and update it in your csproj.

[dasMulli]

@jjgriff93 & others: A popular tool to do this is dotnet-outdated which can show you potential updates and apply them to your project.

[jbrejner]

@mishra14 In an continous integration world, where nuget.exe currently is used to restore the packages in a solution ( with WPF projects ), how do it tell dotnet which packages to add to each project ?

[mishra14]

@jbrejner Please see https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-add-package for documentation on dotnet add package command.

[jjgriff93]

Thanks @mishra14 - I want to do this across multiple packages at the same time as it's quite a time consuming process, @dasMulli this is sort of what I was looking for thank you :)

[jbrejner]

@mishra14 Thank you for your suggestion, but that really does not answer my question. This topic or bug concerns the lack of ability to UPDATE existing packages in a project that already includes has the packages it needs, but when a newer version of a package exists we want to update to the latest version - even pre-release if that is available.
We to not use dotnet.exe - just nuget.exe. Nuget.exe lacks does not have support for updating packages in packagereference style projects - so for the time being we can benefit from package references, but are stuck with packages.config format.
the command dotnet add package command expects a package name - so my previous question was to ask you that given that a package name is required by the command you suggested and give that the package is already referenced by the project how do I get the list of referenced packages in a project so I can feed them to the dotnet add package command.
I hope the above clarifies, if not please do not hesitate to ask.
In our scenario we have around 3500 project files in approximately 20 active development branches so the manual editing of project files outside Visual Studio to implement floating versions is not an option.

[mishra14]

@jbrejner

@nkolev92 @PatoBeltran might be able to comment on nuget.exe and/or dotnet list package that could be helpful.

[jbrejner]

Thank you @mishra14, then I hope @nkolev92 and @PatoBeltran have a suggestion.
But I really believe that what the community wants is nuget.exe to support updating packages even if the project uses packagereference style. As far as I have been able to test it so far, the functionality exists and works for projects like wpf projects ( so not .Net Core ) as long as we use with Visual Studio GUI, but when we attempt to build the code in Continous Integration pipelines, where we need to restore and update packages with command line calls - Nuget fails, because nuget.exe expects a packages.config file.

[Falconne]

@jbrejner I needed to do this a while ago so wrote a script that checks each .csproj and if it's a new style project runs dotnet add package and if it's an old style project run nuget update. It's a bit of hassle but shouldn't be too much work for you to do that in your CI scripts.

[dasMulli]

Last time I tried to script it I ran into https://github.com/NuGet/Home/issues/6521 - dotnet add package just doesn't work for multi-targeting projects with both TFM-specific and TFM-invariant packages (dotnet add package -f net472 foo.pkg.id does the wrong thing).

[PeteBoyRocket]

@Falconne to use dotnet add package to update packages you need to know the current packages used. How did you get that information?

Edit: never mind, found it! https://github.com/NuGet/Home/wiki/dotnet-list-package
dotnet list package

[dcumin39]

What is the dotnet equivalent of this:
nuget update {mysolution}.sln -Source LocalFeed -PreRelease

Alternatively something like this:
nuget update *.csproj -Source LocalFeed -PreRelease

[JWilh]

Any news on that, can we expect getting some like that,
cause it's a relativly simple command which would save us time,
as it would especially help working with custom frequently changed nuget packages?
(where it would be really helpfull to specifiy a packagename (with wildcards) and also a "version, -v" and -source parameter)

[nkolev92]

@JWilh

Have you tried use dotnet add package?
It covers most scenarios?

[JWilh]

@nkolev92
I'm working with a powershell script to "update" from our "CI-CD" Packages to my local debug packages. (Isn't the normal workflow, but in some cases it made thinks a lot easier)

CI CD Package Version is something like 0.1.47, my local packages are build as 0.0.0-local-.
For upgrading to my local packages I use a powershell script to "update" to version "0.0.0-local* (so that on rebuild I only need to do a dotnet restore --force)",
which works as desired (but I have to write some detection with dotnet list package and some regex by myself, which works as desired, but... why not have it directly in the cli?

(The "problem" I have with that solution: I have to manually specify the order for update the projects, so I don't get "Package version downgrade errors" and as I have the requirement to simply switch back after I'm done with the work on the nuget package itself (and PR is accepted) to the newly builded version, I rerun the same script (this time simply not giving dotnet add a version).

It works yes, but: "nice is diffrend".

[Craige]

I would like to throw in another vote for an update/upgrade command. dotnet add package is _not_ functionally equivalent, and it's a little baffling that the dotnet package manager doesn't have this option. This is one of the most basic tasks of a package manager.

[PeteBoyRocket]

Best I've got to get round this is parsing the projects in a solution with dotnet sln $solutionPath list, & parsing dotnet list result then using dotnet add one at a time. Seems clunky but works for me. Would love a better solution though.
https://gist.github.com/PeteBoyRocket/5ae25f8fb2a6505a425e778dd35c20a8

[brendanlong]

I'm not sure if this is the right place for this, but on a system with grep, you can implement this with:

dotnet list package --outdated | grep -o '> \S*' | grep '[^> ]*' -o | xargs -L 1 dotnet add package

This uses dotnet list package --outdated to get a list of packages to update, some grep commands that I should be embarassed about writing to extract the package names, and then dotnet add package to do the actual update.

[scabana]

Hi,

I think the need for this just got bumped since dotnet-outdated seems to be deprecated, the github repo is gone and the nuget https://www.nuget.org/packages/dotnet-outdated/ is marked as legacy. Haven't found any real alternatives that's half as good yet. Anyone got a suggestions.

Ideally, this functionality could be pointed at a sln.

[nkolev92]

@scabana

There's a built in functionality for outdated packages in the CLI: https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-list-package

[scabana]

@nkolev92 Thanks for pointing it out :) I did see that and ended up using it. But then you have to manually update the specific csproj with the updated versions which could become a tedious task. A --update option would be very nice since all information required is already present in memory. Or at least a --output json from which we could write some powershell or bash script to do the work.

[cwe1ss]

fyi: There's a great new feature that allows you to manage all package versions in one file: https://stu.dev/managing-package-versions-centrally/

This also makes updating packages easier because you only have to apply the result of dotnet list package --outdated to this one file.

It would still be great to have an --update flag though that automates this!

[scabana]

@cwe1ss Very interesting to have that centrally managed. I'll be investigating that for sure! Thanks for the info. I wonder if dotnet add package *** does the right thing and add the package to the project and its version to the global file (my guess is probably not just yet).

[cwe1ss]

Very interesting to have that centrally managed. I'll be investigating that for sure! Thanks for the info. I wonder if dotnet add package * does the right thing and add the package to the project and its version to the global file (my guess is probably not just yet).

I just tried this. It does not. :)

It just adds the version to the csproj. As that's not allowed with Directory.Packages.props, it results in a build error.

[AlirezaHaghshenas]

This is really frustrating. Imagine a solution with tens of projects and tens of nuget dependencies per project. This leads to many outdated packages. Right now I ran dotnet list package --outdated on a small solution to get 32 results. I want something like: dotnet list package --outdated --upgrade.

• no dotnet add package <package_name> is not viable. One needs to update all outdated packages, with one command.
• no, dotnet-outdated is discontinued and even the source is not available anymore
• .net development in linux still feels like second class citizen. I really suggest discontinuing windows-only tools like nuget.exe in favor of dotnet cli. That can reduce all duplicate efforts and confusion due to feature mismatch.
• On node projects I just run ncu -u and then npm i to get all packages updated, basic functionality.
• If you plan to not implement this, please let the community know, so that one might decide to take on creating a tool for this. I'm actually interested. If you don't, people are always hoping a feature might be around the corner, and all their efforts to implement this as a third-party tool might easily be discarded
• The piped command kindly suggested by @brendanlong does not work for multi-project solutions and I see no easy fix.
• Any other solution based on dotnet add Fails by reverting package references to default state for instance check the sample after this list. A good solution will only update version numbers.

    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.5" />

Will become:

    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.7">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>

Regards

[pdegenhardt]

It is unbelievable to me that this basic functionality is lacking. Obviously Microsoft is not eating their own dog food here. We are in the process of moving a solution with 76 individual projects (including 5 different servers and 8 client applications) over to utilise CI/CD and cloud hosting in the new Azure world that Microsoft has sold our clients. We have followed Microsoft's guidance on this, breaking libraries into nuget packages, with automated publishing to private and public nuget feeds and automated builds of server and client applications then being deployed to target machines.

It looks like this has been a massive waste of time, because the programming reality does not match the marketing hype for this new world.

This issue looks like the ultimate road block. It means that for each of our 12 solutions, they will need to be manually loaded into Visual Studio by a developer, the packages updated by hand and then a new check-in created to trigger publishing of the various artifacts produced by that solution.

True CI/CD appears to simply not be viable at this time for anything but the simplest of applications.

It is amazing to me that Microsoft is still making the same mistakes as they were 20 years ago -- failing to provide upgrade paths for their technology that work in the real world.

[AlirezaHaghshenas]

@pdegenhardt Agree, this can really be annoying. I don't update dependencies in CI, but while the client side of the project takes less that one minute to update, the server-side used to take more than 30 minutes.

Fortunately, we finally see some activity on this. In the meantime, you can consider this workaround:

If you cd into each project directory and run:

dotnet list package --outdated | grep -o '> \S*' | grep '[^> ]*' -o | xargs -L 1 dotnet add package

as suggested by @brendanlong.
It can mess up project, but it does it in a consistent way. For me it adds some lines to some packages, and changes

    <Target DependsOnTargets="ResolveReferences" Name="CopyProjectReferencesToPackage">
        <ItemGroup>
            <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths->WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
        </ItemGroup>
    </Target>

into

    <Target DependsOnTargets="ResolveReferences" Name="CopyProjectReferencesToPackage">
        <ItemGroup>
            <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths-&gt;WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
        </ItemGroup>
    </Target>

Notice how -> is replaced with ->

But it's totally possible that for your projects all is fine. If so, you can create an update script and include it in your CI while the feature is hopefully being implemented.

[JonDouglas]

You're all very right. This is some basic functionality that doesn't exist in the dotnet CLI. You shouldn't have to work with hacks to the existing dotnet add package behavior. You really should be able to ultimately say something like dotnet update package, dotnet list package --update or simply dotnet update which ultimately will go through your project(s) / solution(s) and update all packages to the best of its ability/compatibility similar to as if you would do this manually or automate it with a script.

NuGet has an update all functionality inside the Package Manager Console experience.

https://docs.microsoft.com/en-us/nuget/consume-packages/install-use-packages-powershell#update-a-package

NuGet.exe even have an update command for all packages:

https://docs.microsoft.com/en-us/nuget/consume-packages/install-use-packages-nuget-cli#update-all-packages

We will need to spec this out & design the functionality for dotnet CLI. When we have a design in mind, I'll post on this thread asking for feedback just to ensure we are capturing all of the cases to the best of our abilities. Then we can move forward with landing this work in the CLI & what timeframe we can realistically include this work.

Although I cannot comment on the past or history of this issue, the best I can do is help improve the present & future. Thank you all for being very patient over the years.

Here is an older spec regarding add/update/remove for reference.

[pdegenhardt]

Thanks for the update Jon. Good to hear someone is taking this seriously.

Just to clarify though: while nuget.exe does provide a solution-wide package update function, it does NOT support update of packagereference references – it only supports packages.config. https://github.com/NuGet/Home/issues/6115

So this is a major roadblock for those wanting to move to the much more elegant and manageable packagereference approach. For those with large codebases wanting to progressively upgrade it’s a showstopper.

Nuget was founded to help developers avoid “DLL Hell”. Unfortunately, it has now become a more torturous hell of its own.

[JonDouglas]

@pdegenhardt Thanks for the note, I definitely hear you and have compared all the scenarios as some are less complete than others. I do believe our initial design for this feature should cover your needs, so be on the look-out for a follow up reply so we can ensure we incorporate feedback to ensure this covers the cases we all need.

[jbrejner]

@JonDouglas
Hi Jon. I think the majority here fails to grasp what is being discussed.
What I - and apparently other enterprise based users - is a fix for nuget.exe.
Nuget.exe supports restoring packages even if the project / solution uses packagereference style as opposed to packages.config.
But while nuget.exe supports updating packages if it package.config style it fails if the we use packagereference. And that is simply because nuget.exe looks for a packages.config file. When such file is not found, it dies.

As an enterprise user, handling around 200 .Net Framework based solutions, which in turn produces 200+ packages to internal repositories, and most of them are in alpha state, manual package update is not an option for a continuous integration ( CI ) setup.
And that flaw in nuget.exe prevents me from switching to package reference style, which is a lot cleaner, but I have no option to update a solution to latest available prerelease packages from command line.

I am in the med-tech business, so introducing a new tool like dotnet.exe is really hard AND totally not required, because you have already coded package update. It works via gui or package manager console in visual studio, but with CI, there is no gui.

So the end conclusion is that if an enterprise development effort follows Microsoft recommendations they are either bound to loose or follow more Microsoft and loose anyway.

Let us not forget that Nuget, was the C# / Microsoft attempt to catch up with packaging systems like Maven, RubyGems, NPM, CPAN etc. and frankly speaking Nuget still has a lot to catch up.

Enough ranting.
Preferred solution would be if you would not mind to fix the flaws in nuget.exe where a simple check for existance of a file named packages.config prevents us from updating packages from command line
That could help us getting out of the pain we suffer from having the same information stored in packages.config as well as in the project file.
Alternatively ask your msbuild development team to implement an nuget update target which support packagereferences - like they managed to do for the restore packages.
If those two are too hard, then document how we can call the powershell modules in package manager console with parameters - if you would just make them available to us...

But really - just fix nuget.exe and stop feeding our desire to quit our jobs in favour of an professional employer that for no apparent reason chose to follow Microsoft recommandations.

[AlirezaHaghshenas]

@JonDouglas
My suggestion is: please make sure that the update makes absolutely no changes to the project file. In my solution I faced two such unwanted changes, reported above, but I'll repeat them here for your convenience. This also includes changes in indentation, line ending, spacing of all lines. Only package numbers will be affected. If you don't test for this, I expect many unhappy users to file issues later and even more effort will be spent on this.

It currently changes:

    <Target DependsOnTargets="ResolveReferences" Name="CopyProjectReferencesToPackage">
        <ItemGroup>
            <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths->WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
        </ItemGroup>
    </Target>

into

    <Target DependsOnTargets="ResolveReferences" Name="CopyProjectReferencesToPackage">
        <ItemGroup>
            <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths-&gt;WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))" />
        </ItemGroup>
    </Target>

Notice how -> is replaced with ->

Also it changes

    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.5" />

Into:

    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.7">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>

This is the default setting for the Microsoft.EntityFrameworkCore.Design package, but might not be what the developer wants.

Also, please make sure this experience is available using dotnet cli, to be usable by all .net developers.

[nkolev92]

I faced two such unwanted changes, reported above

Is this happening with dotnet.exe add package?

We'd appreciate if you can create a separate issue for that. It seems orthogonal from the effort here and might not be something that NuGet can control, so it'd be nice to get it to the right component.

[AlirezaHaghshenas]

No, I'm talking about dotnet cli in linux.
Ok. So, I guess I'm on the wrong repository.

[nkolev92]

@AlirezaHaghshenas The nuget functionality in dotnet.exe is tracked in NuGet/Home.

[adrien-constant]

I agree it would be really nice to be able to update all outdated packages with the dotnet tool from command line, like :

dotnet upgrade package --outdated

So it would handle well the dependencies update order, which is not easy to do right now with command line and without Visual Studio.

[zivkan]

dotnet-outdated-tool might do what you want. Scott Hanselman wrote a blog post about it recently. I know a lot of people want 1st party, official tooling, but if you're willing to use community created tools, the tools available to you are more diverse and available today. Whether or not NuGet should duplicate their tooling so that it's built-in to the official tools I'll leave to the PMs, but I think the community makes the ecosystem stronger.

[adrien-constant]

Indeed, I've tried it yesterday and it does not work with my private AzureDev Ops nuget repository. I have opened an issue on their github page. https://github.com/dotnet-outdated/dotnet-outdated/issues/59

I agree it's difficult to provide official tooling for everything, but I also think it would not require too much effort.
The dotnet list package --outdated command works well, it would only require to call dotnet add package for each outdated package (with the correct package update order).

I am using my own solution right now with powershell scripts to update the nuget references of many microservice projects, but it's not a very clean way.

[Spongman]

dotnet list package --outdated generates an output that is wholly unsuitable for scripting, especially when run in the root of a solution containing multiple projects.

[Craige]

@JonDouglas Any update/ETA on this?

[JonDouglas]

@Craige I have a spec ready to publish, but I'm finalizing some larger plans to release first. Give me a week or so. Apologies in advance.