Home: [PackageReference] Centrally manage NuGet package versions for a solution or a repo

[anangaur]

PackageReference enhancements are summarized as part of this Epic issue: https://github.com/NuGet/Home/issues/6763

• Status: Incubation
• Spec: Centrally manage NuGet package versions for a solution or a repo

[bording]

@anangaur Have you seen https://github.com/Microsoft/MSBuildSdks/tree/master/src/CentralPackageVersions? This seems like it enables the kind of control this requirement is talking about.

[anangaur]

@bording true that. Thanks.
@jeffkl @jainaashish we should sync up on this

[jainaashish]

pretty neat, just wondering how does it work out with VS, if it all it does?

[anangaur]

Lets see how can we make this mainstream in NuGet and if satisfies all the requirements.

[terrajobst]

Do we have a strawman for this, design-wise? I'm not sure where that information would be when this is supported across repo boundaries (as the title suggest we do). At that point, this looks like a feature of the feed. It's almost like we'd like to have snapshotted feeds at that point.

[anangaur]

@terrajobst No designs as such. My current thinking right now is to be able to define it per solution level in a file. And sometimes instead of defining it in the file, it could be imported from some global file present anywhere accessible by NuGet/MSBuild.

The purpose of keeping the term repo is to help not constrain anything for a 'solution' level. There would be some cases where an app is distributed across solutions in a repo.

I agree that if this file starts being a list all versions of packages allowed across a team/product - its becomes similar to just being a curated feed/repository.

[terrajobst]

If we had better tooling for maintaining/populating feeds, creating snapshotted feeds is probably the most logical model. And the file that people need to check in is a nuget.config with the proper feed :-)

[tebeco]

Hello @anangaur @nkolev92

As i changed computer and try to find the docs for Directory.Packages.props i found both :

• https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-package-versions
• https://github.com/microsoft/MSBuildSdks/tree/master/src/CentralPackageVersions

is the Dup' intended ?

Also i spotted this in the Wiki page :
(Sorry the zoom)

but i have no idea where to submit an issue/PR for the Wiki part

[nkolev92]

As i changed computer and try to find the docs for Directory.Packages.props i found both :

It is expected. This is tracking an implementation with similar functionality as the SDK you linked.

Also i spotted this in the Wiki page :

I edited the wiki to point back to this issue because that spec is just a more elaborate version of the one linked in this issue. :)
Lots of people came across the spec you linked but could never figure out the right issue back.

[tebeco]

not sure i understand the wiki part of your comment.
Is it normal to contains twice "github.com" and so that when you click, you are redirected to the wiki home instead of the issue ?

[nkolev92]

oh, I totally missed that. my bad :)
Fixed now :)

[npehrsson]

Any ETA for this?

[ds1709]

Maybe it will be usefull. I have project with two depencies, PackageA and PackageB. They in turn depends (transitive) from Newtonsoft.Json 8.0.1 and Newtonsoft.Json 9.0.1. Because of this, from build to build I have different version of Newtonsoft.Json in output directory. There's more fun when I add package reference on Newtonsoft.Json 12.0.1 directly in project.
In this scenario I need maximum of package version control.

[tebeco]

@ds1709,
As of today you can just use MsBuild variables for that,
There're 2 possible ways since MsBuild 15.x (2017) :

• Use an explicit variable per package at a global level
• Use a target that will apply the Version with an Update

I have created a repository to anticipate the Centralized Nuget Package here:
https://github.com/dotnet-experimentations/CentrallyManagingNuGetPackageVersions

• if you check the folder WithMsBuild on branch master, it's the version of the code using target / Update / Version
• if you check the folder WithMsBuild on branch explicit-variables, it's the version of the code using simple MsBuild Variable at repository level it's simpler, but it won't work in some scenario (like mixing TargetFramework and TargetFrameworks).

It's a bit long to explain but basically:

• Directory.Build.props are applied first.
• TargetFrameworks is transformed to TargetFramework (no 's') by a target

=> this mean you could end up with $(TargetFramework) being unset => empty string if you attempt to use them from Directory.Build.props
=> this is why the <project include="dependencies.props" /> is done in Directory.Build.targets

The approach with Target+Update on the other hand, needs value to be Include(d) first to be updated
=> The line order is important (at the end of the files)

What I mean by Include/Update + Line Order is how ItemGroup works
You need to "Include" something, to be able to match it in a later Update :

<ItemGroup>
  <YourItem Include="MATCH A NAME HERE" OneProps="OneValueOnInclude" />

  <YourItem Update="MATCH A NAME HERE" OneProps="The Later Updated Value" />
</ItemGroup>

[stevenbrix]

Will the VS support for this work for any SDK style project? I know there is no SDK-style for C++ projects, but we are thinking of rolling our own, and would like to make sure that experiences like this will still work

[MidasLamb]

Anything I can do to help with this? I'm very interested in this approach and it would help me out immensely if this gets added.

[tebeco]

if possible that would be awesome if dotnet tool restore and dotnet restore would be able to read RestoreSources from that file too
As we are building sources code from various places :

• Dev Laptop => VPN
• Build Agent => LAN
• Aks Build Agent => Azure

dotnet restore seems to properly loads Directory.Build.xxxx which allow us to do :

<RestoreSources Condition="''$(TF_BUILD)' == 'true'">$(RestoreSources);https://feed/from/aks/network</RestoreSources>
<RestoreSources Condition="''$(TF_BUILD)' == 'true'">$(RestoreSources);https://feed/from/lan/or/vpn</RestoreSources>

Why ?

Because nuget.config is static and url are static/hardcoded

The only alternative is --add-source but that's just the same problem again, while evrything exists in MsBuild, we now have to write a ps1 or sh script to do the if/else to use --add-source

It seems that a Directoy.Packages.props would be a the place to fully replace nuget.config

[nkolev92]

@tebeco
Directory.Packages.props is just another msbuild file. It'll be imported in every project and evaluated in every project individually.
RestoreSources will continue to work the same way it has so far.

RestoreSources has its limitations though: https://github.com/NuGet/Home/issues/5321 & the fact that there's no tooling to edit it, so keep that in mind.

[tebeco]

I suppose I should open an issue on dotnet/sdk so that we could properly use dotnet tool restore with <RestoreSources> ?
We can't reach nuget.org from the LAN (one CI is on the LAN) so we uses Artifactory as a "Remote/Proxy feed"
We also have a CI in AKS and on this side we cannot reach the LAN (on purpose, security reason), but we can reach nuget.org

the goal is to be able to do that:

git clone ...
dotnet tool restore
dotnet restore

For that dotnet tool restore to work, well, it requires that the RestoreSources are "swapped", this works like a charm for dotnet restore
and i can't add a nuget.config as it would have to manually "delete/rename" files right after a clone, with is far form ideal

Do you have idea to solves that ?
I'm just hopping to know if I could replace this :

$NugetSource = "https://lan/url/to/remote/nuget
if($env:TF_BUILD -eq 'true')
{
  $NugetSource = "https://api.nuget.org/v3/index.json"
}

dotnet tool restore --source $NugetSource

[nkolev92]

dotnet tool restore is restore based but is not be considered project based.
Nothing in msbuild truly applies there.

The SDK repo would own that discussion though.

[anangaur]

We do want to think in the direction where you should be able to define sources in this directory.packages.props file. However, as of now, our focus is towards finishing this feature first before venturing on some of the next pieces.

[nkolev92]

Just expanding on @anangaur's point.

Directory.Packages.Props is still MSBuild, so configuration wise everything that works today, will work with centrally managed package versions.
You just will not be able to manage it through the UI. That's covered https://github.com/NuGet/Home/issues/5321.

[tebeco]

thx for the link, it confirms our current use of <RestoreSource> :)
I'll open an issue on dotnet/sdk for dotnet tool to check why/if it can properly load Directory.Build.props

[idlem1nd]

Is there anything we can do to assist with this? We have a hand-rolled targets file that achieves a similar thing, but would prefer to stay with a vanilla usage of NuGet if we can. Is there any ETA for landing the work done so far?

[kant2002]

@idlem1nd depends on your usage. You can use currently proposed design inside VS 16.6
All you need it to create Directory.Packages.props and add property <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>. I add that to Directory.Build.props but maybe you can do that on per-project basic to migrate step-by-step.

[idlem1nd]

@kant2002 thanks but I think that forces _all_ packages to be managed centrally, correct?

[nkolev92]

@idlem1nd

That's the current design of the feature.

You either manage all or none.

Refer to the spec linked in the issue body. There are additional specs linked in that spec.

[tebeco]

If you want to use it right now and only partially you can use something like this:
https://github.com/dotnet-experimentations/CentrallyManagingNuGetPackageVersions
in the WithMsBuild folder

Currently the folder WithDirectoryPackageProps is broken, I created it, waiting for this feature to be delivered so that I can toy with it before switching

[philipreimer]

Hi. I have been pointed to this issue from here: https://github.com/dotnet/sdk/issues/11464

I tried following the discussion here but some things are still not clear to me:

Would what is proposed here be a solution to what I was asking for in the issue mentioned above?

And what is the status of this issue? It's been open for two years and based on some of the comments here there seems to be some kind of usable implementation but it's not clear in what version of dotnet core or nuget.
Directory.packages.props is mentioned several times but I wasn't able to find any documentation about that in Microsoft's docs.

[zivkan]

@philipreimer the feature is still under development. Since the feature is incomplete, the parts that have already been shipped are considered preview, and therefore we haven't documented it yet.

[batzen]

@philipreimer You need the core-sdk 3.1.300 and visual studio 2019.6.
The wiki page was enough documentation for me to get started. The only thing not mentioned there is that you currently have to set ManagePackageVersionsCentrally to true to enable everything.

[philipreimer]

@batzen that would be this wiki page, correct?

I am on Linux - so no VS - but I should be fine with just the dotnet CLI I assume. If I understand the wiki page correctly. So it should also work in a CI environment?

.NET Core SDKs installed:
  3.1.103 [/usr/share/dotnet/sdk]

[philipreimer]

@zivkan any ETA when this feature could be final? Is this planned for a specific upcoming version of dotnet core?

[batzen]

@philipreimer it should also work in CI. It works in our CI system. But you need the 3.1.300 sdk. You seem to be using 3.1.103.

[tfabian]

Hi, I figured out this "work around" in our huge repository:
https://developercommunity.visualstudio.com/content/idea/1019345/nuget-multi-solution-package-references-1.html

In this way I can manage our nuget package references in one place.

[PaulVrugt]

So, what can we do to get this thing moving? We've been waiting for this over 2 years

[batzen]

@PaulVrugt it works since sdk version 3.1.300. Only Visual Studio has no UI support for it yet.

[nkolev92]

Just a plug here to update to the latest 3.1.400 SDK/16.7.
There are a few fixes in the latest version relating to CPVM.

[moly]

Should this feature also centralise lock files? I've set up Directory.Packages.props, but when I run dotnet restore --use-lock-file it still creates separate packages.lock.json files in each of the project folders, which is not what I'd expect.

[tebeco]

I think there are 2 completely different feature and then one should not affect the other so I would not expect this to more or less change anything at how lock files behave today
I think the unified solution lock file is on the table since December 2018 IIRC, but not sure when it will land

[moly]

@tebeco thanks. One other thing, the wiki says that you should be able to add packages through the dotnet cli: https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-package-versions#dotnet-cli-experience

However when I try this it adds the package versions to the .csproj instead of the Directory.Packages.props, which causes an error on restore. I guess this isn't implemented yet either and we have to manually edit the Directory.Packages.props ourselves for now?

[tebeco]

Well, that's a good question (i'm not part of Nuget team / or Msft)
I would guess they'll ask you which specific SDK are you using right now ? I did not test if that worked on my "POC" repo
I think I broke it this weekend when trying to update it

[nkolev92]

@moly

The overall feature is still being worked on. The dotnet.exe commands are not implemented yet, you can track that here: https://github.com/NuGet/Home/issues/9014.

[ps-weber]

When packages.lock.json came out in 2018, a central lock file was suggested:
https://devblogs.microsoft.com/nuget/enable-repeatable-package-restores-using-a-lock-file/#solution-or-repo-lock-file

However, I no longer see this concept mentioned in the spec. Is this still planned?

[tebeco]

hello @ps-weber

This is unrelated to this issue.
Your question is about lockfile while this issue is about "Centrally Managing Package" at once for a solution

Each feature can work without the other
I suggest opening a new issue to avoid the confusion here

[ps-weber]

@tebeco Thanks for the quick answer, it seems I misunderstood the article. I think the author suggests the Directory.Packages.props itself could function as a "central lock file". If I understand the spec correctly, this is not the case.

I also came across an (outdated?) article, which links to this issue and suggests a central package.lock.json: https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-packages

Maybe it should be updated or marked as outdated?

[tebeco]

No worries

Regarding this thread, as @nkolev92 said few message earlier it's being reworked ;)

that's why this page shows it as "reviewing"
https://github.com/NuGet/Home/wiki/Centrally-managing-NuGet-package-versions

[stevenbrix]

Will this be the default behavior for VS/Nuget management of packages? It would be really nice if this was done automagically for you when using VS to manage nuget packages

[tebeco]

I'll let Nuget team answer correctly there. But here is what you can already observe today :

• As of today nothing will change for existing SDK => meaning 3.1.x will need an explicit props to activate it (and the support is limited to few feature IIRC)
• For newer SDK (probably starting 5.0, not sure at all) this will be activated when the file is detected as "existing"
• For VS, if you use Visual Studio Insider you can already observe that there's some sort of integration being worked on, so yes at some point VS will support it

[roji]

I gave this a try, and it's working well using dotnet SDK 5.0.0-rc2. However, when running dotnet msbuild (as opposed to dotnet build) it doesn't get picked up, and I'm getting "X does not contain an inclusive lower bound ...". The reported MSBuild version is 16.8.0-preview-20475-05+aed5e7ed0.

Am I doing something wrong, or is this an issue that needs to be raised with MSBuild or something?

[tebeco]

The error message X does not contain an inclusive lower bound ... means that the attempted PackageReference did not found any associated version.
So there's either

• A typo in package name
• A typo in package version name
• A file not "seen/detected" (file are can sensitive on non-windows OS)
• Might need to explciitly enable that feature for now with a props
• Are you using Cross targettings ? (do you use any TargetFrameworks <= with an s)
• Any condition on Version ?

[roji]

@tebeco please read my above message carefully - things work just fine when doing dotnet build (implying that there's no typo), but fail when doing dotnet msbuild.

[tebeco]

You are right, I should have be more precise,

• This is working fine for me with dotnet build
• no repro to take a look / compare
• The error you could be that it is no able to find the "version string" for that package (but it found a "packagereference include")
• the error could be a side effect of a non-definitive pinned version (Wildcard are denied since 5.0.0 SDK, which look like a very bad idea because it's not supposed to be done by CPVM but lockfile)

as it's working for me here I wondered about possible known edge case I stumbled upon in the past

Can you provide a repro ?

Here is one of the repository that was linked in this issue:
https://github.com/dotnet-experimentations/CentrallyManagingNuGetPackageVersions
I just made few changes right now to make sure it's working with multiple TFM

@roji can you confirm that this repo works for you ?

eg:

Sdk 4.1.403:

wildcard (ok):

"known bound" (warning)

SDK 5.x

Wildcard

Very weird behaviod of dotnet msbuild :D but dotnet restore output the same error msg than dotnet build

error NU1011: Centrally defined floating package versions are not allowed.

Known bounds

I attempted to see but I'm just poking around in the dark here and could not reproduce the difference of behavior.
I managed to have the same warning as you, but when I do, both command seems to output it

[roji]

@tebeco apologies, after looking more closely, dotnet msbuild does seem to work from the command-line for my case; this seems to be more of an IDE-related issue (I'm using Rider). I'll investigate further.

[tebeco]

Arg, this is not going to be fun :s

I hope you find something working for you

[zivkan]

Will this be the default behavior for VS/Nuget management of packages?

It wasn't originally the plan to, but depending on feedback, it could change. Of course, that'd raise a whole bunch of questions about how to make it default without risking breaking projects/solutions. But that's all part of the product development process. But since the feature is still in early preview, I think it's much too early to work out "on by default" problems.

On the topic of being in preview, we're supposed to show a warning message on every restore. It looks like it's not working on the dotnet cli, probably because it has a lower default verbosity level than msbuild.exe. I'll need to get this fixed.

[tebeco]

Will this be the default behavior for VS/Nuget management of packages?

It wasn't originally the plan to, but depending on feedback, it could change. Of course, that'd raise a whole bunch of questions about how to make it default without risking breaking projects/solutions. > But that's all part of the product development process. But since the feature is still in early preview, I think it's much too early to work out "on by default" problems.

Well this is a general issue that can be observed from years now. If the craftmanship does not comes from Microsoft it is not going to be an adopted standard.
So this would definitly be a good thing in the long run that "clean architecture" is being promoted by Nuget Team and Vs Teams:

• Lockfile have a very low adoption rate, and as of today is the best way to "avoid risking project/solution"
• CPVM remove code duplication and should be citizen first
• it should be considered to have at least one template for dotnet new that setup CPVM
• it should be considered to have at least one template for dotnet new that setup lockfile
• OptIn should be part of the culture, rather than "blocking just in case"
• New SDK means that people update their codebase, so just like the AnalyzerLevel in Rsolyn, one could expect adoption of craft / clean practice from the tooling
• As of today, developpers does not know the exists of these feature unless they look for it in Wiki or Github Issue. At some point I expect the CLI to raise warning telling me that this is not what should be done in 2020
• Having VS support would introduce these feature to developer (eg: junior) that had no idea about it without manually opening files, it's now part of the UX

[stevenbrix]

Of course, that'd raise a whole bunch of questions about how to make it default without risking breaking projects/solutions. =

Well this is a general issue that can be observed from years now. If the craftmanship does not comes from Microsoft it is not going to be an adopted standard.

💯. I can't stress how much MS needs to have conviction in the tech we build and guide users down the right path.

This one seems fairly easy to make the default. Does this feature support more than one Directory.Packages.props (like Directory.Build.props)? If so, VS could detect which projects all have the same packages versions and the next time you update packages in VS, drop the files on disk where they will keep your projects on the same packages.

For example if you have the following structure:

  -proja
  -projb
  -projc

if they are all on the same packages, you just drop this:

 Directory.Packages.props
  -proja
  -projb
  -projc

If they don't, then you do something like this:

 Directory.Packages.props
  -proja
   Directory.Packages.props
  -projb
  -projc

At a minimum, if that can't make work because there are edge cases that I don't know of, then a newly created .sln should drop the file on disk. Or there could be a checkbox in the VS Package Manager settings UI. Or even new Items in the "Add New Items" dialogue in VS. I've always felt there needs to be a set of MSBuild items for Directory.Build.props/targets, so it would make sense to have one for Directory.Packages.props as well.

[jmecosta]

comment regarding the override version in project file. Can u instead of have an additional file in the project Directory.Packages.props have override specifiers directlry in the csproj? This is a more clear approach for the end user. since they dont need to guess there is another file they need to bother.

https://github.com/Microsoft/MSBuildSdks/tree/master/src/CentralPackageVersions got it right in my opinion....

[kzu]

I absolutely think that having something like a VersionOverride in the .csproj is the best thing for that scenario. Agreed with @jmecosta .

[tebeco]

Can u instead of have an additional file in the project Directory.Packages.props have override specifiers directlry in the csproj ?

Warning:
do NOT use Directory.Build.props for version but Directory.Build.targets
You're going to have trouble if you cross target : TargetFrameworks + TargetFramework in various project
and when you'll have like:

<ItemGroup Condition=" $(TargetFramework) == 'net5.0'">
  <PackageReference ...... />
</ItemGroup>

props are imported/interpreted at the beginning of the files (csproj, ...) while targets at the end,
so if you use props, then, you will have situation where TargetFramework is null/empty

CPMV is also impacted by this, and this feedback was raised, but not concidered "yet" for the current implementation
I hope that we won't have another overlay to handle TFMs specifically for CPVM, but just straight native support for TFMs and version will be computed properly from targets, and that every TFM declare will create a group automagically

[kzu]

I don't think it's highly relevant that the Directory.Packages.props is instead a .targets, since it just declares the items and it's conceivable useful to be able to override the version via the .csproj (i.e. with a PackageVersion Update="..." Version="..." Condition="...", which would be impossible otherwise (unless you runAfterTargets="CollectCentralPackageVersions"` or something like that).

So, overriding version in the csproj should already be possible, albeit in a more verbose way 🤔