Home: PackRef does not show transitive dependencies - install tab & update tab behavior?

[akamyshanov]

When using "legacy"-style .csproj project files we have a separate packages.config file where all dependencies are listed, including transitive ones. This allows a use case when one installs a package with dependencies and then decides which transitive dependencies can be manually updated. So, the benefits are:

• Dependencies are easily identifiable due to presence of a flat list
• Fine-grain control over all dependency versions

There are drawdowns of course: the flat list can blow up to a huge one.

E.g., after installing Autofac.WebApi2.Owin from NuGet, we have a picture like this:

Transitive dependencies which are clearly viewable can be manually updated very easily.

When using the new Sdk-style .csproj projects NuGet references are added as <PackageReference/> to the project file itself and transitive dependencies are resolved by MSBuild automatically:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net462</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Autofac.WebApi2.Owin" Version="4.0.0" />
  </ItemGroup>
</Project>

So, to update transitive dependencies, one would have to

1. Identify them (e.g. via obj/project.assets.json)
2. Add all of them explicitly to the project
3. Perform updates

And this has to be done after each update and for every (!) transitive dependency in the project which is nearly impossible.

This seems like a "regression" to me.

The best possible resolution would be to show transitive dependency updates in NuGet GUI or via CLI. If the user decides to update one of the dependencies, it would get explicitly added to the project. (similar to https://github.com/NuGet/Home/issues/3159)

Maybe there is some workaround to achieve the specified behaviour?

[davidfowl]

I'm curious, why do you need this? Once you update the transitive dependency do you expect it to be hoisted into your project? Packages can remove dependencies in future versions so why does it matter that the dependency in transitive in the first place?

Why not just add the ones you care about explicitly updating manually to your project?

[akamyshanov]

@davidfowl
Thank you for the answer.

why do you need this?

• Bug/security fixes in the dependencies
• performance improvements
• other reasons

Once you update the transitive dependency do you expect it to be hoisted into your project?

When I explicitly update the dependency, I would like this fact to be somehow respected by NuGet and build tools, yes.

Packages can remove dependencies in future versions so why does it matter that the dependency in transitive in the first place?

Sorry, I do not fully understand your question. If I haven't updated a transitive dependency, it would be removed as it is now. If I have updated it, it should probably stay in the project but it's open for discussion.

My key point here is lack of control from the developer. In my opinion it is important to see the full list of dependencies a project references. I understand that this list can be quite big or depend on the target framework.

• With packages.config I could see all the dependencies and their updates. I may or may not want to update them, but I do see them and can make the decision based on the change in version, release notes, etc.
• Currently with <PackageReference/> the dependencies are a black box with almost zero control from the developer. The behaviour is quite similar to NPM which has always been a headache for frontend developers.

Currently it's not possible to obtain the list of dependencies and to update them in a human-friendly manner (NuGet GUI, CLI).

In my opinion, the most simple solution would be to add a <PackageReference/> to my project in case of a manual update of a particular package. Maybe, the element can be marked that it's just an update and not a direct reference by the project, so it can be removed if there are no packages depending on it.

[davidfowl]

Bug/security fixes in the dependencies
performance improvements
other reasons

Security fixes is a fair concern but honestly this is a rabbit hole, if you have a deep dependency chain and your other dependencies don't update, it's on you to directly reference and update those dependencies transitively. If those transitive dependencies are removed in a future version and you're still using it directly nothing is going to tell you to remove it. In the worst case what you'll end up with is the flattened graph

I'm not sure what you mean by performance improvements means.

I can't think of a single package manager that supports transitive dependencies that treats those transitive dependencies as if they were added top level (showing updates and allowing direct manipulation).

In my opinion, the most simple solution would be to add a to my project in case of a manual update of a particular package. Maybe, the element can be marked that it's just an update and not a direct reference by the project, so it can be removed if there are no packages depending on it.

I agree with the first part but not the rest. Once a dependency is hoisted to the top level, it's treated differently (like any other top level dependency) and you're responsible for updating it. IMO, the default experience should be the way it is today and you can have an option (checkbox in the ui or a command line switch) that shows updates for all transitive dependencies. Clicking update on one of these hoists it to a top level dependency and from then on you're on your own maintaining it like a top level dependency.

[akamyshanov]

if you have a deep dependency chain and your other dependencies don't update, it's on you to directly reference and update those dependencies transitively.

Correct, I just want a human-friendly way to find and perform these updates as it has been with packages.config.

If those transitive dependencies are removed in a future version and you're still using it directly nothing is going to tell you to remove it. In the worst case what you'll end up with is the flattened graph

Yes, correct.

I'm not sure what you mean by performance improvements means.

I mean performance improvements in the updates. But it doesn't really matter, what matters is that reasons may be different.

I can't think of a single package manager that supports transitive dependencies that treats those transitive dependencies as if they were added top level (showing updates and allowing direct manipulation).

NuGet working with packages.config? :) (yes, I know that it just adds all dependencies as top-level ones)

I agree with the first part but not the rest. Once a dependency is hoisted to the top level, it's treated differently (like any other top level dependency) and you're responsible for updating it.

Fair enough.

IMO, the default experience should be the way it is today and you can have an option (checkbox in the ui or a command line switch) that shows updates for all transitive dependencies. Clicking update on one of these hoists it to a top level dependency and from then on you're on your own maintaining it like a top level dependency.

Sure, that was my idea in the first place, the "improvement" was just a proposal.

To sum up, IMO additional tab(s) in the NuGet UI with transitive dependencies and updates (and similar CLI switches) would bring back workflows that are similar to NuGet's behaviour with packages.config that some developers (including me) are so used to. Enabling such a thing can be implemented with a global switch in the settings and/or a project-level setting.

[jackbond]

I'm in the process of updating our projects to the new project format, and found this issue by searching around a bit because I'm facing the same issue. Having read through the existing responses, I must say I'm both confused and a little stunned. Perhaps someone can clarify this scenario a bit.

In one of our projects we're using Autofac and Autofac.Owin. Now in the "old" packages.config world, these would both be "top" level. However in the new approach, I don't need Autofac to be top level because Autofac.Owin will indirectly import Autofac. With this in mind, I removed the explicit top level Autofac import for one of our projects. I got a build error because one of its project references imported v4.6 and the Autofac.Owin only depended on 3.5.2, but that's irrelevant for this discussion.

What is relevant, is that when I go to manage the Nuget packages for the project (or solution) there's nothing telling me "Hey, there's an updated version of Autofac." For my money, that's a showstopper. As @akamyshanov pointed out, what if there's a critical security update in the update? Now you can say, "well, you can add Autofac back as a top level" but that then entirely defeats the point of having transitive dependencies. Further, now I've got to obsessively track a Nuget package's dependencies to make sure they've always got a top level reference. That's work that I've never had to do before. How is this better?

Unless I am fundamentally not understanding something, PackageReferences are a ticking timebomb that are 100% worthless until the Nuget package manager will recursively search a package's dependencies and display ALL the available updates. Please tell me I'm missing something.

[jackbond]

Having reread the comments here, it seems like the concept of references may need to be extended a bit. Right now you can have a top level dependency or a transitive one, but there is clearly a need for something in between. Here's the primary use case,

Package A v4 depends on Package B v2.
Package B updates to v2.1 with critical security fixes

The current solution is to make Package B top level. The problem with that is, if Package A releases v5 that no longer depends on Package B, Package B hangs around for no reason. Ideally what we need is a mechanism to mark that the 2.1 update of Package B is a "TransitiveUpdate" and that if no top level projects or their dependencies depend on PackageB, it should be removed.

[davidfowl]

As @akamyshanov pointed out, what if there's a critical security update in the update? Now you can say, "well, you can add Autofac back as a top level" but that then entirely defeats the point of having transitive dependencies.

Ideally, you'd want the things that depend on the thing with the security update to update as well right? That way you don't end up copying lots of leaf nodes to each project. Neither approach scales IMO, either way you'll end up with top level dependencies.

Ideally what we need is a mechanism to mark that the 2.1 update of Package B is a "TransitiveUpdate" and that if no top level projects or their dependencies depend on PackageB, it should be removed.

Lets take the scenario you mentioned, you start with this project:

Autofac.Owin 4.0.0 -> Autofac 4.0.0 -> System.Magic 1.0.0

<PackageReference Include="Autofac.Owin" Version="4.0.0" />

Time goes by and there's a new version of Autofac 4.0.1 but there's no new Autofac.Owin. So you use this fancy new feature we just invented:

<PackageReference Include="Autofac.Owin" Version="4.0.0" />
<PackageReference Include="Autofac" Version="4.0.1" Transitive="true" />

Then there's another security update for System.Magic 1.0.3:

<PackageReference Include="Autofac.Owin" Version="4.0.0" />
<PackageReference Include="Autofac" Version="4.0.1" Transitive="true" />
<PackageReference Include="System.Magic" Version="1.0.3" Transitive="true" />

Adding this new reference type to avoid the dangling reference problem doesn't buy you much IMO. Instead, maybe a prune command could be introduced to do the same operation (removed things that aren't used).

[jackbond]

@davidfowl

Let me preface all this by saying you've obviously got way more experience with this, and have put in a lot more time thinking about it. My observations are from someone who hasn't needed to, but always had the nice option of right-clicking a solution, selecting "Manage Nuget Packages for Solution", and viewing the Updates tab, and having a pretty good idea of what's changed in the last 24 hours (although I have always wanted the view to be a DataGrid with the name of the package, the version, AND the release date, but that seems even less likely now.) I follow lots of tech news outlets, but this is really the fastest way to know that say the Service Fabric team has put out an update.

Ideally, you'd want the things that depend on the thing with the security update to update as well right?

Sure, but with the old style packages.config approach, they didn't really need to, so there was no rush in putting out an update. And if package authors are truly professional, they'll want to fully test their package with the new dependency before releasing an update. So now, Package B gets a security update, it might be a few weeks before Package A releases an update. Not a big deal in the past; devs would see the Package B update in the updates tab, and independently decide whether to update it. In addition, not all package authors provide great details about what's changed, so if the Package B team releases an update, the Package A team may not know whether the update contains a vital security fix, a regular fix, or perhaps a feature enhancement. If you're the scrum lead for Package Team A, are you going to prioritize updating your Nuget definition on the off chance that Package B has a vital security update?

Instead, maybe a prune command could be introduced to do the same operation

I think the prune command would be a BEAUTIFUL option if there is a redundant top level PackageReference, but it would have to be paired with the ability to see updates for both top level and transitive references. Thus, if a transitive reference released an update, it could temporarily become a top level reference, and when a "real" top level reference included the new version, the temporary top level reference could be removed.

To conclude, if I had to summarize my biggest issue right now, it's that I can't see which transitive packages have updates. So I'm either going to not use PackageReferences or I'm going to have to write a utility that calculates all the packages used by my solution (top level and transitive) and then queries Nuget for available updates. Once I have the list of updates, I'm going to have to decide which packages I'm going to force an update of by promoting them to top level references.

Given the two options, not use PackageReferences or write a tool, I'm probably going to go with option 1 and wait for you guys to "fix" the Nuget tooling. I simply don't have the option of waiting for package authors to update their transitive references. Companies like Equifax do, I don't.

[rrelyea]

Interesting customer feedback from people who have migrated from Packages.Config to PackageReference. @rohit21agrawal, @nkolev92, @jainaashish - please add this user experience different to our PC->PR list of issues, so we can consider as a whole.

[EamonNerbonne]

I think the security hotfix concept is a distraction here. Almost all updates aren't hotfixes; the vast majority of packages you use you will not use in a fashion that is typically exploitable - things that are exposed to the outside world are rare, so typically you'll be left with libs that don't check input for hostile content (but then again, it's not always the case this should be the libs responsibility in the first place).

There's a tradeoff here: do you want the normal workflow to be as smooth as possible (and thus not cluttered with indirect package references that you can really only mess up)? Or do you want to force everyone to explicitly decide about updates, so that potentially security relevant updates (those rare cases!) aren't missed? And don't forget that semver isn't (and cannot really) be enforced: so by updating indirect dependencies, even minor version, you can break your app. It's not even all that unlikely; you merely need to depend on some implementation detail the library does not consider relevant (e.g. the type of return value of some method is unchanged, but the order unspecified and now different!) but you, perhaps unintentionally, depend on.

I think explicitly choosing for simplicity is wise here; the new-style package references with implicit transitive dependencies were the right choice, and that remains so.

If indeed the risk of missing security updates is real - and i'm not saying it isn't - then perhaps there are other approaches to dealing with that with fewer downsides. E.g., it would be cool if a package (version) could be marked obsolete or even explicitly insecure; something like the suggestions made in #2867 sounds more targeted, and really even better for security too.

[akamyshanov]

@EamonNerbonne Everything you're saying is correct, however security updates are just an example. It's not the reason I'd like to focus on but the choice.

do you want the normal workflow to be as smooth as possible (and thus not cluttered with indirect package references that you can really only mess up)?

Yes, of course, smooth workflows are cool :)

Or do you want to force everyone to explicitly decide about updates, so that potentially security relevant updates (those rare cases!) aren't missed?

Absolutely not, I do not want anyone forcing anybody their workflows, that's the point. As stated earlier, this can be an optional switch that enables separate tabs in the UI.

I think explicitly choosing for simplicity is wise here; the new-style package references with implicit transitive dependencies were the right choice, and that remains so.

Not arguing that it's the right choice, I do believe that it really is for a majority of developers.

I just want a way to be able to view all transitive references as a flat list (without going through the tree manually) and decide if I want to update them.

[markusschaber]

Apart from the security issues example discussed above, we have a different use case:

We need this transitive dependency list for license compliance. If we don't know all the indirect dependencies, we don't know which licenses we have to obey (and which packages we should avoid because one of their transitive dependencies comes with a license which is not acceptable in the current project).

Thus, we don't care that much about adding transient references, but we strongly need a way to list all (direct and indirect) dependencies of a project / solution - best if it also contains license indicator.

[anangaur]

Related https://github.com/NuGet/Home/issues/5553

[gordon-matt]

I have a good reason for needing to be able to update transitive dependencies. In my case, I am using the Peachpie PHP compiler for .NET Core. I ran into an issue, which left me stuck unable to update all the dependencies, because 1 of the dependencies was a transitive one and COULD NOT be referenced directly in the main project, as it is the SDK and makes changes to the project (like removing the "Dependencies" folder. You can see an image of the problem here: https://ibb.co/nvix2H.

So I need to be able to update that transitive SDK dependency WITHOUT adding it to the main project.

[jainaashish]

@gordon-matt you can't update transitive dependency without making it top level. This issue is all about making it convenient to show all the transitive dependencies coming along with your top-level and allows you to update any of those transitive by making it as top level too.

Your issue is very specific to your file manager library and it's dependency on peachpie. You'd need to keep releasing new version in order to consume updated peachpie SDK.

[gordon-matt]

@jainaashish Thank you for your response. The point is there are valid reasons (even if they are rare) for wanting to be able to update transitive dependencies without making them top-level. How hard could it be to enable such a thing? Maybe just an additional "Update" option in the right-click context menu when clicking on that transitive dependency in the project. Is this really not possible?

[anangaur]

It boils down to the information that NuGet/project has about these dependencies. With PackageReference, the only information that the project has is the direct dependencies. The transitive dependencies are computed at the time of restore.

In order to persist the transitive dependency's version, the only way possible today will be capturing the information as PackageReference in the project file that will make it direct/top-level dependency. Persisting the transitive dependency information will be a huge change.

We are planning a couple of features to improve PackageReference experiences as captured as part of this Epic issue. I guess Centrally managing package dependencies will solve this problem but it will also change the way you manage package dependencies. We would love hear your feedback.

[nkolev92]

As @anangaur said, there are ways to address your scenarios and plans for improvements for the same.

@gordon-matt
NuGet resolution is rather elaborate, for reference look at our docs

The versions in direct dependencies are the most important ones and NuGet will always attempt to restore those exact one if available, while for the transitive ones, they're dynamic and NuGet would attempt to find the best matching ones, satisfying all the conditions in the direct references (2 different direct package can depend on 2 different version of the same package etc).

Adding a specific version for a transitive dependencies would introduce a lot of complexity, and likely lead to problems trying to resolve the graph. It effectively elevates a transitive dependency to same importance as a direct one (which is what we're recommending be done anyways)
So this is a rather significant undertaking.

[pantonis]

I have just updated to PackageReference. I have one question thought. How can I view the transitive dependencies of a nuget package? Solution explorer just shows the top-level dependencies. Is there any way of viewing them. Maybe adding an expand option?

Thanks

[anangaur]

It already has an expand option for SDK style projects (.NET Core).

[pantonis]

What about .NET?

[mungojam]

For .net projects you can try the project converter that I've contributed to and used : https://github.com/hvanbakel/CsprojToVs2017 or wait for Microsoft's version which they announced at Build conference

[EmilAlipiev]

@mungojam is it not same as right click on packages.config and select "migrate from packages.config to projectreferences" option?

[mungojam]

@EmilAlipiev no, that just converts the way nuget packages are handled. It doesn't convert the project format so the legacy project loader is still used and you don't get essential features like the transitive dependency tree.

[JamesPoli]

I'd just like to get a transitive report; something similar to what ivy does: see http://ant.apache.org/ivy/history/2.1.0/use/report.html, or gradle, see https://stackoverflow.com/questions/21645071/using-gradle-to-find-dependency-tree
Perhaps an option to generate a report to the restore/install commands?

[Euphoric]

This seems relevant, but I would like to also see Consolidate searching for transitive dependencies, and allow to force those dependencies to specific version in projects where there are possible conflicts.
Right now, I'm trying to solve a problem of my application not being able to find correct version of System.ValueTuple, and I identified it as some 3rd party library draging old version of that library, thus breaking BindingRedirects.

[aserraric]

Would it be possible to adopt a system like npm's or yarn's lock files? Wherein transitive dependencies are listed with a absolute versions to ensure that restoring packages always restores the same, on every machine the project is built on.

A transitive dependency can then be updated by just changing the lock file, without touching the direct dependencies. (see "yarn upgrade")

[akamyshanov]

@aserraric
http://lmgtfy.com/?q=nuget+lock
Sorry, couldn't help myself 😃

https://blog.nuget.org/20181217/Enable-repeatable-package-restores-using-a-lock-file.html

[aserraric]

@akamyshanov

Well, then we are halfway there. Now the only thing missing is a way to update transitive dependencies in the lock file, since manual editing is discouraged and error-prone.

[anangaur]

Related suggestion feature request on developercommunity: https://developercommunity.visualstudio.com/content/idea/538119/add-dependency-tree-view-for-packagereferences.html

[JohanLarsson]

Another use case is if the package ships with an analyzer and an update to the analyzer is published. The whole use case discussion is strange, natural to update to the latest versions that satisfies the constraints.

In general the strategy to resolve minimum versions is weird.

Paket updates transitives and uses a lock file for deterministic resolution. Paket also defaults to resolving max version that satisfies constraints but it is configurable.

[TFTomSun]

I would like to add another aspect. We have the semver Version specification. Projects that follow that specification define which future versions are compatible (non-breaking). Up to now we don't have an out of the Box solution to live that semver approach with Nuget to automatically resolve to the latest compatible dependencies.

I would like to have it configurable whether the transitive dependencies of a package references are resolved to the max within the constraints or to the exact version.

An example:

MyApp -> MyLib 1.0.0 -> ExternalLib 1.* (1.0.0 when MyLib was created)

if there's an update for ExternalLib to 1.0.1 I would like to configure it on MyLib PackageReference whether it pulls up ExternalLib or not
e.g.

<PackageReference Include="MyLib" Version="*" DependencyResolution="SemVer"/>

would lead to:

MyApp -> MyLib 1.0.0 -> ExternalLib 1.0.1

when a 2.0.0 update for ExternalLib is release, the resolution of MyLib 1.0.0 would stick with 1.* because that's the given constraint of MyLib to ExternalLib

Today, if I as MyLib developer wants to give access to the latest bits of ExternalLib, I have to create a new Version for MyLib, even if the existing version is fully compatible with the new version of the ExternalLib. Of course the user can explicitly reference the ExternalLib, but if we always have to do it that way, why do we have transitive dependencies? Additionally, the user would have to duplicate the constraint that I put on MyLib, to automatically upgrade only to Versions of the ExternalLib which are supported by MyLib