Home: Machine readable output for dotnet list package
Details about Problem
It is difficult to parse the output of dotnet list package. We have requirements in our organization to audit packages that we are using in our projects. Without having a good way to parse the output of this command, it makes it difficult to structure the list of packages in a way that can be fed into another auditing system.
Running the following command produces the following output:
dotnet list package --include-transitive
Project 'UniversalDashboard' has the following package references
[net472]:
Top-level Package Requested Resolved
> Microsoft.AspNetCore 2.1.3 2.1.3
> Microsoft.AspNetCore.Authentication.Cookies 2.1.2 2.1.2
> Microsoft.AspNetCore.Authentication.Facebook 2.1.2 2.1.2
> Microsoft.AspNetCore.Authentication.Google 2.1.2 2.1.2
> Microsoft.AspNetCore.Authentication.JwtBearer 2.1.2 2.1.2
> Microsoft.AspNetCore.Authentication.MicrosoftAccount 2.1.2 2.1.2
> Microsoft.AspNetCore.Authentication.Twitter 2.1.2 2.1.2
> Microsoft.AspNetCore.Mvc 2.1.2 2.1.2
> Microsoft.AspNetCore.ResponseCompression 2.1.1 2.1.1
> Microsoft.AspNetCore.Routing 2.1.1 2.1.1
> Microsoft.AspNetCore.Server.Kestrel.Https 2.1.3 2.1.3
> Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv 2.1.3 2.1.3
> Microsoft.AspNetCore.Session 2.1.1 2.1.1
> Microsoft.AspNetCore.SignalR 1.0.3 1.0.3
> Microsoft.AspNetCore.StaticFiles 2.1.1 2.1.1
> Microsoft.AspNetCore.WebSockets 2.1.1 2.1.1
> Microsoft.DotNet.PlatformAbstractions 2.1.0 2.1.0
> Microsoft.Extensions.Caching.Memory 2.1.2 2.1.2
> Microsoft.Extensions.Logging.Debug 2.1.1 2.1.1
> Microsoft.Extensions.Logging.Filter 1.1.2 1.1.2
> Newtonsoft.Json 11.0.2 11.0.2
> NLog 4.5.8 4.5.8
> NLog.Web.AspNetCore 4.6.0 4.6.0
> PowerShellStandard.Library 5.1.0-preview-06 5.1.0-preview-06
> Quartz 3.0.6 3.0.6
> System.Net.NameResolution 4.3.0 4.3.0
> System.Reflection.Emit 4.3.0 4.3.0
> System.Runtime.Loader 4.3.0 4.3.0
> System.Security.Principal 4.3.0 4.3.0
Transitive Package Resolved
> Libuv 1.10.0
> Microsoft.AspNetCore.Antiforgery 2.1.1
> Microsoft.AspNetCore.Authentication 2.1.2
> Microsoft.AspNetCore.Authentication.Abstractions 2.1.1
> Microsoft.AspNetCore.Authentication.Core 2.1.1
> Microsoft.AspNetCore.Authentication.OAuth 2.1.2
> Microsoft.AspNetCore.Authorization 2.1.1
> Microsoft.AspNetCore.Authorization.Policy 2.1.1
> Microsoft.AspNetCore.Connections.Abstractions 2.1.3
> Microsoft.AspNetCore.Cors 2.1.1
> Microsoft.AspNetCore.Cryptography.Internal 2.1.1
> Microsoft.AspNetCore.DataProtection 2.1.1
> Microsoft.AspNetCore.DataProtection.Abstractions 2.1.1
> Microsoft.AspNetCore.Diagnostics 2.1.1
> Microsoft.AspNetCore.Diagnostics.Abstractions 2.1.1
> Microsoft.AspNetCore.HostFiltering 2.1.1
> Microsoft.AspNetCore.Hosting 2.1.1
> Microsoft.AspNetCore.Hosting.Abstractions 2.1.1
> Microsoft.AspNetCore.Hosting.Server.Abstractions 2.1.1
> Microsoft.AspNetCore.Html.Abstractions 2.1.1
> Microsoft.AspNetCore.Http 2.1.1
> Microsoft.AspNetCore.Http.Abstractions 2.1.1
> Microsoft.AspNetCore.Http.Connections 1.0.3
> Microsoft.AspNetCore.Http.Connections.Common 1.0.3
> Microsoft.AspNetCore.Http.Extensions 2.1.1
> Microsoft.AspNetCore.Http.Features 2.1.1
> Microsoft.AspNetCore.HttpOverrides 2.1.1
> Microsoft.AspNetCore.JsonPatch 2.1.1
> Microsoft.AspNetCore.Localization 2.1.1
> Microsoft.AspNetCore.Mvc.Abstractions 2.1.2
> Microsoft.AspNetCore.Mvc.ApiExplorer 2.1.2
> Microsoft.AspNetCore.Mvc.Core 2.1.2
> Microsoft.AspNetCore.Mvc.Cors 2.1.2
> Microsoft.AspNetCore.Mvc.DataAnnotations 2.1.2
> Microsoft.AspNetCore.Mvc.Formatters.Json 2.1.2
> Microsoft.AspNetCore.Mvc.Localization 2.1.2
> Microsoft.AspNetCore.Mvc.Razor 2.1.2
> Microsoft.AspNetCore.Mvc.Razor.Extensions 2.1.1
> Microsoft.AspNetCore.Mvc.RazorPages 2.1.2
> Microsoft.AspNetCore.Mvc.TagHelpers 2.1.2
> Microsoft.AspNetCore.Mvc.ViewFeatures 2.1.2
> Microsoft.AspNetCore.Razor 2.1.1
> Microsoft.AspNetCore.Razor.Design 2.1.1
> Microsoft.AspNetCore.Razor.Language 2.1.1
> Microsoft.AspNetCore.Razor.Runtime 2.1.1
> Microsoft.AspNetCore.ResponseCaching.Abstractions 2.1.1
> Microsoft.AspNetCore.Routing.Abstractions 2.1.1
> Microsoft.AspNetCore.Server.IISIntegration 2.1.1
> Microsoft.AspNetCore.Server.Kestrel 2.1.3
> Microsoft.AspNetCore.Server.Kestrel.Core 2.1.3
> Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions 2.1.3
> Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets 2.1.3
> Microsoft.AspNetCore.SignalR.Common 1.0.3
> Microsoft.AspNetCore.SignalR.Core 1.0.3
> Microsoft.AspNetCore.SignalR.Protocols.Json 1.0.3
> Microsoft.AspNetCore.WebUtilities 2.1.1
> Microsoft.CodeAnalysis.Analyzers 1.1.0
> Microsoft.CodeAnalysis.Common 2.8.0
> Microsoft.CodeAnalysis.CSharp 2.8.0
> Microsoft.CodeAnalysis.Razor 2.1.1
> Microsoft.CSharp 4.5.0
> Microsoft.DiaSymReader.Native 1.7.0
The problem with this type of output is that it is difficult to parse. npm provides two options that make this easier when listing packages. They include both a -parsable and a -json parameter. I think dotnet list package should provide something similar.
I think that dotnet list package should do something similar.
F:\universal-dashboard\src\UniversalDashboard>dotnet --version
2.2.102
I'm happy to work on a PR for this. I've found the responsible code and could implement a "parse-friendly" version but wanted feedback from the community\team before doing anything.
adamdriscoll
All 21 comments
Hey @adamdriscoll
Thanks for opening this issue.
It is something we considered initially, but wasn't added for V1.
@rrelyea @anangaur to chip in on how they'd want this done.
I personally prefer a json option.
nkolev92
on 1 Feb 2019
Json seems fine as most of NuGet generated artifacts are json too.
@adamdriscoll , If you could start with a one pager that includes the proposal on output i.e. the structure of the data and how sample output would look like with different options, that can be a good start. We can review the proposal and then suggest/make changes as required. Since this is not on our priority list, it would be difficult for us to propose a full spec on this in near term.
anangaur
on 2 Feb 2019
json will be good to have.
ignatandrei
on 19 Mar 2019
Having a --json option for dotnet list package would be very convenient indeed, especially when --outdated and --deprecated come into the picture. Making the output machine-readable unlocks additional tooling and automation scenarios.
Based on today's (and planned) console output, here's a suggestion.
Would love feedback/input and make sure we understand what data and format would be desired for this JSON output.
outdated.json
Command: dotnet list package --outdated --include-transitive --json
{
"outdated": [
{
"framework": "netcoreapp2.0",
"topLevelPackages": [
{
"id": "NuGet.Common",
"requestedVersion": "4.8.0-preview1.5146",
"resolvedVersion": "4.8.0-preview1.5146",
"latestVersion": "4.9.1-zlocal.52990"
}
],
"transitivePackages": [
{
"id": "NuGet.Frameworks",
"requestedVersion": "4.8.0-preview3.5278",
"latestVersion": "4.9.1-zlocal.52990"
}
]
}
]
}
deprecated.json
Command: dotnet list package --deprecated --json
{
"deprecated": [
{
"framework": "netcoreapp2.0",
"topLevelPackages": [
{
"id": "My.Legacy.Package",
"resolvedVersion": "2.0.0",
"reasons": ["Legacy"],
"alternativePackage": {
"id": "My.Awesome.Package",
"versionRange": "[3.0.0,)"
}
},
{
"id": "My.Buggy.Package",
"resolvedVersion": "1.1.0",
"reasons": ["Critical Bugs"],
"alternativePackage": {
"id": "My.NotBuggy.Package",
"versionRange": "*"
}
},
{
"id": "My.CompletelyBroken.Package",
"resolvedVersion": "0.9.0",
"reasons": ["Legacy", "Critical Bugs"],
"alternativePackage": null
}
],
"transitivePackages": null
}
]
}
installed-packages.json
Command: dotnet list package --json
This would be very similar to the outdated.json example above but simply listing all installed packages.
xavierdecoster
on 18 Jul 2019
Should it be null or empty object list for no transitivePackages?
For multi-targetting, there is a trade-off (size of the file) between listing the packages separately for each framework vs. having this info as an attribute for each package. I like the current approach better though.
anangaur
on 18 Jul 2019
Should it be null or empty object list for no transitivePackages?
No strong opinion :)
For multi-targetting, there is a trade-off (size of the file) between listing the packages separately for each framework vs. having this info as an attribute for each package. I like the current approach better though.
The current model happens to correspond with the in-memory model used for console printing, which is rather convenient.
xavierdecoster
on 18 Jul 2019
Announcement for the spec review: https://github.com/NuGet/Announcements/issues/35
Spec: Machine readable output for dotnet list package
anangaur
on 29 Jul 2019
@anangaur @xavierdecoster Can you please put the spec in a PR to the https://github.com/NuGet/Client.Engineering repo?
It's very difficult to leave comments on a spec that's in the wiki.
nkolev92
on 29 Jul 2019
I agree with moving the spec to a PR so feedback can be provided more easily.
In addition, the json format needs to take into account that it's possible to run dotnet list package on a solution, in which case it needs to show each project separately. The proposed output only looks suitable for single projects.
zivkan
on 29 Jul 2019
Agree with having the specs as files than wiki. I will try to find a place, for next specs. A common place than having client vs. server places.
anangaur
on 29 Jul 2019
A couple of comments from a first look (please PR it!).
- TransitivePackage should not be null. Just do an empty array.
- How about framework as the key rather than having a framework kvp?
- How do we handle multiple projects?
nkolev92
on 30 Jul 2019
Agree with having the specs as files than wiki. I will try to find a place, for next specs. A common place than having client vs. server places.
A couple of comments from a first look (please PR it!).
Thanks, will update. Where should the spec files go (and the PR)? :)
xavierdecoster
on 31 Jul 2019
Start with https://github.com/NuGet/Client.Engineering, unless @anangaur is ready with a new repo? :)
nkolev92
on 31 Jul 2019
Home/wiki, for now :)
anangaur
on 26 Aug 2019
Another dup in https://github.com/NuGet/Home/issues/9524
nkolev92
on 5 May 2020
Any progress on this issue?
Really need this for handling a farm of projects.
renemadsen
on 30 Jul 2020
/cc: @JonDouglas
We are evaluating this as part of our package vulnerability work.
@renemadsen, it may help if you could elaborate on your scenario a bit.
anangaur
on 30 Jul 2020
Any progress on this?
delepster
on 14 Oct 2020
Hello, any updates?
olivierhelin
on 18 Nov 2020
@anangaur: our use case is to have a documentation over a complete solution (IoT with backend) . We would like to have a distinct list of used libraries and version over the complete solution.
The JSON list could be filtered for duplicates during the build process (instead of dropping a text document to the artifacts) and projected into a new data structure to show library --> applications.
MPapst
on 7 Dec 2020
There are two goals to improve output for dotnet CLI commands using NuGet.
Human-readable output is first and foremost. This should be covered with the current list package command, however we may have some improvements coming to the command to make this better to glance at.
Machine-readable output is second to make output easier to handle, especially complex lists this command can produce (especially with transitive dependencies included)
The original spec: https://github.com/NuGet/Home/wiki/%5BSpec%5D-Machine-readable-output-for-dotnet-list-package includes one aspect of machine-readable output by including --json. We also would like to provide a parseable output such as ---parseable which will provide output in a plain text format separated by tab characters otherwise known as (grepable)
We are working towards our vulnerability work & should be able to consider adding output such as --json & --parseable relatively soon next year.
JonDouglas
on 7 Dec 2020
Related issues
livarcocc
路
3Comments
rrelyea
路
3Comments
msaraf1
路
3Comments
zhili1208
路
3Comments
clairernovotny
路
3Comments
Most helpful comment
Any progress on this?