Home: Get list of all packages using v3 API

[colelloa]

Hey,

I am a software engineer for the open-source specialist company Black Duck Software (https://www.blackducksoftware.com/) and while we currently track NuGet, we are working on upgrading our software so that it uses v3 of the Nuget API.

As I'm sure you know, there is not much documentation available for it. I have reverse engineered most of the API calls that I need, but I cannot figure out how to query the API to get a list of all packages currently hosted on Nuget. We need to get this as an entry point so we can start getting metadata for each package from https://api.nuget.org/v3/registration0/[PKGNAME]/index.json .

Is there any place that has this information, or are there any contacts you can refer me to for answers to this and other basic API questions?

Thank you so much for taking the time to read this.
-Andrew

[emgarten]

The list of all packages on nuget.org can be found in the feed catalog: https://api.nuget.org/v3/catalog0/index.json

There will be a nuget package with an API to help read this in the not so distant future but it is not available yet.

For more information on reading v3 feeds and the catalog take a look at: https://emgarten.com/2016/06/27/understanding-nuget-v3-feeds/

[colelloa]

Awesome - this was exactly what I was looking for. Thank you so much!

[colelloa]

Another followup question about the URL:

Why use catalog0 in https://api.nuget.org/v3/catalog0/index.json ? Will this ever become catalog1, catalog2 etc. as the catalogue becomes full? Is there a way to predict this behavior as scale increases?

[maartenba]

Always check api.nuget.org/v3/index.json for the latest link (type is always "Catalog")

[colelloa]

Got it - thanks again!

[colelloa]

Is there a way to find license information with the API? I've noticed that there is very little metadata information in the catalog0, registration0, and v2v3search links, yet license information is often available on pages such as https://www.nuget.org/packages/NLog/2.0.1.2

[emgarten]

The licenseUrl is available in the catalog. The license name comes from Sonatype and isn't on the v3 feed as far as I know.

[maartenba]

The license URL is also in:

• Search
• Registration blobs

The friendly license name is used only on the gallery (and also only available there, no plans to add it to the API)

[colelloa]

Okay thanks, that'll work. Another question - where does normalized versioning get put into place here? For example, when I query an index page such as https://api.nuget.org/v3/registration1/sleet/index.json#page/0.1.0-alpha-01/0.1.0-alpha-01
and get a response with one to many fields in the 'items', will the 'version' field in each item's 'catalogEntry' be normalized or original? Is the normalized version used in any other aspect besides feed URLs?

[emgarten]

For example, when I query an index page such as https://api.nuget.org/v3/registration1/sleet/index.json#page/0.1.0-alpha-01/0.1.0-alpha-01

The # symbol in the URI means that it's referring to part of the same page. So what you are probably seeing is the entire index come back. Instead you need to resolve that link by finding "@id": "https://api.nuget.org/v3/registration1/sleet/0.1.0-alpha-01.json" and just parsing that json object.

Normalizing the version refers to how you construct version string in the URI, for example 0.1.0-alpha-01 is normalized, but 00000.0000001.0000000-alpha-01 is not normalized.

[colelloa]

I see now, thanks again. I noticed something in v2v3 search - both https://www.nuget.org/packages/bingmapappsdk and https://api.nuget.org/v3/registration0/bingmapappsdk/index.json are valid URIs, but https://api-v2v3search-0.nuget.org/query?q=bingmapappsdk returns nothing. Is this because every version of the package has been unlisted?

[emgarten]

Is this because every version of the package has been unlisted?

Yes, unlisting a package removes it from search

[colelloa]

Hey,

So I have been trying to optimize my connection by using a single TCP session (specifically I am using the python 2 package requests.Session(), if that matters) for my many GET requests to the API.

After a few hours or so with this constant session active, I get a "connection reset by peer" socket error indicating the server has sent an RST bit.

Do you have any insight as to what may be going on?

Thanks.

[emgarten]

nuget.org v3 uses Azure CDN on top of Azure storage, so there isn't anything specific to NuGet that would cause this.

[emgarten]

After a few hours or so with this constant session active, I get a "connection reset by peer" socket error indicating the server has sent an RST bit.

This sounds reasonable for any web server, you should handle this and retry on the client side.

[rrelyea]

Think this is a question thread that is closable now. Please reactivate if I've interpreted incorrectly.