Home: Emit X-NuGet-Session-Id header to correlate across requests [feature proposal]

[zarenner]

As someone who supports a NuGet repository service, using server logs to correlate multiple HTTP requests from a single invocation of NuGet.exe is extremely difficult today. Being able to do so would be extremely useful for customer support as well as usage metrics.

I propose that NuGet.exe emits a header X-NUGET-SESSION on all requests, which contains a GUID unique per invocation of nuget.exe

I'm currently re-implementing such a feature for npm, which has a "npm-session" header: https://github.com/npm/npm/issues/16889

Note that npm also sends other information such as the command / command line arguments - I am not proposing sending those today in NuGet due to privacy concerns, although I think it could be worth discussing.

I would be happy to implement this feature for NuGet if it's agreeable.

[emgarten]

What is the scope of session, for example would a session last as long as Visual Studio is open, or would it be per operation?

[zarenner]

I think per operation would make more sense, although @jmyersmsft is much more familiar with the codebase than I and mentioned to me that might be difficult today given the code structure. Also, even agreeing on operations may be difficult (e.g. is upgrading a package in multiple projects a single operation? how about multiple packages? multiple searches for a package? etc...)

To be quite honest, since (at least for VSTS Package Management) support requests for VS scenarios are rare in comparison to CI/nuget.exe scenarios, I'd be completely fine not implementing this for VS right now as it might not be worth the implementation cost. Others may disagree.

Thoughts?

[zarenner]

@emgarten Will NuGet accept this change if we implement it?

fyi @jmyersmsft @tkrick

[xavierdecoster]

I'd love to see this happening too btw. Imho, being able to correlate requests to a single user interaction with the package management client/server could be extremely useful for all NuGet repository maintainers out there. As one of the maintainers of MyGet, I'd be happy to see this realized as part of the documented NuGet protocol.

[maartenba]

Any chance this can be added? Would be a troubleshooting life saver!

[emgarten]

being able to correlate requests to a single user interaction with the package management client/server could be extremely useful for all NuGet repository maintainers out there

Single action sounds like a good scope for this. Ex: 1 solution restore, 1 install from the UI across all projects in the solution.

[maartenba]

Yep exactly that. Helps correlate an entire command that performs multiple actions against the server.

[emgarten]

I think this would be a great feature, I can provide help on where this id could be passed around if someone else is interested in creating a PR for this.

For PackageReference restore SourceCacheContext is created once for each restore and passed all the way to the http client: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Protocol/SourceCacheContext.cs

For packages.config https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.PackageManagement/Context/ResolutionContext.cs is typically used across an entire install/update operation.

packages.config restore context is here: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.PackageManagement/IDE/PackageRestoreContext.cs

[maartenba]

Thanks, Justin! Just to test the waters, would it make sense to implement this as a cookie instead? E.g. first request of the session gets a cookie from the server, then passes it to the server on subsequent requests?

[emgarten]

@maartenba what is the benefit of having the server provide the id instead of using a new GUID? Would the server combine multiple sessions if it wanted to?

[maartenba]

Or add additional info into the cookie. But session header feels cleaner and leaner.

[emgarten]

@joelverhagen what would fit best with the current protocol?

[joelverhagen]

I love the idea.

Cookie seems like a close fit but has implementation complexities. For example, in .NET if you want to have a cookie to group a sequence of HTTP requests within a single UI action, I believe you would have to clear the CookieContainer of the cookie after each operation or re-instantiate the HttpClient. Cookies have too much "automatic" whereas this feature we want granular control about when to use a new session ID.

Additionally, there are European laws about cookies. I am no lawyer but having a client mastered header rather than a server mastered cookie seems like a better place to start. We should review with our legal people if there are any concerns with such a session header.

Note that a custom header like this is adopted by many other Microsoft services under the name x-ms-correlation-id. We should have our own header, though, x-nuget-session seems fine.

Some questions to y'all:

1. This is only a request header, correct? Perhaps there is debugging value in having it always returned (and defaulted to Guid.NewGuid() if not provided). Therefore, clients that sent server administrators their HTTP logs would automatically be including correlation data.
2. V2 and V3 API? It general adding x-* headers is harmless so if implemented I think we should add it everywhere.

One gap I foresee is for server administrators that use blob storage. Custom headers do not appear in blob storage logs AFAIK so correlation with blob storage hits will not be complete. A CDN in front of blob storage may have additional features though to fill this gap.

I am working on documenting V3 HTTP protocol (https://github.com/NuGet/NuGetGallery/issues/4103) so once that documentation is done we will have a good place to include the meaning behind this header.

[joelverhagen]

Some other thoughts discussed offline:

1. Client should somehow output the X-NuGet-Session header on a higher-than-default verbosity level. This will allow customers to boost the verbosity and then send client logs to server administrators allows easy correlation.
2. Client can implement this first, and servers can catch up at their own rate.

[joelverhagen]

I propose the capitalization: X-NuGet-Session

This aligns with API key:
https://github.com/NuGet/NuGet.Client/blob/53abd678a121e2a14e8883fe11e6d23d1058725b/src/NuGet.Core/NuGet.Protocol/Resources/PackageUpdateResource.cs#L28

... and client version...
https://github.com/NuGet/NuGet.Client/blob/4eed67e7e159796ae486d2cca406b283e23b6ac8/src/NuGet.Core/NuGet.Protocol/UserAgent.cs#L44

[anangaur]

Shouldn't this be tied to VS session for the NuGet VSIX on VS? Will that be counter-productive? Or do we need two sets of information?

PS: I am not sure what a session means on the CLIs that is equivalent to VS session.

[xavierdecoster]

The _protocol_ should be independent from VS imo. Having a true session id for log correlation is a concept that should be supported by the protocol, no matter what client or server is used. Therefor, I like the proposed X-NuGet-Session HTTP header.

Of course, if desired, the VSIX could reuse some internal VS session id in its implementation of this protocol. Other clients, such as NuGet CLI, Paket, Rider, or even downstream servers, could just generate an id per operation to be used in all HTTP requests sent as part of that operation.

A 'Session' in this context is a set of http requests that are part of the same operation, that could/should be correlated.

e.g. nuget.exe restore solution.sln would be a single session.

[emgarten]

@anangaur I think you've raised a good point, it would be nice if we could correlate these two things. I would expect the the X-NuGet-Session guids used to be logged to the VS telemetry for the session. Using the same session across multiple operations would make it difficult to use however since all operations would end up merged.

[anangaur]

As discussed offline, while i get the idea of collating the NuGet calls arising out of a single NuGet operation but I am unclear about the use-cases.

Can someone specify the kind of issues that you get today and is not easy to diagnose without the header information? @maartenba @xavierdecoster @zarenner

The answer of above question may have an answer to whether or not using VS session id makes sense for NuGet.

[maartenba]

In random order, we would like to:

• Emit an X-NuGet-Warning once per session
• Correlate accessing registrations with packages downloaded (2 separate operations)
• Troubleshoot restore/install/update knowing the session id in AppInsights
• ...

For the CLI, one session is the moment you invoke NuGet.exe until it returns an exit code. E.g. running a restore on a solution.

[xavierdecoster]

Adding to the list:

• preferably avoid having to tell customers they have to reproduce, or run the same command again (which is error-prone, and may be impacted by changes to the machine/environment the command was run on the first time)

That's exactly why having a built-in session-id would be very useful for correlation purposes across client and server(s).

[zarenner]

I agree with @maartenba and @xavierdecoster's points, and want to further point out that correlation is incredibly difficult today especially when multiple operations or even builds take place within a short time period, and/or multiple machines are behind a single IP.

In my experience, working through IDE scenarios with customers tends to be less frequent, more reproducible, easier to correlate without a session ID, and generally less critical scenarios than CLI. So personally I don't have a strong opinion on the IDs emitted within VS or even if we do so now.

[zjrunner]

If there is a need for the VS session and that is a longer scoped session, I say send it in a different header. Keep X-NuGet-Session as close to a single operation from the customer point of view as possible.

When it hits our server, I want to see how one customer action took 1 request (a point restore) or exploded into 500 requests (pages and pages of downloads and dependencies) to understand experience and where to optimize for wider impact. I want to see that the failing request was 1 of 1000 in a huge restore and did it come near the beginning or end of the user operation.

[emgarten]

I've merged this change for NuGet 4.7.

The header name is: X-NuGet-Session-Id

For PackageReference restore operations there be a single session id, for other scenarios such as auto complete, and packages.config restore there may be several different session ids due to how the code is factored. Session id in the code base is now attached to SourceCacheContext objects and in some scenarios multiple contexts are created.