Home: Revisit deterministic pack

[nkolev92]

The pack implementation for deterministic was reverted in 5.3 due to https://github.com/NuGet/Home/issues/8599.

This issue tracks re-enabling that feature.

Important:

The implementation of this feature should account for https://github.com/NuGet/Home/issues/7001.
Consider the #8603 scenario.

[nkolev92]

Important scenario in https://github.com/NuGet/Home/issues/8603, hit because of https://github.com/NuGet/Home/issues/7001.

tl;dr; 1980-1-1 might not be the best standard date because of timezones.

[poke]

I personally think that having a _real_ date is somewhat useful. Since NuGet probably won't be able to figure out a deterministic _real_ date for a package, I think having a way to externally set the date would be useful.

@tmat said in the dotnet issue the following:

Making the timestamp settable just shifts the problem elsewhere but it does not address the problem.

But I actually think that's exactly a valid way to solve this. NuGet does not have this information, nor has MSBuild. But there are systems that _do_ have this kind of information: version control systems.

When generating a deterministic build and pack output, I would expect that to always happen based on a specific version in the VCS. So when packing deterministically, we could pass the commit date from the source revision the build is based on. That way, there would be a fixed and deterministic date (1) that makes sense (because it's real), (2) and that's even able to identify the source a pack is based on.

I could see that actually figuring out that date is not a responsibility for NuGet (since it would have to interact with the VCS then), so the default could be a different behavior. But then I would really like there to be some way to externally set the date during pack, so that e.g. a build process can pass the revision date to the pack step.

[olivier-spinelli]

Yes, having a real date is somewhat useful, and yes the commit date is the best candidate (when a VCS is used), otherwise the date can be derived from... whatever you can imagine.

Honnestly a simple PackageDate[Time]Utc or a more general BuildDate[Time]Utc property that could be initialized with UtcNow seems perfect to me...

...or did I miss something huge?

[tmat]

I don't see why is there a need for timestamps on the items zipped in the NuGet package. Why wouldn't the version stored in these files be sufficient/better.
NuGet packages already have commit SHA metadata from VCS, so it is easy to find out what source snapshot were they built from.

[nkolev92]

@olivier-spinelli Using UTC now would break determinism.

I don't see why is there a need for timestamps on the items zipped in the NuGet package. Why wouldn't the version stored in these files be sufficient/better.

I agree.

NuGet packages already have commit SHA metadata from VCS, so it is easy to find out what source snapshot were they built from.

The problem is not every package does that. Even then, depending on the timestamp is a bad idea.

[tmat]

he problem is not every package does that. Even then, depending on the timestamp is a bad idea.

Agreed. Just saying we already have a mechanism that associates package with the source snapshot.

[olivier-spinelli]

You got me wrong: I absolutely don't care of all these date/time. My goal is only to achieve "Deterministic Packages".

My "proposal" is only because there are dates at some places (and that purely removing them seems unrealistic).

I'm just saying: let's define ONE MSBuild date property, let's it empty by default (following the MSBuild empty property pattern for (re)definition), at the very last moment, use the Conditional == '' to set the UtcNow time (and yes, in such default, unspecified, scenario, we have lost the "Deterministic Package")...

But for ANY other scenario, we can set this MSBuild Date to be what "we" (we being the tool) want...

[mikeblakeuk]

Where are we with this? It's super confusing to me.

[erdembayar]

@zivkan
I'm solving this problem with together with other problem #7001 and #7395. Please check this comment.

[nkolev92]

@erdembayar

Deterministic pack being enabled by default broke a lot of tooling (admittedly a fragile one, a dependency that should've never been taken).

Please do not do anything for this issue at this point.

[erdembayar]

@erdembayar

Deterministic pack being enabled by default broke a lot of tooling (admittedly a fragile one, a dependency that should've never been taken).

Please do not do anything for this issue at this point.

Sorry for miscommunication. Actually I'm not really fixing it, but I'm making sure my change it ready for deterministic pack re-enabling later. It seems my change it good for it, I tested locally by manually enabling it, currently it's explicitly disabled in code.
Anywhere I'm not going to do anything for this issue at this point as you suggested.

[olivier-spinelli]

... I'm lost ... @nkolev92, please, could you explain the proposal above that could be named "Tool, Please Choose the Date!" MAY weaken anything?
It's not enabled by default, it's just about deciding a release date (and working in UTC of course...).

If it's the (unfortunate choice of) DateTimeOffset that is blocking it can obviously stay (whith a 0 offset) WHEN the <PackageReleaseDate> property is set.

[zivkan]

@olivier-spinelli if you follow the links, you'll find this issue where customers were saying that web deploy was not updating binaries.

In other words, it appears that web deploy is only checking file timestamp. Therefore, if packge.dll has the timestamp in both version 1 and version 2 of the package, web deploy may not detect that the file has changed and therefore needs to be deployed.

Your proposal only works if the package author changes the MSBuild property specifying what file timestamp they want every time they increment the package version. However, the lowest effort, and therefore most likely, usage of your proposal is to set the date once in the project file, and keep using it forever. This will not cause immediate problems to the package author, but will to consumers of the package who are also using web deploy or zip deploy, as explained by the linked issue.