Project-system: Force U2DCheckVerbosity to always be true

[KirillOsenkov]

U2DCheckVerbosity is a very useful flag in the old project system that prints why it was decided that a project needs to be rebuild. Currently you have to go through hell with loading and patching a privateregistry.bin file.

I think there's tremendous value in making this switch true by default.

Additionally, there are places in the old project system where it logs a message such as The input XX was newer than output ''. Where the output is an empty string. Ideally we should investigate why that string is empty.

[panopticoncentral]

This is a bug on the other project system, not this project system. I'd forgotten about it. Can you move it there?

[davkean]

Move it where? This is where we have a bunch of legacy bugs: https://github.com/dotnet/project-system/labels/Area-Legacy-Project-System.

[panopticoncentral]

I didn't realize that we were tracking non-SDK project bugs in GitHub (I had this one and another one, but I'm trying to clean them up). In my understanding, issues with the non-SDK project system should go in https://developercommunity.visualstudio.com. This creates work items in the internal VS repo, which is what we (MSFT folks) almost always are gonna need to link to for PRs in the internal VS repro. It also allows triaging all of, say, the non-SDK project system up-to-date check bugs together (since the vast majority of them are internal VS repo work items). And the system will automatically flow work item resolution back to the developer community report so that you can know exactly when a fixed item is going to show up in an external release.

My feeling is that we should migrate all non-SDK bugs to the developer community over time, or am I off-base? @Pilchie?

[davkean]

Devs don't know what project system is running what, they just know projects; sometimes it's just easier to have one place to file bugs that relate to them.

[panopticoncentral]

Yes, I fully realize that, but what I'm trying to say is that while it might be easier, in my experience bugs related to non-SDK projects tend to receive more attention, are more closely tracked, are more likely to get fixed in a timely manner, and are more likely to receive progress updates (including better notifications on when the fix is actually released) if they are filed on the Developer Community website and not here. Which is why I almost always suggest that non-SDK project bugs be refiled in the Developer Community. I am not saying that this is the best way of organizing things out of all possible options, I'm just suggesting what seems to me to generally produce the best outcome for people wanting something fixed. But, of course, YMMV. ¯_(ツ)_/¯

[KirillOsenkov]

Where is this being tracked now? ;)

[panopticoncentral]

Customer's choice! If you want to reopen here, we can track here, or you can open an issue on developer community. :)

[KirillOsenkov]

Leaving it here as it has context.

[tmeschter]

Tom's triage notes: A useful not not essential feature. As there is also a (painful) workaround, I recommend we close this issue without fixing it.

[KirillOsenkov]

But the fix is super easy, I think? Don't read the registry and just hardcode the value to true? A lot of people still have to investigate the old project system unfortunately...

[tmeschter]

How much additional text would that print to the Output Window? One line of text per project, or one line per project input, or something else?

[KirillOsenkov]

A couple lines per project I think.

[jjmew]

This is legacy and it doesn't meet the bar

[KirillOsenkov]

@davkean @drewnoakes any chance we could reconsider this? A lot of people are going through pain, and messing with registry became really difficult in VS 2019.

[KirillOsenkov]

@aolszowka

[aolszowka]

Yes, or at least have something I can copy and paste to figure it out, I think I've gotten to the root cause of it this time (RAR isn't creating the state file in all cases, bug reporting incoming, soon as soon as I can confirm that my work around fixes it). But it would sure be helpful the next time.

[drewnoakes]

@KirillOsenkov to be clear, you're asking for the default behaviour of the legacy project system to be changed, or at least for an easier way to opt into that logging for legacy projects?

Enabling logging in CPS-based projects is achievable via Tools|Options.

Personally I have very little experience with the legacy project system. @tmeschter and @panopticoncentral would be in a better place to comment here, as well as @jjmew from a prioritisation standpoint.

Additionally, there are places in the old project system where it logs a message such as The input XX was newer than output ''. Where the output is an empty string. Ideally we should investigate why that string is empty.

I haven't seen this before so assume it's a non-issue nowadays. If you have a repro, please let us know.

[KirillOsenkov]

I'm asking to find a place in the legacy project system where we read the registry key U2DCheckVerbosity and replace it with return true; so that we always log the reason why a project is being rebuilt. It should be easy enough and save people from the painful process of figuring out how to patch the registry hive of VS to set that value.

The empty output '' string is a separate issue which I think is unlikely to be fixed because I guess it's more expensive.

But I honestly think we should really turn the logging on by default for the legacy project system so that it no longer requires a registry key to turn on. This should be easy to do and not have bad performance implications (as I've been running with this key my whole life pretty much).

[aolszowka]

I'm asking to find a place in the legacy project system where we read the registry key U2DCheckVerbosity and replace it with return true; so that we always log the reason why a project is being rebuilt. It should be easy enough and save people from the painful process of figuring out how to patch the registry hive of VS to set that value.

This is the best idea out of them all, I can't imagine that there is any reason why we wouldn't ever want that turned on? The only thing I can think of is that there was not a lot of confidence in what it was saying, but at this point something is better than nothing.

[MarkKharitonov]

@KirillOsenkov - I am with you 100%. We have 300+ non SDK projects with 2 solutions holding over 100 projects. Any time the Fast Up To Date heuristic breaks is painful. So the issue is closed, what is the punchline? Was it done (I do not think so)? If not, how on Earth do we set this flag for VS 2019?

[KirillOsenkov]

@jjmew please reconsider, there is a ton of customer demand to fix this painpoint.

[KirillOsenkov]

@davkean

[MarkKharitonov]

For reference, these are my questions on SO:

https://stackoverflow.com/questions/63064150/how-to-troubleshoot-visual-studios-fast-up-to-date-heuristic-unsafe-misfires?r=SearchResults&s=1|47.4196
https://stackoverflow.com/questions/22704068/why-msbuild-is-so-slow-when-nothing-has-changed-since-the-last-build-of-the-solu?r=SearchResults&s=4|26.7205
https://stackoverflow.com/questions/58723870/vs-2017-does-not-perform-the-fast-up-to-date-check-if-shared-bin-directory-is-us

Having this flag saves the mess of diagnostic logging. As if it really works for a solution of 100+ projects. The VS is overwhelmed with the amount of output going to the Output pane and just hangs at some point.

[aolszowka]

@MarkKharitonov You and me friend are on the same ship (We've got 3,000+ non-SDK projects so its even worse) we've seen the same hang in the output panel when trying to troubleshoot it. I assume you've already found MSBuild Log from Kirill above? That tool is the only way you will survive https://msbuildlog.com/

Looking at some of your SO Posts make sure you've already corrected the low laying fruits of the following:

1. On your paths you need to have it resolve the path fully, this is because in the legacy MSBuild system when you put a relative path it simply appends it then thinks you run into long path errors. I can't remember which SO Post I found the answer in but this solved a bunch of our problems, failure to do so results in various issues (some with the WPF Generators having the .g.cs files too long, some with NuGet trying to use a package path too long, etc):

    <BaseIntermediateOutputPath>$([System.IO.Path]::GetFullPath('$(MSBuildProjectDirectory)\..\..\..\..\..\bin\obj\{8827FAD9-177B-4DF2-B910-4664E0DB92FE}'))</BaseIntermediateOutputPath>
    <IntermediateOutputPath>$([System.IO.Path]::GetFullPath('$(MSBuildProjectDirectory)\..\..\..\..\..\bin\obj\{8827FAD9-177B-4DF2-B910-4664E0DB92FE}'))</IntermediateOutputPath>

1. Make sure nothing is marked as "CopyAlways" again this is low hanging fruit you probably already got it
2. All of these hacks are required for us still see the linked issues for write ups:

  <Target Name="GenerateCompiledExpressionsTempFile">
    <!--This is a no-op to overwrite the existing target that ships with MSBuild. When we upgrade to Visual Studio 2019 (MSBuild 16.0) we can remove this. See https://github.com/Microsoft/msbuild/issues/1648-->
  </Target>
  <PropertyGroup>
    <!--This is a hack to try and fix https://github.com/Microsoft/msbuild/issues/4217-->
    <MarkupCompilePass1DependsOn>$(MarkupCompilePass1DependsOn);GenerateTargetFrameworkMonikerAttribute</MarkupCompilePass1DependsOn>
  </PropertyGroup>

1. Watch out for missing Items, on large enough projects your bound to have a few of them. Always forces a rebuild. I wrote this tool to try and identify them in our code base: https://github.com/aolszowka/MsBuildFindMissingCompileItems (MIT Licensed so feel free to fork and take what you need)
2. If you are attempting to restore NuGet packages or build outside of MSBuild you will probably encounter this gem of a bug (still not fixed): https://github.com/NuGet/Home/issues/8272

[MarkKharitonov]

Yes, I am aware of all these things. We have a build time validation that no items have CopyAlways.

And I use msbuild log viewer all the time. Alas, it is useless for troubleshooting Fast Up-To-Date Heuristic misfires, because the first line in the diag log in VS is output by the Heuristic, BEFORE it passes the control to msbuild.

[MarkKharitonov]

Guys, so does anyone know how to set this flag for VS 2019? I really need this.

[fabricefou]

Please, how to set this flag for vs2019? I need it to investigate vcxproj (c++/CLI) systematic rebuild.
Thanks