Runtime: How should we address pop-up dialogs during Windows test runs?

[RussKeldorph]

dotnet/runtime#6827 dotnet/runtime#6828 and dotnet/runtime#6829 are examples of assertion failures that pop up and hang the automation system, wasting hours of time before they are finally killed by a "whole job" timeout.

I would have expected COMPlus_NoGuiOnAssert=1 to suppress these, but that doesn't appear to be the case, i.e. the path through System_Private_CoreLib_ni!System.Diagnostics.Assert.Fail, CoreCLR!DebuggerAssert::ShowDefaultAssertDialog, etc. doesn't seem to query that environment variable.

I think it makes sense to do more to prevent popups rather than relying on timeouts.

My naive preference would be that COMPlus_NoGuiOnAssert be made to divert all internal error paths away from UI popup, but I'm not confident enough in the design to claim that outright. (TBH, I would also prefer that the effect of COMPlus_NoGuiOnAssert=1 be the default setting). I feel pretty strongly that a "dialog-killing daemon" that monitors for pop-ups and automatically dismisses them is the wrong solution.

Thoughts?

[RussKeldorph]

@dotnet/coreclr-contrib @dotnet/jit-contrib

[RussKeldorph]

@mmitche

[jashook]

Is it worth disabling this error reporting via modifying a registry setting?

[RussKeldorph]

@jashook Are you saying there's already a registry setting to suppress it? If so, which? In any case, relying on ambient machine state being set or making the automation set (and restore?) it is also less than ideal IMO.

[BruceForstall]

pop-up blocker :-(

I've come to believe it's virtually impossible to block all the pop-ups that want to appear.

[jashook]

I never got around to testing some of the stuff I found when I looked into this problem for arm64. However, there seem to be several examples of suppressing error dialogs around using registry keys. A simple search should give a fair amount of content.

Once again I haven't verified them, just curious if it would be considered an acceptable solution.

[janvorli]

I also got hit by the flood of assert dialogs several times during my local testing. I have made a change that was incorrect, the assert started to fire for all the tests and I got a lot of the assert dialogs on my screen.
I think that we might want to add check for that COMPlus_NoGuiOnAssert variable to the DebuggerAssert::ShowDefaultAssertDialog and return FailTerminate if it is set.

[mmitche]

You should not put up popups if at all possible. It's bad testing hygiene in the first place.

[gkhanna79]

Checked build asserts are there for a reason. Trying to suppress them does not sound the right thing to do. Are you suggesting these asserts were incorrect/false to indicate true failures in the product?

[mmitche]

No, i'm suggesting they should be sent to the console. Having them pop up in dialogs is not useful.

[janvorli]

That's what I was suggesting too. I believe the assert goes to the console too, so disabling the popup with COMPlus_NoGuiOnAssert in the DebuggerAssert::ShowDefaultAssertDialog would have the right effect.

[jkotas]

.NET Core is runtime optimized for clould. It does not ever make sense to display Win32 message boxes in runtime optimized for cloud by default. We should change the default to not display the Win32 message boxes by default. What made sense for WinForms apps in 2001 does not always make sense for cloud today...