Runtime: LoadedFiles_NonStaticallyReferencedAssembly fails in CI with NotEmptyException

[ahsonkhan]

From https://github.com/dotnet/corefx/pull/28320

OSX.1012.Amd64.Open-x64-Debug
https://mc.dot.net/#/user/sputier/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/47503888725962aaad41a7b1384de12938aae16c/workItem/System.ComponentModel.Composition.Tests/analysis/xunit/System.ComponentModel.Composition.DirectoryCatalogTests~2FLoadedFiles_NonStaticallyReferencedAssembly

Unhandled Exception of Type Xunit.Sdk.NotEmptyException
Message :
Assert.NotEmpty() Failure
Stack Trace :
   at System.ComponentModel.Composition.DirectoryCatalogTests.LoadedFiles_NonStaticallyReferencedAssembly() in /Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/src/System.ComponentModel.Composition/tests/System/ComponentModel/Composition/Hosting/DirectoryCatalogTests.cs:line 356

[safern]

This test was just added here: https://github.com/dotnet/corefx/pull/28173/files

@MarcoRossignoli would you mind taking a look?

[MarcoRossignoli]

@safern @ahsonkhan sorry for delay(in my timezone i was in bed, inefficiencies of the human body). It's weird in last CI i saw no issue:
https://mc.dot.net/#/user/MarcoRossignoli/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/0b3f0316440f69a75062f880533ee82d2ce22a35
(unrelated System.Net.Sockets.Tests).
I'm in trouble, i can dev/test code on Win and Ubuntu(VM) but i don't have a OSX.
Do you have advice?

[danmosemsft]

Has this happened again?

[safern]

I haven't seen it, and no one has reported it. I will try to repro it locally on my mac, if it doesn't repro, I'll close it.

[MarcoRossignoli]

@safern TYVM i apologize!

[MarcoRossignoli]

@safern @danmosemsft i stole a mac, i tested System.ComponentModel.Composition.Tests with ../../../Tools/msbuild.sh /v:m /t:rebuildandtest /p:OSGroup=OSX /p:ConfigurationGroup=Debug /p:ArchGroup=x64 i didn't get any fail.

[safern]

I've ran this multiple times on my mac and haven't been able to repro... also it's been a week already and we haven't had any reports of this failing again on CI. I will close it and if we see it again then we can re-open and try to use the repro tool.

[MarcoRossignoli]

@safern "repro tools"?

[benaadams]

Occurred again https://github.com/dotnet/corefx/pull/28643#issuecomment-377766657

[benaadams]

https://mc.dot.net/#/user/benaadams/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Fmaster~2F/test~2Ffunctional~2Fcli~2F/1ec79ef0959dcb61a9455e4ffef1dbcd1bc6788c/workItem/System.ComponentModel.Composition.Tests/analysis/xunit/System.ComponentModel.Composition.DirectoryCatalogTests~2FLoadedFiles_NonStaticallyReferencedAssembly

[MarcoRossignoli]

@safern can i help with "repro tools" somehow?I'm curious to understand what happen, maybe some overlapped test remove copied file in tmp dir for some reason. Same test here for assembly catalog used internally by DirectoryCatalog.

[karelz]

Late reopening - marking as Post-ZBB

[danmosemsft]

@safern this is still in 2.1. I suggest moving it to 2.2 unless you have a plan/reason to look now.

[safern]

It seems super flaky, I even tried repro tool and it didn’t repro. I think it has to do with the paths where for some reason it is getting a locked file and not able to load the assembly because of that. So I think moving to 2.2 is fine.

[MarcoRossignoli]

@safern if i can help you somehow, i'm here.

[safern]

@danmosemsft have you seen this fail again?

[danmosemsft]

Closing for now then.