Runtime: AttributeArray_SetGetAttributesAndModify_DoesCopy failed in CI

[stephentoub]

https://dev.azure.com/dnceng/public/_build/results?buildId=345574&view=ms.vss-test-web.build-test-results-tab&runId=10345416&resultId=120899&paneView=attachments

    System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy [FAIL]
      Assert.Same() Failure
      Expected: MockAttribute1 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute1) }
      Actual:   MockAttribute2 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute2) }
      Stack Trace:
        /_/src/System.ComponentModel.TypeConverter/tests/MemberDescriptorTests.cs(322,0): at System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy()

https://github.com/dotnet/corefx/blob/f610925aee0ee4ad0586f677815dd0601209560f/src/System.ComponentModel.TypeConverter/tests/MemberDescriptorTests.cs#L310-L323

[AriNuer]

Same test System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy failed here:

Build:-20190919.9(Master)

Details:
https://helix.dot.net/api/2019-06-17/jobs/83f0b164-d647-4873-ad48-7042933bf457/workitems/System.ComponentModel.TypeConverter.Tests/console

System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy [FAIL]
      Assert.Same() Failure
      Expected: MockAttribute1 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute1) }
      Actual:   MockAttribute2 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute2) }
      Stack Trace:
        /_/src/System.ComponentModel.TypeConverter/tests/MemberDescriptorTests.cs(322,0): at System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy()
  Finished:    System.ComponentModel.TypeConverter.Tests
=== TEST EXECUTION SUMMARY ===
   System.ComponentModel.TypeConverter.Tests  Total: 7723, Errors: 0, Failed: 1, Skipped: 0, Time: 5.932s
----- end Thu 09/19/2019 18:13:32.63 ----- exit code 1 ----------------------------------------------------------

[ViktorHofer]

Happened again on netcoreapp-OSX-Release-x64-OSX.1014.Amd64.Open: https://dnceng.visualstudio.com/public/_build/results?buildId=380360&view=ms.vss-test-web.build-test-results-tab&runId=11756584&resultId=101242&paneView=debug

System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy

Assert.Same() Failure\nExpected: MockAttribute1 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute1) }\nActual: MockAttribute2 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute2) }
   at System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_SetGetAttributesAndModify_DoesCopy() in /_/src/System.ComponentModel.TypeConverter/tests/MemberDescriptorTests.cs:line 323

[ViktorHofer]

cc @maryamariyan

[danmosemsft]

Looking at the code it's not obvious how this could happen. Would probably need to step through it to make sure it's clear what's going on.

[stephentoub]

It most likely has to do with:
https://github.com/dotnet/corefx/blob/b49a8a9be1d53cd9e50cb68fd8540be25c65d433/src/System.ComponentModel.TypeConverter/src/System/ComponentModel/MemberDescriptor.cs#L26
This gets compared to a mutable static here:
https://github.com/dotnet/corefx/blob/b49a8a9be1d53cd9e50cb68fd8540be25c65d433/src/System.ComponentModel.TypeConverter/src/System/ComponentModel/MemberDescriptor.cs#L230-L235
such that if one of the static methods like AddProvider or Refresh is called concurrently at the "wrong" time by another test in the process and causes that static to be incremented, the collection will be forced to reinitialize and the test will be invalidated.

Assuming that's the case, it's a test bug due to the concurrency involved. A quick fix would be to isolate the test with either RemoteInvoke or a serialized test collection.

[AriNuer]

Similar test System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_Set_GetReturnsExpected failed here.

Build: 20191021.64(Master)

Details:
https://dnceng.visualstudio.com/public/_build/results?buildId=396464&view=ms.vss-test-web.build-test-results-tab&runId=12367594&resultId=101657&paneView=debug

Log:

Error message
Assert.Same() Failure
Expected: AttributeCollection [MockAttribute1 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute1) }]
Actual: AttributeCollection [MockAttribute1 { TypeId = typeof(System.ComponentModel.Tests.MemberDescriptorTests+MockAttribute1) }]

Stack trace
   at System.ComponentModel.Tests.MemberDescriptorTests.AttributeArray_Set_GetReturnsExpected(Attribute[] value, Attribute[] expected) in /_/src/System.ComponentModel.TypeConverter/tests/MemberDescriptorTests.cs:line 265

Queue name:
netcoreapp-Linux-Release-x64-(Fedora.29.Amd64.Open)ubuntu.1604.amd64.open