Runtime: [feature/utf8string] Several tests failing with InvalidProgramException related to System.Uri.PrivateParseMinimal

Created on 26 Jul 2018  路  13Comments  路  Source: dotnet/runtime

From https://github.com/dotnet/corefx/pull/31372

This issue is specific to the utf8string branch.

https://mc.dot.net/#/user/ahsonkhan/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Ffeature~2Futf8string~2F/test~2Ffunctional~2Fcli~2F/2e0fd3918e8f373ca7e16da0e3b8d49bd25a135e

These may be due to recent runtime changes: https://github.com/dotnet/coreclr/pull/18841 / https://github.com/dotnet/coreclr/pull/19045

cc @GrabYourPitchforks, @davidwrighton, @AtsushiKan

Here are some example failures:
System.Data.Tests.DataSetInferXmlSchemaTest/ContainsSchema

Unhandled Exception of Type System.InvalidProgramException
Message :
System.InvalidProgramException : Common Language Runtime detected an invalid program.
Stack Trace :
   at System.Uri.PrivateParseMinimal()
   at System.Uri.InitializeUri(ParsingError err, UriKind uriKind, UriFormatException& e) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 99
   at System.Uri.CreateHelper(String uriString, Boolean dontEscape, UriKind uriKind, UriFormatException& e) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 628
   at System.Uri.TryCreate(String uriString, UriKind uriKind, Uri& result) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 253
   at System.Xml.Extensions.ExtensionMethods.ToUri(String s) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Extensions\ExtensionMethods.cs:line 36
   at System.Xml.Serialization.XmlSerializerNamespaces.Add(String prefix, String ns) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Serialization\XmlSerializerNamespaces.cs:line 62
   at System.Xml.Schema.Preprocessor.GetBuildInSchema() in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Schema\Preprocessor.cs:line 436
   at System.Xml.Schema.Preprocessor.Execute(XmlSchema schema, String targetNamespace, Boolean loadExternals) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Schema\Preprocessor.cs:line 160
   at System.Xml.Schema.XmlSchemaSet.PreprocessSchema(XmlSchema& schema, String targetNamespace) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Schema\XmlSchemaSet.cs:line 1226
   at System.Xml.Schema.XmlSchemaSet.Add(String targetNamespace, XmlSchema schema) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Schema\XmlSchemaSet.cs:line 846
   at System.Xml.Schema.XmlSchemaSet.Add(XmlSchema schema) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\src\System\Xml\Schema\XmlSchemaSet.cs:line 489
   at System.Data.DataSet.ReadXSDSchema(XmlReader reader, Boolean denyResolving) in D:\j\workspace\windows-TGrou---545ec183\src\System.Data.Common\src\System\Data\DataSet.cs:line 1764
   at System.Data.DataSet.ReadXml(XmlReader reader, Boolean denyResolving) in D:\j\workspace\windows-TGrou---545ec183\src\System.Data.Common\src\System\Data\DataSet.cs:line 2087
   at System.Data.DataSet.ReadXml(TextReader reader) in D:\j\workspace\windows-TGrou---545ec183\src\System.Data.Common\src\System\Data\DataSet.cs:line 2265
   at System.Data.Tests.DataSetInferXmlSchemaTest.ContainsSchema() in D:\j\workspace\windows-TGrou---545ec183\src\System.Data.Common\tests\System\Data\DataSetInferXmlSchemaTest.cs:line 491

System.Tests.UriCreateStringTests/IsFile_IsUnc(uriString: \"https://host/\", isFile: False, isUnc: False)

Unhandled Exception of Type System.InvalidProgramException
Message :
System.InvalidProgramException : Common Language Runtime detected an invalid program.
Stack Trace :
   at System.Uri.PrivateParseMinimal()
   at System.Uri.InitializeUri(ParsingError err, UriKind uriKind, UriFormatException& e) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 99
   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 34
   at System.Uri..ctor(String uriString) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\Uri.cs:line 338
   at System.Tests.UriCreateStringTests.PerformAction(String uriString, UriKind uriKind, Action`1 action) in D:\j\workspace\windows-TGrou---545ec183\src\System.Runtime\tests\System\Uri.CreateStringTests.cs:line 1260
   at System.Tests.UriCreateStringTests.IsFile_IsUnc(String uriString, Boolean isFile, Boolean isUnc) in D:\j\workspace\windows-TGrou---545ec183\src\System.Runtime\tests\System\Uri.CreateStringTests.cs:line 967

CreateWithAbsolutePathGivesAbsoluteBaseUri

Unhandled Exception of Type System.InvalidProgramException
Message :
System.InvalidProgramException : Common Language Runtime detected an invalid program.
Stack Trace :
   at System.Uri.PrivateParseMinimal()
   at System.Uri.InitializeUri(ParsingError err, UriKind uriKind, UriFormatException& e) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 99
   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\UriExt.cs:line 34
   at System.Uri..ctor(String uriString, UriKind uriKind) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Uri\src\System\Uri.cs:line 381
   at System.Xml.Tests.BaseUriTests.CreateWithAbsolutePathGivesAbsoluteBaseUri(Func`2 factory) in D:\j\workspace\windows-TGrou---545ec183\src\System.Private.Xml\tests\XmlReader\Tests\BaseUriTests.cs:line 35

Also, all the exception checking tests from System.Collections are failing:
System.Collections.Tests.Dictionary_Generic_Tests_SimpleInt_int_With_Comparer_WrapStructural_SimpleInt/IEnumerable_Generic_Enumerator_MoveNext_ModifiedAfterEnumeration_ThrowsInvalidOperationException(count: 1)

Unhandled Exception of Type Xunit.Sdk.AllException
Message :
Assert.All() Failure: 1 out of 4 items in the collection did not pass.
[2]: Xunit.Sdk.ThrowsException: Assert.Throws() Failure
     Expected: typeof(System.InvalidOperationException)
     Actual:   (No exception was thrown)
        at Xunit.Assert.Throws(Type exceptionType, Exception exception) in C:\\BuildAgent\\work\\cb37e9acf085d108\\src\\xunit.assert\\Asserts\\ExceptionAsserts.cs:line 143
        at Xunit.Assert.Throws[T](Func`1 testCode) in C:\\BuildAgent\\work\\cb37e9acf085d108\\src\\xunit.assert\\Asserts\\ExceptionAsserts.cs:line 36
        at System.Collections.Tests.IEnumerable_Generic_Tests`1.<>c__DisplayClass25_0.<IEnumerable_Generic_Enumerator_MoveNext_ModifiedAfterEnumeration_ThrowsInvalidOperationException>b__0(ModifyEnumerable ModifyEnumerable) in D:\\j\\workspace\\windows-TGrou---545ec183\\src\\Common\\tests\\System\\Collections\\IEnumerable.Generic.Tests.cs:line 422
        at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) in C:\\BuildAgent\\work\\cb37e9acf085d108\\src\\xunit.assert\\Asserts\\CollectionAsserts.cs:line 31
Stack Trace :
   at System.Collections.Tests.IEnumerable_Generic_Tests`1.IEnumerable_Generic_Enumerator_MoveNext_ModifiedAfterEnumeration_ThrowsInvalidOperationException(Int32 count) in D:\j\workspace\windows-TGrou---545ec183\src\Common\tests\System\Collections\IEnumerable.Generic.Tests.cs:line 412
area-System.Runtime bug
Source
picture ahsonkhan

All 13 comments

This has been fixed in the coreclr branch.

davidwrighton picture davidwrighton on 27 Jul 2018
👍1

I ran some of the tests in corefx locally and they are now passing. However, we weren't able to get a successful coreclr build with the change yet:
https://github.com/dotnet/versions/blob/370e36164ef030daf1086ac5ef8a7de8956d5724/build-info/dotnet/coreclr/feature/utf8string/Latest.txt#L1

Hence, the auto-PR doesn't include a coreclr update: https://github.com/dotnet/corefx/pull/31374

I have queued a new build, but looking at the past builds, almost all show up as failing: https://devdiv.visualstudio.com/DevDiv/DevDiv%20Team/_Build/index?_a=allDefinitions&path=%5C

cc @MattGal, can you please help with getting a coreclr build going?

ahsonkhan picture ahsonkhan on 27 Jul 2018

cc @dagood

ahsonkhan picture ahsonkhan on 27 Jul 2018

For future reference, the errors are failure to sign with PB_SignType real, because the definitions aren't whitelisted. I noticed the last successful build from the root definition had PB_SignType test, so @ahsonkhan just kicked off a new build with PB_SignType test.

There was also some confusion about where the alphautf8string-26724-08 build came from, because it isn't in the history of DotNet-CoreClr-PipeBuild-Feature-Utf8String. I found it was actually built at DotNet-CoreClr-PipeBuild-Master 20180724-08, which has source branch set to feature/utf8string. I imagine this was to make real signing work. I don't know what the plan is to have these builds happen continuously, and if they need to be signed.

dagood picture dagood on 27 Jul 2018
👍1

The following PR consumes the latest coreclr build and the InvalidProgramException test failures have been fixed after @davidwrighton's change: https://github.com/dotnet/corefx/pull/31374.

However, we still see several test failures related to System.Collections.Tests:
https://mc.dot.net/#/user/dotnet-maestro-bot/pr~2Fjenkins~2Fdotnet~2Fcorefx~2Ffeature~2Futf8string~2F/test~2Ffunctional~2Fcli~2F/08ca1652d9be31d43c29c70e558f06f48f500ba1/workItem/System.Collections.Tests

And the following System.Net.Http test is failing:
System.Net.Http.Functional.Tests.PlatformHandler_HttpClientHandlerTest/Dispose_DisposingHandlerCancelsActiveOperationsWithoutResponses

Unhandled Exception of Type System.Net.Http.HttpRequestException
Message :
System.Net.Http.HttpRequestException : Error while copying content to a stream.
---- System.IO.IOException : The read operation failed, see inner exception.
-------- System.Net.WebException : The request was aborted: The request was canceled.
Stack Trace :
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest.<>c__DisplayClass88_1.<<Dispose_DisposingHandlerCancelsActiveOperationsWithoutResponses>b__2>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Test.Common.LoopbackServer.<CreateServerAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest.<>c__DisplayClass88_0.<<Dispose_DisposingHandlerCancelsActiveOperationsWithoutResponses>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Test.Common.LoopbackServer.<CreateServerAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest.<<Dispose_DisposingHandlerCancelsActiveOperationsWithoutResponses>b__88_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Test.Common.LoopbackServer.<CreateServerAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest.<Dispose_DisposingHandlerCancelsActiveOperationsWithoutResponses>d__88.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
----- Inner Stack Trace -----
   at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
   at System.Net.Http.StreamToStreamCopy.StartRead()
----- Inner Stack Trace -----
   at System.Net.ConnectStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
   at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state)
ahsonkhan picture ahsonkhan on 28 Jul 2018

@ahsonkhan / @dagood not sure if this is resolved yet, but from my discussions with @joshfree and Levi, we've already had clean, real-signed builds of this branch for coreclr, corefx, and core-setup. Unless something's changed since thursday, we're using the master definition, manually queueing, and specifying the branch @ queue time, so the definitions used have to either be white-listed or master would be broken too.

@mmitche had made a cloned definition (only for CoreCLR) but given the infrequent need to build these plus the process of whitelisting definitions, we chose to go this way.

MattGal picture MattGal on 31 Jul 2018

@MattGal That sounds like what I observed, I just don't know where it was communicated. But now that we know it's true, that is resolved. For clarity I think it would be nice to clean up the unused new root definition and the legs it spawned.

dagood picture dagood on 31 Jul 2018

To clarify, though: the issue with not being able to get a green build is resolved, but this issue actually tracks a test failure (not blocking the build) that I don't know the status of.

dagood picture dagood on 31 Jul 2018
👍1

To clarify, though: the issue with not being able to get a green build is resolved, but this issue actually tracks a test failure (not blocking the build) that I don't know the status of.

That is correct. We are successfully getting and consuming builds across coreclr/corefx. This issue is only capturing the specific test failures that might be caused by missing/extra commits.

For instance, @davidwrighton merged a fix for a JIT bug that was introduced recently which resolved most of the test failures. I would imagine the System.Collections.Tests have a similar issue (possibly missing commits within the branch).

ahsonkhan picture ahsonkhan on 31 Jul 2018

@ahsonkhan the collections failures mean that your corelib is missing dotnet/coreclr@a6b9bbbf284496f87c02555aca29ff8f2b762805

danmosemsft picture danmosemsft on 3 Aug 2018
👍1

Thanks @danmosemsft

joshfree picture joshfree on 3 Aug 2018

cc @GrabYourPitchforks - how are you keeping the feature branch in sync with master (if at all)?

ahsonkhan picture ahsonkhan on 3 Aug 2018

This has been fixed in the feature/utf8string branch.

GrabYourPitchforks picture GrabYourPitchforks on 28 Aug 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jamesqo picture jamesqo  路  3Comments
btecu picture btecu  路  3Comments
nalywa picture nalywa  路  3Comments
noahfalk picture noahfalk  路  3Comments
v0l picture v0l  路  3Comments