Mono: BCL bootstrap regressions introduced by CoreFX System import on AIX (regex, decimal)
Steps to Reproduce
Build the runtime as normal.
Current Behavior
MCS [basic] mscorlib.dll
../../../external/corefx/src/Common/src/CoreLib/System/Collections/Generic/Dictionary.cs(231,22): warning CS0219: The variable `modified' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Collections/Generic/Dictionary.cs(240,18): warning CS0219: The variable `modified' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs(950,21): warning CS0219: The variable `i' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Number.Formatting.cs(2125,17): warning CS0219: The variable `i' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Runtime/InteropServices/MemoryMarshal.cs(70,22): warning CS0219: The variable `localManager' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Runtime/InteropServices/MemoryMarshal.cs(89,22): warning CS0219: The variable `localManager' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Text/StringBuilder.cs(1760,17): warning CS0219: The variable `deltaLength' is assigned but its value is never used
../../../external/corefx/src/Common/src/CoreLib/System/Text/StringBuilder.cs(1798,25): warning CS0219: The variable `indexBeforeAdjustment' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Date.G.cs(42,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Date.O.cs(52,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Guid.cs(120,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Guid.cs(136,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Guid.cs(150,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Guid.cs(164,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corefx/src/System.Memory/src/System/Buffers/Text/Utf8Formatter/Utf8Formatter.Guid.cs(178,19): warning CS0219: The variable `unused' is assigned but its value is never used
../../../external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/FutureFactory.cs(551,26): warning CS0219: The variable `bWonSetException' is assigned but its value is never used
../../../external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/TaskScheduler.cs(384,53): warning CS0420: `System.Threading.Tasks.TaskScheduler.m_taskSchedulerId': A volatile field references will not be treated as volatile
System.Diagnostics.Tracing/EventSource.cs(161,15): warning CS3006: Overloaded method `System.Diagnostics.Tracing.EventSource.Write<T>(string, ref System.Diagnostics.Tracing.EventSourceOptions, ref T)' differing only in ref or out, or in array rank, is not CLS-compliant
System.Diagnostics.Tracing/EventSource.cs(156,15): (Location of the symbol related to previous warning)
corert/Decimal.cs(684,32): warning CS3006: Overloaded method `System.Decimal.Round(ref decimal, int, System.MidpointRounding)' differing only in ref or out, or in array rank, is not CLS-compliant
corert/Decimal.cs(682,31): (Location of the symbol related to previous warning)
corert/Decimal.cs(892,29): warning CS3006: Overloaded method `System.Decimal.Truncate(ref decimal)' differing only in ref or out, or in array rank, is not CLS-compliant
corert/Decimal.cs(885,31): (Location of the symbol related to previous warning)
../../../external/corefx/src/Common/src/CoreLib/System/Boolean.cs(29,22): warning CS0649: Field `System.Boolean.m_value' is never assigned to, and will always have its default value `false'
../../../external/corefx/src/System.Collections.NonGeneric/src/System/Collections/KeyValuePairs.cs(22,33): warning CS0414: The private field `System.Collections.KeyValuePairs._key' is assigned but its value is never used
../../../external/corefx/src/System.Collections.NonGeneric/src/System/Collections/KeyValuePairs.cs(25,33): warning CS0414: The private field `System.Collections.KeyValuePairs._value' is assigned but its value is never used
Compilation succeeded - 23 warning(s)
Unhandled Exception:
System.ArgumentException: parsing "^(?<flag>--|-|/)(?<name>[^:=]+)((?<sep>[:=])(?<value>.*))?$" - Invalid group name: Group names must begin with a word character.
at System.Text.RegularExpressions.RegexParser.ScanGroupOpen () [0x00229] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.RegexParser.ScanRegex () [0x001aa] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.RegexParser.Parse (System.String re, System.Text.RegularExpressions.RegexOptions op) [0x00036] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout, System.Boolean addToCache) [0x00097] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern) [0x00000] in <a906101b128e42548210c6ff25a3096c>:0
at Mono.Options.OptionSet..ctor (System.Converter`2[TInput,TOutput] localizer) [0x0000b] in <414bbd5b618b4b24a8858aaca22bc9da>:0
at Mono.Options.OptionSet..ctor () [0x00000] in <414bbd5b618b4b24a8858aaca22bc9da>:0
at Program.Main (System.String[] args) [0x00011] in <414bbd5b618b4b24a8858aaca22bc9da>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentException: parsing "^(?<flag>--|-|/)(?<name>[^:=]+)((?<sep>[:=])(?<value>.*))?$" - Invalid group name: Group names must begin with a word character.
at System.Text.RegularExpressions.RegexParser.ScanGroupOpen () [0x00229] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.RegexParser.ScanRegex () [0x001aa] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.RegexParser.Parse (System.String re, System.Text.RegularExpressions.RegexOptions op) [0x00036] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout, System.Boolean addToCache) [0x00097] in <a906101b128e42548210c6ff25a3096c>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern) [0x00000] in <a906101b128e42548210c6ff25a3096c>:0
at Mono.Options.OptionSet..ctor (System.Converter`2[TInput,TOutput] localizer) [0x0000b] in <414bbd5b618b4b24a8858aaca22bc9da>:0
at Mono.Options.OptionSet..ctor () [0x00000] in <414bbd5b618b4b24a8858aaca22bc9da>:0
at Program.Main (System.String[] args) [0x00011] in <414bbd5b618b4b24a8858aaca22bc9da>:0
../../build/library.make:352: recipe for target '../../class/lib/basic/mscorlib.dll' failed
gmake[8]: *** [../../class/lib/basic/mscorlib.dll] Error 1
I'm not sure why the regex parser is broken - I did some testing and it appears on a simple case of an IsMatch on (?<name>.*), the regex on master throws up at RegexParser after a CaptureSlotFromName call, and the position seems to imply that its current position is at the n in name and is expecting a terminator, whereas the pre-breakage one has consumed the name and finds the terminator.
My suspicion is breakage somewhere in Char, maybe interaction with RegexCharClass? Unfortunately, since I can't bootstrap, I can only try to run binaries against monolite and view traces.
Since the file builds, unlike Jenkins, you can try to press on further until you hit an issue on System.Security.
Expected Behavior
Compiles fine.
On which platforms did you notice this
[ ] macOS
[ ] Linux
[ ] Windows
[X} AIX
My suspicion is that this is a general big endian issue, but I don't have access to another big endian platform to verify.
Version Used:
Mono JIT compiler version 5.17.0 (master/e09d690 Sat Jul 21 23:38:14 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS:
SIGSEGV: normal
Notification: Thread + polling
Architecture: ppc
Disabled: none
Misc: softdebug
Interpreter: yes
Suspend: preemptive
GC: sgen
NattyNarwhal
All 34 comments
@NattyNarwhal I'll take a look, it built without errors on the account you gave me, probably I missed some commit
EgorBo
on 22 Jul 2018
Thanks for trying to look into this. There's a lot of moving parts and I have no clue where to start, especially for the actual problem of "being able to bootstrap enough to compile enough core libraries to start WriteLine debugging."
NattyNarwhal
on 22 Jul 2018
@ViktorHofer Looks like a problem that might have been introduced by your recent regex changes.
jkotas
on 22 Jul 2018
I don't think the PR for System imports into Mono added any Regex stuff, checking the diff on the csproj/sources files. I tried unwinding the RegexCharClass class, (unchanged since import) which was a bit convoluted, and I didn't manage to figure it out.
However, you might be onto something if changes on in say, Char, affected things in Regex?
NattyNarwhal
on 22 Jul 2018
Please let me know if you are using Regex from corefx, if yes I'll dig deeper into it. This doesn't repro on netcoreapp21 at least.
ViktorHofer
on 22 Jul 2018
It affects a big endian platform, of which CoreCLR affects none.
Mono is still using Regex from referencesoource.
NattyNarwhal
on 22 Jul 2018
@ViktorHofer Ahahahahahahaha disregard that, I made a mistake when reading, Mono does use Regex from CoreFX. I'll debug there; and slap myself for being that dumb.
NattyNarwhal
on 22 Jul 2018
A diff against Mono CoreFX:
diff --git a/src/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexParser.cs b/src/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexParser.cs
index 971e2dd8f6..77ef4f1935 100644
--- a/src/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexParser.cs
+++ b/src/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexParser.cs
@@ -705,7 +705,9 @@ namespace System.Text.RegularExpressions
if (CharsRight() == 0)
break;
- switch (ch = RightCharMoveRight())
+ ch = RightCharMoveRight();
+ System.Console.WriteLine(" ** RegexParser ** ch {0} len {1} pos {2}", ch, _pattern.Length, _currentPos);
+ switch (ch)
{
case ':':
// noncapturing group
@@ -774,8 +776,9 @@ namespace System.Text.RegularExpressions
capnum = -1;
// check if we have bogus characters after the number
- if (CharsRight() > 0 && !(RightChar() == close || RightChar() == '-'))
- throw MakeException(SR.InvalidGroupName);
+ if (CharsRight() > 0 && !(RightChar() == close || RightChar() == '-')) {
+ System.Console.WriteLine(" ** RegexParser ** Error #1: capnum {0}", capnum);
+ throw MakeException(SR.InvalidGroupName); }
if (capnum == 0)
throw MakeException(SR.CapnumNotZero);
}
@@ -787,8 +790,9 @@ namespace System.Text.RegularExpressions
capnum = CaptureSlotFromName(capname);
// check if we have bogus character after the name
- if (CharsRight() > 0 && !(RightChar() == close || RightChar() == '-'))
- throw MakeException(SR.InvalidGroupName);
+ if (CharsRight() > 0 && !(RightChar() == close || RightChar() == '-')) {
+ System.Console.WriteLine(" ** RegexParser ** Error #2: capnum {0} capname {1} chars right {2} right char {3} close {4}", capnum, capname, CharsRight(), RightChar(), close);
+ throw MakeException(SR.InvalidGroupName); }
}
else if (ch == '-')
{
@@ -797,6 +801,7 @@ namespace System.Text.RegularExpressions
else
{
// bad group name - starts with something other than a word character and isn't a number
+ System.Console.WriteLine(" ** RegexParser ** Error #3: capnum {0}", capnum);
throw MakeException(SR.InvalidGroupName);
}
@@ -815,8 +820,9 @@ namespace System.Text.RegularExpressions
throw MakeException(SR.Format(SR.UndefinedBackref, uncapnum));
// check if we have bogus characters after the number
- if (CharsRight() > 0 && RightChar() != close)
- throw MakeException(SR.InvalidGroupName);
+ if (CharsRight() > 0 && RightChar() != close) {
+ System.Console.WriteLine(" ** RegexParser ** Error #4: capnum {0} uncapnum {1}", capnum, uncapnum);
+ throw MakeException(SR.InvalidGroupName); }
}
else if (RegexCharClass.IsWordChar(ch))
{
@@ -828,12 +834,14 @@ namespace System.Text.RegularExpressions
throw MakeException(SR.Format(SR.UndefinedNameRef, uncapname));
// check if we have bogus character after the name
- if (CharsRight() > 0 && RightChar() != close)
- throw MakeException(SR.InvalidGroupName);
+ if (CharsRight() > 0 && RightChar() != close) {
+ System.Console.WriteLine(" ** RegexParser ** Error #5: capnum {0} uncapnum {1} uncapname {2}", capnum, uncapnum, uncapname);
+ throw MakeException(SR.InvalidGroupName); }
}
else
{
// bad group name - starts with something other than a word character and isn't a number
+ System.Console.WriteLine(" ** RegexParser ** Error #6");
throw MakeException(SR.InvalidGroupName);
}
}
The exception case #2 is what's triggered by my test program, just something that runs IsMatch with (?<name>.*) on the first argument.
Output on aix/ppc64be, which has been broken by the PR:
aix$ MONO_PATH="mcs/class/lib/basic" runtime/mono-wrapper /home/calvin/regex-fucked.exe abcdef
** RegexParser ** ch < len 11 pos 3
** RegexParser ** Error #2: capnum 1 capname n chars right 7 right char a close >
Unhandled Exception:
System.ArgumentException: parsing "(?<name>.*)" - Invalid group name: Group names must begin with a word character.
at System.Text.RegularExpressions.RegexParser.ScanGroupOpen () [0x002ad] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.RegexParser.ScanRegex () [0x001aa] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.RegexParser.Parse (System.String re, System.Text.RegularExpressions.RegexOptions op) [0x00036] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout, System.Boolean addToCache) [0x00097] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex.IsMatch (System.String input, System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout) [0x00000] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex.IsMatch (System.String input, System.String pattern) [0x00000] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at regexFucked.Main (System.String[] args) [0x00006] in <0ede8a6db4384b8598aacba63dcfb32d>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentException: parsing "(?<name>.*)" - Invalid group name: Group names must begin with a word character.
at System.Text.RegularExpressions.RegexParser.ScanGroupOpen () [0x002ad] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.RegexParser.ScanRegex () [0x001aa] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.RegexParser.Parse (System.String re, System.Text.RegularExpressions.RegexOptions op) [0x00036] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex..ctor (System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout, System.Boolean addToCache) [0x00097] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex.IsMatch (System.String input, System.String pattern, System.Text.RegularExpressions.RegexOptions options, System.TimeSpan matchTimeout) [0x00000] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at System.Text.RegularExpressions.Regex.IsMatch (System.String input, System.String pattern) [0x00000] in <9a5729ce8a4a49ff85197ddc732ac983>:0
at regexFucked.Main (System.String[] args) [0x00006] in <0ede8a6db4384b8598aacba63dcfb32d>:0
Output on FreeBSD/amd64, which works:
** RegexParser ** ch < len 11 pos 3
True
(Note that I'm getting the core libraries in on AIX even though I can't build on AIX by just copying the profile artifact directories from the FreeBSD VM to the AIX box; for when @EgorBo gets the time to look and needs to debug.)
NattyNarwhal
on 22 Jul 2018
With some more printf, I think the problem is in ScanCapname; my FreeBSD VM returns ** ScanCapname ** returns name startpos 3 curpos 7 with a printf, and the AIX system has ** ScanCapname ** returns n startpos 3 curpos 4. Since ScanCapname basically is just a counter until it hits a "non-word char," and the PR didn't touch RegexCharClass, I assume something in Char might have broke it? Or the Char and RegexCharClass are desynced? (the ctor used to init the values for RegexCharClass looked pretty hairy and I couldn't make out what it was trying, but I was looking at the referencesource version, so....
NattyNarwhal
on 23 Jul 2018
Yeah, something's fucky:
AIX:
** IsWordChar ** testing n.... True
** IsWordChar ** testing n.... True
** IsWordChar ** testing a.... False
** ScanCapname ** returns n startpos 3 curpos 4
[...]
FreeBSD/amd64:
** IsWordChar ** testing n.... True
** IsWordChar ** testing n.... True
** IsWordChar ** testing a.... True
** IsWordChar ** testing m.... True
** IsWordChar ** testing e.... True
** IsWordChar ** testing >.... False
** ScanCapname ** returns name startpos 3 curpos 7
[...]
Comparing outputs of CharInClassRecursive:
AIX:
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing n against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing n against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch a mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b False
** CharInClassRecursive ** after the b inversion: b False return value of b && !subtracted False
** IsWordChar ** testing a against WordClass.... False
** CharInClassRecursive ** before the b inversion: ch a mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b False
** CharInClassRecursive ** after the b inversion: b False return value of b && !subtracted False
** ScanCapname ** returns n startpos 3 curpos 4
[...]
FreeBSD/amd64:
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing n against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing n against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch n mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch a mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing a against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch a mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch m mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing m against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch m mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch e mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** IsWordChar ** testing e against WordClass.... True
** CharInClassRecursive ** before the b inversion: ch e mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b True
** CharInClassRecursive ** after the b inversion: b True return value of b && !subtracted True
** CharInClassRecursive ** before the b inversion: ch > mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b False
** CharInClassRecursive ** after the b inversion: b False return value of b && !subtracted False
** IsWordChar ** testing > against WordClass.... False
** CharInClassRecursive ** before the b inversion: ch > mySetLength 0 myCategoryLength 10 myEndPosition 13 subtracted False b False
** CharInClassRecursive ** after the b inversion: b False return value of b && !subtracted False
** ScanCapname ** returns name startpos 3 curpos 7
Sorry if the volume of comments is high and getting annoying, I'm debugging while doing other stuff and periodically checking it while adding more printf/trying to grok what things do.
NattyNarwhal
on 23 Jul 2018
Spot the difference in CharInCategory:
** CharInCategory ** begin: ch a i 3 end 13 chcat ModifierSymbol
** CharInCategory ** begin: ch a i 3 end 13 chcat LowercaseLetter
NattyNarwhal
on 23 Jul 2018
In the PR on csproj:
+ <Compile Include="..\..\..\external\corefx\src\Common\src\CoreLib\System\Globalization\CharUnicodeInfo.cs" />
And what chcat is:
UnicodeCategory chcategory = CharUnicodeInfo.GetUnicodeCategory(ch);
And what's in this class:
//--------------------------------------------------------------------//
// Internal Information //
//--------------------------------------------------------------------//
//
// Native methods to access the Unicode category data tables in charinfo.nlp.
//
internal const char HIGH_SURROGATE_START = '\ud800';
internal const char HIGH_SURROGATE_END = '\udbff';
internal const char LOW_SURROGATE_START = '\udc00';
internal const char LOW_SURROGATE_END = '\udfff';
internal const int UNICODE_CATEGORY_OFFSET = 0;
internal const int BIDI_CATEGORY_OFFSET = 1;
// The starting codepoint for Unicode plane 1. Plane 1 contains 0x010000 ~ 0x01ffff.
internal const int UNICODE_PLANE01_START = 0x10000;
I can tell this will be fun already 🤢
NattyNarwhal
on 23 Jul 2018
Oh my god, it's full of bit shifts and bitwise ands, with a heaping of unsafe (the entire file is like that, god knows what other horrors for big endian could lurk inside 😱)
But more spot the difference:
** GetUnicodeCategory ** entering for 'a'
** InternalGetCategoryValue ** BITSHIFT: ch 97 ch >> 8 0 leads to s_pCategoryLevel1Index[ch >> 8] 4352
** InternalGetCategoryValue ** BITSHIFT: ch 97 ((ch >> 4) & 0x000f) 6 leads to s_pCategoryLevel1Index[index + ((ch >> 4) & 0x000f)] 4883
** InternalGetCategoryValue ** BITWISE AND: ch 97 ch & 0x000f 1 leads to pBytePtr[(ch & 0x000f)] 17
** CharInCategory ** begin: ch a i 3 end 13 chcat ModifierSymbol
** GetUnicodeCategory ** entering for 'a'
** InternalGetCategoryValue ** BITSHIFT: ch 97 ch >> 8 0 leads to s_pCategoryLevel1Index[ch >> 8] 4352
** InternalGetCategoryValue ** BITSHIFT: ch 97 ((ch >> 4) & 0x000f) 6 leads to s_pCategoryLevel1Index[index + ((ch >> 4) & 0x000f)] 4883
** InternalGetCategoryValue ** BITWISE AND: ch 97 ch & 0x000f 1 leads to pBytePtr[(ch & 0x000f)] 19
** CharInCategory ** begin: ch a i 3 end 13 chcat LowercaseLetter
So I don't think the bitshifts/ands are the problem (I could be wrong) but instead the unsafe code that "gently massages" an array of ushorts (that I have no idea where they're coming from, I grepped and I can't find the source file for that field) - this is hairy pointer arithmetic and it seems like it'd be grabbing the wrong part for a big endian system?
referencesource's version of the file knows this, but it's not using fixed; I'm not sure how this would map out to the current version. Any ideas, @EgorBo ? (Oh hey, and it was patched by Neale...)
NattyNarwhal
on 23 Jul 2018
Relevant commits:
NattyNarwhal
on 23 Jul 2018
@EgorBo could you revert charunicodeinfo.cs for now?
marek-safar
on 23 Jul 2018
@marek-safar already did, it's being compiled on the ppc64be machine now
EgorBo
on 23 Jul 2018
@NattyNarwhal yeah, looks like changing Char back to
../referencesource/mscorlib/system/char.cs
../referencesource/mscorlib/system/charenumerator.cs
../referencesource/mscorlib/system/globalization/charunicodeinfo.cs
and
MONO_CORLIB_COUNTER=24 in configure.ac
helped.
I'll try to localize and upstream the BE fixes
EgorBo
on 23 Jul 2018
Nice. My suspicion is the decimal issue that I had after the regex issue (if I pressed on further, which Jenkins didn't do) is a parsing issue also possibly caused by this; though we'll find out.
NattyNarwhal
on 23 Jul 2018
@NattyNarwhal it looks like it's not a decimal issue, the mcs error happens here: https://github.com/mono/corefx/blob/master/src/Common/src/System/Security/Cryptography/AsnWriter.cs#L1217 (floatingTicks is too big)
EgorBo
on 23 Jul 2018
Hmmm, not sure why it happens there then, if Decimal is in working order, and we've had that from Cryptography for a while? Again, I mention it could be another parsing issue caused by CharUnicodeInfo, but I'm still unsure about that?
NattyNarwhal
on 23 Jul 2018
That could easily be the case as I made some changes to the parsing of patterns with Unicode surrogate pairs (which we don’t support): https://github.com/dotnet/corefx/commit/2f259de0bc566f4ef72e486133f71210416e440a
From: Calvin notifications@github.com
Sent: Monday, July 23, 2018 5:18:23 PM
To: mono/mono
Cc: Viktor Hofer; Mention
Subject: Re: [mono/mono] BCL bootstrap regressions introduced by CoreFX System import on AIX (regex, decimal) (#9684)
Hmmm, not sure why it happens there then, if Decimal is in working order, and we've had that from Cryptography for a while? Again, I mention it could be another parsing issue caused by CharUnicodeInfo, but I'm still unsure about that?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmono%2Fmono%2Fissues%2F9684%23issuecomment-407094981&data=02%7C01%7Cviktor.hofer%40microsoft.com%7Cf72753e78d67447eeac408d5f0af8840%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636679559076560684&sdata=W55u7s3Lox0xToC22%2BSjPUzgkfnDwqJXBZLGjfpTnOo%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHEbqz6DgKrtarVk3USJx2QglEGlfh3-ks5uJek_gaJpZM4VaAPC&data=02%7C01%7Cviktor.hofer%40microsoft.com%7Cf72753e78d67447eeac408d5f0af8840%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636679559076570688&sdata=TI7thROTenTMV3aYjEM7JqoUIcO7NPWwd1O53pSWikI%3D&reserved=0.
ViktorHofer
on 23 Jul 2018
@ViktorHofer this is with 2.1 branch of CoreFX
marek-safar
on 23 Jul 2018
I see… Then it must be something else. I also did some Regex changes for 2.1 but nothing that sounds familiar to what you are facing here.
From: Marek Safar notifications@github.com
Sent: Monday, July 23, 2018 5:30:37 PM
To: mono/mono
Cc: Viktor Hofer; Mention
Subject: Re: [mono/mono] BCL bootstrap regressions introduced by CoreFX System import on AIX (regex, decimal) (#9684)
@ViktorHoferhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FViktorHofer&data=02%7C01%7Cviktor.hofer%40microsoft.com%7C8c1ab6f642b6418572c908d5f0b13def%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636679566391436630&sdata=GbBx5kYC019SmORshEHjlhXtxn%2By5jc9MMvRPxcIkfs%3D&reserved=0 this is with 2.1 branch of CoreFX
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmono%2Fmono%2Fissues%2F9684%23issuecomment-407099224&data=02%7C01%7Cviktor.hofer%40microsoft.com%7C8c1ab6f642b6418572c908d5f0b13def%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636679566391436630&sdata=d%2BlgW5k3dccZ8xESsubSUU7Y9MRX2CPaIf%2BJovo870I%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHEbq72t3rtz10BeCum-hZk2F6z_chBXks5uJewdgaJpZM4VaAPC&data=02%7C01%7Cviktor.hofer%40microsoft.com%7C8c1ab6f642b6418572c908d5f0b13def%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636679566391436630&sdata=hJ50OI4jFQhVaqA8bAfKjtL857QCL2iaQH2rqbuShac%3D&reserved=0.
ViktorHofer
on 23 Jul 2018
@ViktorHofer @marek-safar @NattyNarwhal yeah it's not a Regex related issue, there are two issues:
1) CharUnicodeInfo.cs in .NET Core is not BE friendly (I'll try to upstream the patches)
2) a small mcs related issue (I am investigating) here: https://github.com/mono/corefx/blob/master/src/Common/src/System/Security/Cryptography/AsnWriter.cs#L1217 (for some reason this operation involves decimal.ToInt32() but it should not, I've temporarily commented out this line and the rest of the build is completely fine
EgorBo
on 23 Jul 2018
While the big endian issue in CharUnicodeInfo is mitigated for now, there's another issue (in mcs or CoreFX System?) that blocks a compile of System.Security and thus, the runtime:
make[7]: Entering directory '/home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/mcs/class/System.Security'
MONO_PATH="./../../class/lib/monolite-unix/1051700026:./../../class/lib/monolite-unix/1051700026/Facades:$MONO_PATH" /home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/runtime/mono-wrapper --debug ./../../build/gensources.exe --strict "../../build/deps/_basic__System.Security.dll.sources" "System.Security.dll" "" "basic"
make all-local
make[8]: Entering directory '/home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/mcs/class/System.Security'
cat <./../../build/deps/_basic__System.Security.dll.sources >../../build/deps/_basic__System.Security.dll.response
MONO_PATH="./../../class/lib/monolite-unix/1051700026:./../../class/lib/monolite-unix/1051700026/Facades:$MONO_PATH" /home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/runtime/mono-wrapper ./../../class/lib/monolite-unix/1051700026/mcs.exe /codepage:65001 /nologo /noconfig /deterministic -d:NET_4_0 -d:NET_4_5 -d:MONO -d:WIN_PLATFORM -d:BOOTSTRAP_BASIC -nowarn:1699 -nostdlib -optimize /features:peverify-compat /langversion:latest -unsafe -nowarn:414,618 -d:SECURITY_DEP -r:./../../../external/binary-reference-assemblies/v4.7.1/System.Numerics.dll -r:./../../../external/binary-reference-assemblies/v4.7.1/System.Core.dll -r:./../../class/lib/basic/Mono.Security.dll -r:./../../class/lib/basic/System.dll -r:./../../class/lib/basic/System.Xml.dll -r:./../../class/lib/basic/mscorlib.dll /keyfile:../msfinal.pub -target:library -out:../../class/lib/basic/System.Security.dll @./../../build/deps/_basic__System.Security.dll.response
../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnReader.cs(294,21): warning CS0219: The variable `tag' is assigned but its value is never used
../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnReader.cs(317,21): warning CS0219: The variable `tag' is assigned but its value is never used
Unhandled Exception:
Mono.CSharp.InternalErrorException: ../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnWriter.cs(1171,22): System.Security.Cryptography.Asn1.AsnWriter.WriteGeneralizedTimeCore(System.Security.Cryptography.Asn1.Asn1Tag, System.DateTimeOffset, bool) ---> Mono.CSharp.InternalErrorException: ../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnWriter.cs(1172,9): ---> System.OverflowException: Value was either too large or too small for an Int32.
at System.Decimal.ToInt32 (System.Decimal d) [0x00035] in <f753a15472ea46d3a1cdfc28ed2b0394>:0
at System.Decimal.op_Explicit (System.Decimal value) [0x00000] in <f753a15472ea46d3a1cdfc28ed2b0394>:0
at Mono.CSharp.DecimalConstant.Emit (Mono.CSharp.EmitContext ec) [0x0006d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Argument.Emit (Mono.CSharp.EmitContext ec) [0x0002d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Arguments.Emit (Mono.CSharp.EmitContext ec, System.Boolean dup_args, System.Boolean prepareAwait) [0x0004c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.CallEmitter.EmitPredefined (Mono.CSharp.EmitContext ec, Mono.CSharp.MethodSpec method, Mono.CSharp.Arguments Arguments, System.Boolean statement, System.Nullable`1[T] loc) [0x0012f] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.CallEmitter.Emit (Mono.CSharp.EmitContext ec, Mono.CSharp.MethodSpec method, Mono.CSharp.Arguments Arguments, Mono.CSharp.Location loc) [0x0000c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.UserOperatorCall.Emit (Mono.CSharp.EmitContext ec) [0x00008] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.VariableReference.EmitAssign (Mono.CSharp.EmitContext ec, Mono.CSharp.Expression source, System.Boolean leave_copy, System.Boolean prepare_for_load) [0x00070] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Assign.Emit (Mono.CSharp.EmitContext ec, System.Boolean is_statement) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Assign.EmitStatement (Mono.CSharp.EmitContext ec) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.StatementExpression.DoEmit (Mono.CSharp.EmitContext ec) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.If.DoEmit (Mono.CSharp.EmitContext ec) [0x00055] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.If.DoEmit (Mono.CSharp.EmitContext ec) [0x00055] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ParametersBlock.Emit (Mono.CSharp.EmitContext ec) [0x00030] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ToplevelBlock.Emit (Mono.CSharp.EmitContext ec) [0x00038] in <c3cc7360f1974efe94a8df2db7373648>:0
--- End of inner exception stack trace ---
at Mono.CSharp.ToplevelBlock.Emit (Mono.CSharp.EmitContext ec) [0x000bb] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.MethodData.Emit (Mono.CSharp.TypeDefinition parent) [0x0008c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.MethodOrOperator.Emit () [0x001e6] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Method.Emit () [0x00104] in <c3cc7360f1974efe94a8df2db7373648>:0
--- End of inner exception stack trace ---
at Mono.CSharp.Method.Emit () [0x00114] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeDefinition.Emit () [0x0027b] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ClassOrStruct.Emit () [0x0001d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Class.Emit () [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeDefinition.EmitContainer () [0x0000c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.NamespaceContainer.EmitContainer () [0x00007] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.NamespaceContainer.EmitContainer () [0x00007] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ModuleContainer.EmitContainer () [0x00085] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.AssemblyDefinition.Emit () [0x0008c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.AssemblyDefinitionStatic.Emit () [0x0010e] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Driver.Compile () [0x002a8] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Driver.Main (System.String[] args) [0x0004c] in <c3cc7360f1974efe94a8df2db7373648>:0
[ERROR] FATAL UNHANDLED EXCEPTION: Mono.CSharp.InternalErrorException: ../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnWriter.cs(1171,22): System.Security.Cryptography.Asn1.AsnWriter.WriteGeneralizedTimeCore(System.Security.Cryptography.Asn1.Asn1Tag, System.DateTimeOffset, bool) ---> Mono.CSharp.InternalErrorException: ../../../external/corefx/src/Common/src/System/Security/Cryptography/AsnWriter.cs(1172,9): ---> System.OverflowException: Value was either too large or too small for an Int32.
at System.Decimal.ToInt32 (System.Decimal d) [0x00035] in <f753a15472ea46d3a1cdfc28ed2b0394>:0
at System.Decimal.op_Explicit (System.Decimal value) [0x00000] in <f753a15472ea46d3a1cdfc28ed2b0394>:0
at Mono.CSharp.DecimalConstant.Emit (Mono.CSharp.EmitContext ec) [0x0006d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Argument.Emit (Mono.CSharp.EmitContext ec) [0x0002d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Arguments.Emit (Mono.CSharp.EmitContext ec, System.Boolean dup_args, System.Boolean prepareAwait) [0x0004c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.CallEmitter.EmitPredefined (Mono.CSharp.EmitContext ec, Mono.CSharp.MethodSpec method, Mono.CSharp.Arguments Arguments, System.Boolean statement, System.Nullable`1[T] loc) [0x0012f] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.CallEmitter.Emit (Mono.CSharp.EmitContext ec, Mono.CSharp.MethodSpec method, Mono.CSharp.Arguments Arguments, Mono.CSharp.Location loc) [0x0000c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.UserOperatorCall.Emit (Mono.CSharp.EmitContext ec) [0x00008] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.VariableReference.EmitAssign (Mono.CSharp.EmitContext ec, Mono.CSharp.Expression source, System.Boolean leave_copy, System.Boolean prepare_for_load) [0x00070] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Assign.Emit (Mono.CSharp.EmitContext ec, System.Boolean is_statement) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Assign.EmitStatement (Mono.CSharp.EmitContext ec) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.StatementExpression.DoEmit (Mono.CSharp.EmitContext ec) [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.If.DoEmit (Mono.CSharp.EmitContext ec) [0x00055] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.If.DoEmit (Mono.CSharp.EmitContext ec) [0x00055] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Statement.Emit (Mono.CSharp.EmitContext ec) [0x0000d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Block.DoEmit (Mono.CSharp.EmitContext ec) [0x00010] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ExplicitBlock.Emit (Mono.CSharp.EmitContext ec) [0x00042] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ParametersBlock.Emit (Mono.CSharp.EmitContext ec) [0x00030] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ToplevelBlock.Emit (Mono.CSharp.EmitContext ec) [0x00038] in <c3cc7360f1974efe94a8df2db7373648>:0
--- End of inner exception stack trace ---
at Mono.CSharp.ToplevelBlock.Emit (Mono.CSharp.EmitContext ec) [0x000bb] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.MethodData.Emit (Mono.CSharp.TypeDefinition parent) [0x0008c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.MethodOrOperator.Emit () [0x001e6] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Method.Emit () [0x00104] in <c3cc7360f1974efe94a8df2db7373648>:0
--- End of inner exception stack trace ---
at Mono.CSharp.Method.Emit () [0x00114] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeDefinition.Emit () [0x0027b] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ClassOrStruct.Emit () [0x0001d] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Class.Emit () [0x00000] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeDefinition.EmitContainer () [0x0000c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.NamespaceContainer.EmitContainer () [0x00007] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.NamespaceContainer.EmitContainer () [0x00007] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.TypeContainer.EmitContainer () [0x00018] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.ModuleContainer.EmitContainer () [0x00085] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.AssemblyDefinition.Emit () [0x0008c] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.AssemblyDefinitionStatic.Emit () [0x0010e] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Driver.Compile () [0x002a8] in <c3cc7360f1974efe94a8df2db7373648>:0
at Mono.CSharp.Driver.Main (System.String[] args) [0x0004c] in <c3cc7360f1974efe94a8df2db7373648>:0
../../build/library.make:352: recipe for target '../../class/lib/basic/System.Security.dll' failed
make[8]: *** [../../class/lib/basic/System.Security.dll] Error 1
make[8]: Leaving directory '/home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/mcs/class/System.Security'
../../build/rules.make:211: recipe for target 'do-all' failed
make[7]: *** [do-all] Error 2
make[7]: Leaving directory '/home/jenkins/workspace/test-mono-mainline-community/label/aix-ppc64/mcs/class/System.Security'
possible repro case, need to do the whole dance of compiling basic profile with compiling with amd64 Roslyn again:
using System;
class MaybeDecimalRepro {
public static void Main() {
DateTimeOffset normalized = value.ToUniversalTime();
long floatingTicks = normalized.Ticks % TimeSpan.TicksPerSecond;
decimal decimalTicks = floatingTicks;
decimalTicks /= TimeSpan.TicksPerSecond;
Console.WriteLine(decimalTicks);
}
}
It hasn't finished building yet, but studying the function in mcs that popped up near the top of the call stack had this which I'm pretty sure translates to an op_Explicit for Decimal?
NattyNarwhal
on 25 Jul 2018
@NattyNarwhal
yeah, I mentioned that issue (sorry, the issue was closed automatically)
Can be fixed by
decimalTicks /= TimeSpan.TicksPerSecond;
replaced by
decimalTicks = decimal.Divide(decimalTicks, new decimal(TimeSpan.TicksPerSecond));
For some reason MCS picks a wrong overload for the divide operation (casts TimeSpan.TicksPerSecond to int).
EgorBo
on 25 Jul 2018
@EgorBo It seems that patch is working fine for doing a manual build, thanks! Unfortunately, that doesn't work out for Jenkins, and I don't want to fall of CI. I wonder if a workaround ifdefing on MonoCS would work until it's fixed....
NattyNarwhal
on 25 Jul 2018
Ok, it works, until I hit an error in Cecil, that happens after compilation of the .NET 4.x profile of mscorlib; (I think I can reproduce this on amd64, so I'll file another issue) so I can try to press on. If I do that, I hit another System.Security error?
MCS [net_4_x-unix] System.Security.dll
../../../external/corefx/src/Common/src/System/Security/Cryptography/Asn1V2.Serializer.cs(561,13): error CS0433: The imported type `System.Diagnostics.Debug' is defined multiple times
/home/calvin/mono-build/mono/mcs/class/System.Security/./../../class/lib/net_4_x-unix/mscorlib.dll (Location of the symbol related to previous error)
/home/calvin/mono-build/mono/mcs/class/System.Security/./../../class/lib/net_4_x-unix/System.dll (Location of the symbol related to previous error)
../../../external/corefx/src/System.Security.Cryptography.Pkcs/src/Internal/Cryptography/Pal/AnyOS/ManagedPal.Encrypt.cs(82,55): warning CS0436: The type `Internal.Cryptography.Helpers' conflicts with the imported type of same name'. Ignoring the imported type definition
../../../external/corefx/src/System.Security.Cryptography.Pkcs/src/Internal/Cryptography/Helpers.cs(20,35): (Location of the symbol related to previous warning)
/home/calvin/mono-build/mono/mcs/class/System.Security/./../../class/lib/net_4_x-unix/mscorlib.dll (Location of the symbol related to previous warning)
Compilation failed: 1 error(s), 1 warnings
../../build/library.make:352: recipe for target '../../class/lib/net_4_x-unix/System.Security.dll' failed
OK, I can reproduce that System.Security error on amd64 too, not just the Cecil error, filing a new issue for that as well. (as it's not aix/big endian related)
NattyNarwhal
on 25 Jul 2018
Related issues
grendello
·
33Comments
dbelcher
·
35Comments
danroth27
·
39Comments
filipnavara
·
29Comments
bhaskar-infotmt
·
31Comments