Xamarin-macios: NullReferenceException when using System.Linq.Queryable (Linker issue)

[slang25]

Description

When using System.Linq.Queryable with an expression, like this:

var list = new List<string> {"hello hello" };
output.Text = list.AsQueryable().GroupBy(x => x).FirstOrDefault()?.FirstOrDefault();

with "Link Framework SDKs Only" enabled and running on a physical iPhone it throws a NullReferenceException. However, when running on the simulator I do not see this issue.

Steps to Reproduce

1. git clone https://github.com/slang25/XamLinkerBug.git
2. Run in Debug (or Release) from Visual Studio for Mac on a physical iPhone
3. Feel the NullReferenceException

Expected Behavior

No NullReferenceException.

Actual Behavior

NullReferenceException (see here for stacktrace)

Environment

=== Visual Studio Community 2019 (Preview) for Mac ===

Version 8.1 Preview (8.1 build 2734)
Installation UUID: d0f72ff2-1b2f-4611-a068-e1411a4b68e5
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 5.6.0.25 (d16-0 / 50f75273)

    Package version: 518010028

=== Mono Framework MDK ===

Runtime:
    Mono 5.18.1.28 (2018-08/223ea7ef92e) (64-bit)
    Package version: 518010028

=== NuGet ===

Version: 5.0.2.5988

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    3.0.0-preview3-27503-5
    3.0.0-preview-27122-01
    2.2.5
    2.2.4
    2.2.3
    2.2.1
    2.2.0
    2.1.9
    2.1.6
    2.1.4
    2.1.2
    2.1.1
SDK: /usr/local/share/dotnet/sdk/3.0.100-preview3-010431/Sdks
SDK Versions:
    3.0.100-preview3-010431
    3.0.100-preview-009812
    2.2.300
    2.2.203
    2.2.105
    2.2.102
    2.2.101
    2.2.100
    2.1.505
    2.1.500
    2.1.402
    2.1.400
    2.1.302
    2.1.301
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.18.1/lib/mono/msbuild/Current/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.6.10
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 10.2.1 (14490.122)
Build 10E1001

=== Xamarin.Android ===

Version: 9.3.0.22 (Visual Studio Community)
Commit: HEAD/8e7764fdf
Android SDK: /Users/stuart/Library/Developer/Xamarin/android-sdk-macosx
    Supported Android versions:
        7.0 (API level 24)
        7.1 (API level 25)
        8.0 (API level 26)
        8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 28.0.1
SDK Build Tools Version: 27.0.3

Build Information: 
Mono: mono/mono/2018-08@3cb36842fc4
Java.Interop: xamarin/java.interop/d16-1@5ddc3e3
LibZipSharp: grendello/LibZipSharp/d16-1@44de300
LibZip: nih-at/libzip/rel-1-5-1@b95cf3f
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.27.1@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-1@acabd26

=== Microsoft Mobile OpenJDK ===

Java SDK: /Users/stuart/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android Device Manager ===

Version: 1.2.0.44
Hash: aac645b
Branch: remotes/origin/d16-1
Build date: 2019-05-29 19:55:24 UTC

=== Xamarin.Mac ===

Version: 5.10.0.153 (Visual Studio Community)
Hash: 750a8798
Branch: d16-1-artifacts
Build date: 2019-04-30 15:17:54-0400

=== Xamarin.iOS ===

Version: 12.10.0.153 (Visual Studio Community)
Hash: 750a8798
Branch: d16-1-artifacts
Build date: 2019-04-30 15:17:54-0400

=== Xamarin Designer ===

Version: 16.1.0.464
Hash: 66bb7b43f
Branch: remotes/origin/d16-1-new-document-model
Build date: 2019-06-07 07:10:32 UTC

=== Xamarin Inspector ===

Version: 1.4.3
Hash: db27525
Branch: 1.4-release
Build date: Mon, 09 Jul 2018 21:20:18 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 801002734
Git revision: 6c95d6d393b9950dc48ddb7e845a6570071ae133
Build date: 2019-06-07 20:04:04+00
Build branch: release-8.1
Xamarin extensions: 0ac46762465f7fde32f6e8d8710f68c77e819e9f

=== Operating System ===

Mac OS X 10.14.4
Darwin 18.5.0 Darwin Kernel Version 18.5.0
    Mon Mar 11 20:40:32 PDT 2019
    root:xnu-4903.251.3~3/RELEASE_X86_64 x86_64

=== Enabled user installed extensions ===

Xamarin.Forms HotReload extension 1.3.2
HotReloading 0.3.1

Build Logs

https://gist.github.com/slang25/0681b9110fabe5560c9762111bafa8fa

Example Project (If Possible)

Minimal repro here, along with details:
https://github.com/slang25/XamLinkerBug

[spouliot]

This is due to reflection usage inside corefx

csharp MethodInfo mi = s_seqMethods[name].FirstOrDefault(m => ArgsMatch(m, args, typeArgs));

and mi is null then the linker is enabled.

[spouliot]

However, when running on the simulator I do not see this issue.

By default the simulator uses Don't link (since it results in much faster builds)

Workaround

Inside your Main.cs file add the following line

[assembly: Preserve (typeof (Enumerable), AllMembers = true)]

between your using clauses and your namespace

[slang25]

That makes sense, however my understanding was that Link Framework SDKs Only was supposed to be "safe", I think that's my mistake.

I would have hoped for a compiler warner to indicate that I'm using a method that relies on reflection and let me know I should be prepared to preserve stuff. Would this be a feasible feature to request?

Now that the linker is becoming part of .NET Core 3, more people are going to be running into these sorts of issues.

[spouliot]

Link Framework SDKs Only was supposed to be "safe"

Yes, that's why there's a bug label on this issue :)
and this will be fixed in a future release - so the workaround is not needed

compiler warner to indicate that I'm using a method that relies on reflection

That's not really possible. You would have too many false-positives (for an easy implementation) so that would not help you. Also reflection is dynamic so it's easy to have it work (when linking) for 90+% of the cases and still miss a few (i.e. so an audited list would still not be perfect).

more people are going to be running into these sorts of issues.

That;'s a good thing :) since we're moving to share the same code (for BCL) and more people means we'll spot any breakages (usually due to code changes not tested w/the linker - which should not be so common anymore) earlier - like in previews (if not in already automated tests)

[slang25]

Ah awesome, please ignore my comment about compiler warnings etc... I didn't appreciate this was being treated as a bug 😃

I look forward to more people getting the benefits of the linker, and selfishly the extra hardening that will happen as a result.

[ronwatson71]

We found the problem with ThenByDecending using this simple example in ViewDidLoad.

        Test[] data = {new Test() { field1 = "B", quantity = 100  },
            new Test() { field1 = "A", quantity = 12 },
            new Test() { field1 = "C", quantity = 4 },
            new Test() { field1 = "E", quantity = 32 },
            new Test() { field1 = "F", quantity = 200 },
        };

        var qry = data.AsQueryable().OrderBy(o => o.field1).ThenByDescending(o => o.quantity);

        var lst = qry.ToList();
        return;

The problem does not appear on the simulator.

[slang25]

The workaround here should work for you: https://github.com/xamarin/xamarin-macios/issues/6346#issuecomment-503272500