Xamarin-macios: ARWorldTrackingConfiguration.SupportsFrameSemantics() returns false on compatible device

[andrew-benson]

Steps to Reproduce

1. Run TestingXamarinAPI project on a device that can support ARKit 3 feature - person segmentation. I've been using iPhone 11.
2. Read output from SupportsFrameSemantics() method - output available from view and diagnostic console output
3. Optionally use the xcode project file included which is more or less the same to see correct output

Expected Behavior

Expect boolean values returned by SupportsFrameSemantics to be 'true', as per xcode project.

Actual Behavior

Debug diagnostics output:

Device Model: iPhone12,1
Device Version: 13.5.1
Person segmentation supported: False
Person segmentation with depth supported: False

Environment

=== Visual Studio Community 2019 for Mac ===

Version 8.6.8 (build 2)
Installation UUID: 783eef36-3f21-403f-a1ec-a1e33ef5d215
    GTK+ 2.24.23 (Raleigh theme)
    Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

    Package version: 610000106

=== Mono Framework MDK ===

Runtime:
    Mono 6.10.0.106 (2019-12/77769615db1) (64-bit)
    Package version: 610000106

=== Roslyn (Language Service) ===

3.6.0-3.20210.9+4eafdcb1bcbd8d3573f2ba6065e56d9b9ce4f8a3

=== NuGet ===

Version: 5.6.0.6591

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/3.1.302/Sdks
SDK Versions:
    3.1.302
    3.1.301
    3.1.200
    3.1.102
    3.1.101
    3.1.100
    3.0.101
    3.0.100
    2.2.300
    2.2.104
    2.1.505
    2.1.504
    2.1.302
    2.1.301
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
    3.1.6
    3.1.5
    3.1.2
    3.1.1
    3.1.0
    3.0.1
    3.0.0
    2.2.5
    2.2.2
    2.1.20
    2.1.19
    2.1.17
    2.1.16
    2.1.15
    2.1.14
    2.1.13
    2.1.9
    2.1.8
    2.1.2
    2.1.1

=== Xamarin.Profiler ===

Version: 1.6.13.11
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 11.6 (16141)
Build 11E708

=== Xamarin.Mac ===

Version: 6.18.3.2 (Visual Studio Community)
Hash: ce0cc74a3
Branch: d16-6-xcode11.6
Build date: 2020-07-16 18:15:27-0400

=== Xamarin.iOS ===

Version: 13.18.3.2 (Visual Studio Community)
Hash: ce0cc74a3
Branch: d16-6-xcode11.6
Build date: 2020-07-16 18:15:27-0400

=== Xamarin Designer ===

Version: 16.6.0.329
Hash: d4f8bcd13
Branch: remotes/origin/d16-6
Build date: 2020-04-24 02:16:02 UTC

=== Xamarin.Android ===

Version: 10.3.1.4 (Visual Studio Community)
Commit: xamarin-android/d16-6/3a10de9
Android SDK: /Users/andrewb/Library/Developer/Xamarin/android-sdk-macosx
    Supported Android versions:
        7.1 (API level 25)
        8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.4
SDK Build Tools Version: 28.0.3

Build Information: 
Mono: 165f4b0
Java.Interop: xamarin/java.interop/d16-6@2cab35c
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.31.1@49232bc
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-6@bfb66f3

=== Microsoft OpenJDK for Mobile ===

Java SDK: /Users/andrewb/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.6.0.50
Hash: 5901879
Branch: remotes/origin/d16-6
Build date: 2020-06-10 22:42:50 UTC

=== Android Device Manager ===

Version: 16.6.0.96
Hash: 6e8b80b
Branch: remotes/origin/d16-6
Build date: 2020-06-10 22:43:28 UTC

=== Build Information ===

Release ID: 806080002
Git revision: e999e2934e8c771fdf6494b80361dacef9427565
Build date: 2020-07-24 15:17:52-04
Build branch: release-8.6
Xamarin extensions: e999e2934e8c771fdf6494b80361dacef9427565

=== Operating System ===

Mac OS X 10.15.5
Darwin 19.5.0 Darwin Kernel Version 19.5.0
    Tue May 26 20:41:44 PDT 2020
    root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64

Build Logs

Launched application 'com.AndrewBenson.TestingAPI' on 'Andy???s iPhone' with pid 536
2020-08-04 21:54:59.813 TestingXamarinAPI[536:67633] Xamarin.iOS: IDE Port: 10000 Transport: USB
2020-08-04 21:54:59.846 TestingXamarinAPI[536:67633] Xamarin.iOS: Successfully received USB connection from the IDE on port 10000, fd: 4
2020-08-04 21:54:59.847 TestingXamarinAPI[536:67633] Xamarin.iOS: Processing: 'start debugger: sdb'
2020-08-04 21:54:59.848 TestingXamarinAPI[536:67633] Xamarin.iOS: Successfully received USB connection from the IDE on port 10000, fd: 5
2020-08-04 21:54:59.848 TestingXamarinAPI[536:67628] Xamarin.iOS: Debugger loaded with custom transport (fd: 4)
2020-08-04 21:54:59.848 TestingXamarinAPI[536:67633] Xamarin.iOS: Processing: 'connect output'
2020-08-04 21:54:59.849 TestingXamarinAPI[536:67633] Xamarin.iOS: Successfully received USB connection from the IDE on port 10000, fd: 6
2020-08-04 21:54:59.849 TestingXamarinAPI[536:67633] Xamarin.iOS: Processing: 'start profiler: no'
2020-08-04 21:54:59.849 TestingXamarinAPI[536:67628] Xamarin.iOS: Profiler not loaded (disabled)
Loaded assembly: /private/var/containers/Bundle/Application/3460CF2C-31FB-4B6B-9983-57F695605A83/TestingXamarinAPI.app/System.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/3460CF2C-31FB-4B6B-9983-57F695605A83/TestingXamarinAPI.app/Xamarin.iOS.dll [External]
2020-08-04 21:54:59.914 TestingXamarinAPI[536:67628] Xamarin.iOS: Skipping assembly registration for Xamarin.iOS since it's not needed (dynamic registration is not supported)
2020-08-04 21:54:59.916 TestingXamarinAPI[536:67628] Xamarin.iOS: Skipping assembly registration for Xamarin.Essentials since it's not needed (dynamic registration is not supported)
2020-08-04 21:54:59.917 TestingXamarinAPI[536:67628] Xamarin.iOS: Skipping assembly registration for TestingXamarinAPI since it's not needed (dynamic registration is not supported)
Thread started: #2
Loaded assembly: /private/var/containers/Bundle/Application/3460CF2C-31FB-4B6B-9983-57F695605A83/TestingXamarinAPI.app/System.Core.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/3460CF2C-31FB-4B6B-9983-57F695605A83/TestingXamarinAPI.app/Xamarin.Essentials.dll [External]
Loaded assembly: /private/var/containers/Bundle/Application/3460CF2C-31FB-4B6B-9983-57F695605A83/TestingXamarinAPI.app/TestingXamarinAPI.exe
Device Model: iPhone12,1
Device Version: 13.5.1
Person segmentation supported: False
Person segmentation with depth supported: False

Example Project (If Possible)

TestingXamarinAPI.zip

Optionally use this XCode project to see the SupportsFrameSemantics method return true
TestingXcodeAPI.zip

I think the problem might be related to the fact the base class ARConfiguration is being called (declaration points to it) even though the Apple documentation states not to do this. This may not be the problem though I'm speculating.
Apple Docs for SupportsFrameSemantics

[whitneyschmidt]

@andrew-benson Good catch - this is a bug in the SDK. SupportsFrameSemantics should never be called directly from ARConfiguration, as you pointed out. Thank you for opening an issue!

I've opened a PR with a fix, in the meantime as a workaround you can add the following code to ViewController:

[DllImport ("/usr/lib/libobjc.dylib", EntryPoint = "objc_msgSend")]
[MethodImpl (MethodImplOptions.NoInlining)]
public static extern bool bool_objc_msgSend_Int64 (IntPtr receiver, IntPtr selector, long arg1);

[BindingImpl (BindingImplOptions.GeneratedCode | BindingImplOptions.Optimizable)]
public static bool SupportsFrameSemantics (Class klass, ARFrameSemantics frameSemantics)
{
        return bool_objc_msgSend_Int64 (klass.Handle, Selector.GetHandle ("supportsFrameSemantics:"), (Int64)frameSemantics);
}

And calls to SupportsFrameSemantics will look something like:

bool isSupported = SupportsFrameSemantics (new Class(typeof (ARWorldTrackingConfiguration)), ARFrameSemantics.PersonSegmentation);

You can pass any other subclass of ARConfiguration into SupportsFrameSemantics the same way as above :)

Due to a design limitation, you'll need to use either new Class (typeof (...)) or create an instance of the ARConfiguration subclass that you want to use, even though SupportsFrameSemantics is a static method and you wouldn't expect to need an instance to invoke it.

[andrew-benson]

@andrew-benson Good catch - this is a bug in the SDK. SupportsFrameSemantics should never be called directly from ARConfiguration, as you pointed out. Thank you for opening an issue!

I've opened a PR with a fix, in the meantime as a workaround you can add the following code to ViewController:

[DllImport ("/usr/lib/libobjc.dylib", EntryPoint = "objc_msgSend")]
[MethodImpl (MethodImplOptions.NoInlining)]
public static extern bool bool_objc_msgSend_Int64 (IntPtr receiver, IntPtr selector, long arg1);

[BindingImpl (BindingImplOptions.GeneratedCode | BindingImplOptions.Optimizable)]
public static bool SupportsFrameSemantics (Class klass, ARFrameSemantics frameSemantics)
{
        return bool_objc_msgSend_Int64 (klass.Handle, Selector.GetHandle ("supportsFrameSemantics:"), (Int64)frameSemantics);
}

And calls to SupportsFrameSemantics will look something like:

bool isSupported = SupportsFrameSemantics (new Class(typeof (ARWorldTrackingConfiguration)), ARFrameSemantics.PersonSegmentation);

You can pass any other subclass of ARConfiguration into SupportsFrameSemantics the same way as above :)

Due to a design limitation, you'll need to use either new Class (typeof (...)) or create an instance of the ARConfiguration subclass that you want to use, even though SupportsFrameSemantics is a static method and you wouldn't expect to need an instance to invoke it.

Thank you for this workaround, works perfectly. And no problem glad I could help :)