Imagesharp: Crash when running in AOT-only mode (Release mode) in SixLabors.ImageSharp.PixelAccessor

[Lapinou42]

Prerequisites

• [x] I have written a descriptive issue title
• [X] I have verified that I am running the latest version of ImageSharp
• [X] I have verified if the problem exist in both DEBUG and RELEASE mode
• [X] I have searched open and closed issues to ensure it has not already been reported

Description

I have a crash on release mode on iOS (not Android) and when "enable debugging" is enable on Release mode (not cool), the app doesn't crash. It happens when I call this method:
public static IImage ToImageRgba32(this UIImage image)

{
        byte[] bytes = image.AsJPEG(1.0f).ToArray();
        var rgba32image = SixLabors.ImageSharp.Image.Load(bytes);
        return new Rgba32Image(rgba32image);
 }

Here is the stack trace :

Attempting to JIT compile method 'System.Runtime.CompilerServices.Unsafe:Add (byte&,int)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.

1
JpegEncoderCore.ToYCbCr[TPixel] (SixLabors.ImageSharp.PixelAccessor1[TPixel] pixels, SixLabors.ImageSharp.Formats.Jpeg.GolangPort.Components.Encoder.RgbToYCbCrTables* tables, System.Int32 x, System.Int32 y, SixLabors.ImageSharp.Formats.Jpeg.Common.Block8x8F* yBlock, SixLabors.ImageSharp.Formats.Jpeg.Common.Block8x8F* cbBlock, SixLabors.ImageSharp.Formats.Jpeg.Common.Block8x8F* crBlock, SixLabors.ImageSharp.PixelArea1[TPixel] rgbBytes)
2
(wrapper unknown) System.Object:gsharedvt_in ()
3
(wrapper unknown) System.Object:gsharedvt_out ()
4
JpegEncoderCore.Encode420[TPixel] (SixLabors.ImageSharp.PixelAccessor`1[TPixel] pixels)
5
(wrapper unknown) System.Object:gsharedvt_in ()
6
(wrapper unknown) System.Object:gsharedvt_out ()

System Configuration

• ImageSharp version: 1.0.0-beta0002
• Other ImageSharp packages and versions:
  => Sixlabors.ImageSharp.Drawing : 1.0.0-beta0002
• Environment (Operating system, version and so on): Mac OS 10.2.6
• Additional information:
  === Visual Studio Enterprise 2017 for Mac ===

Version 7.2.2 (build 11)
Runtime:
Mono 5.4.1.6 (2017-06/1f4613aa1ac) (64-bit)
GTK+ 2.24.23 (Raleigh theme)

Package version: 504010006

=== NuGet ===

Version: 4.3.1.4445

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
2.0.0
2.0.0-preview2-25407-01
1.1.4
1.0.7
SDK: /usr/local/share/dotnet/sdk/2.0.0/Sdks
SDK Versions:
2.0.0
2.0.0-preview2-006497
1.1.4
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.4.1/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.5.6
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Apple Developer Tools ===

Xcode 9.1 (13532)
Build 9B55

=== Xamarin.iOS ===

Version: 11.3.0.47 (Visual Studio Enterprise)
Hash: 51128b8c
Branch: xcode9.1
Build date: 2017-10-31 22:42:13-0400

=== Xamarin.Android ===

Version: 8.0.2.1 (Visual Studio Enterprise)
Android SDK: /Users/Christophe/Library/Android/sdk
Supported Android versions:
4.1 (API level 16)
5.1 (API level 22)
6.0 (API level 23)
7.0 (API level 24)
7.1 (API level 25)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 26.0.1
SDK Build Tools Version: 25.0.3

Java SDK: /usr
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin Inspector ===

Version: 1.3.2
Hash: 461f09a
Branch: 1.3-release
Build date: Tue, 03 Oct 2017 18:26:57 GMT
Client compatibility: 1

=== Xamarin.Mac ===

Version: 3.8.1.0 (Visual Studio Enterprise)

=== Build Information ===

Release ID: 702020011
Git revision: b604c37c5a4a2f0919b45ffbe2aaad9fe040af31
Build date: 2017-11-01 08:31:43-04
Xamarin addins: d57dc14cbd4eb166ee62bab585965ab78d3650bc
Build lane: monodevelop-lion-d15-4

=== Operating System ===

Mac OS X 10.12.6
Darwin 16.7.0 Darwin Kernel Version 16.7.0
Wed Oct 4 00:17:00 PDT 2017
root:xnu-3789.71.6~1/RELEASE_X86_64 x86_64

=== Enabled user installed addins ===

Android Signature Tool 2.1.2
AddinMaker 1.4.1
NuGet Package Explorer 0.2
Redth's Addins 1.0.9
Internet of Things (IoT) development (Preview) 7.1

[Lapinou42]

It seems to be a runtime error when reflection is used.
I got an error of this kind with EF Core. I wrote a linker.xml file for this. Maybe I could solve this issue by using this linker file, but a don't know of to write this.

I tried something like :

<?xml version="1.0" encoding="utf-8"?>
<linker>
    <assembly fullname="SixLabors.ImageSharp">
        <type fullname="SixLabors.ImageSharp.PixelAccessor`1">
        </type>
    </assembly>
</linker>

But it does nothing.

[JimBobSquarePants]

Hi @Lapinou42

We're not using actually reflection in that method.

Could you please attempt to narrow down the issue by creating a simple application that calls only Unsafe.Add(ref byte, int)

If that throws then you will need to raise an issue with Xamarin.

Cheers

James

[Lapinou42]

Hi @JimBobSquarePants ,

Thank you for your reply.
I tried to call this method but it doesn't crash and I have no exception.

Chris.

[JimBobSquarePants]

Thanks. Can you post the code you used to narrow it down please?

[Lapinou42]

Hmmm... Finally, It seems the crash comes from this method :

public static UIImage ToUIImage(this IImage image)
        {
            using (var stream = new MemoryStream())
            {
                var rgba32image = image.ToRgba32Image();

                rgba32image.Save(stream, ImageFormats.Jpeg);

                var bytes = stream.ToArray();

                var imageData = NSData.FromArray(bytes);

                return UIImage.LoadFromData(imageData);
            }
        }

IImage interface :

public interface IImage
    {
        Image<Rgba32> ToRgba32Image();
    }

    public class Rgba32Image : IImage
    {
        private readonly Image<Rgba32> _innerImage;

        public Rgba32Image(Image<Rgba32> innerImage)
        {
            _innerImage = innerImage;
        }

        public Image<Rgba32> ToRgba32Image()
        {
            return _innerImage;
        }
    }

I'm investigating to understand what's going wrong.

[antonfirsov]

@Lapinou42 @JimBobSquarePants
I think the AOT compiler somehow fails to discover, that on this execution path it has to compile Unsafe.Add<T>(ref T, int) with byte as a generic argument.

It's definitely an issue with the Xamarin/mono AOT compiler. I will try to find out where is the official place to post issues for them.

Meanwhile an amateurish workaround idea:
Try doing a dummy Unsafe.Add<byte>(ref byte, int) call somewhere around the entry point of your application. I hope, it will make it to AOT compile the method for the whole app. I'm not familiar with Xamarin, but you probably can achive the same with linker.xml.

[Lapinou42]

@antonfirsov
I think you are right.
Thank you for your investigation. It helped me to understand what's wrong ;)

[JimBobSquarePants]

Thanks @antonfirsov I was thinking the same after a bit of reading.

[Lapinou42]

@antonfirsov

I tried your workaround but it doesn't work.

Sorry for this post that actually doesn't related to ImageSharp.

Btw, do you have post the issue on Xamarin Bugzilla or must I do ?

EDIT: I opened an issue on Xamarin Bugzilla ;)

Kind regards,
Chris.

[antonfirsov]

@Lapinou42 thanks for opening the issue on Xamarin Bugzilla! I also posted a comment, hope it helps.

I suppose, the workaround does not work, because the Unsafe.Add method is AggressiveInlined.

[antonfirsov]

@Lapinou42 I've noticed your latest comment on Bugzilla. Regarding this comment:

We noticed it's very slow on both side (iOS / Android) when we need to convert from Image to native image object.

I'm being curious about your native image pixel format, and how do you implement conversion from Image<Rgba32> to native image objects? :)

[Lapinou42]

@antonfirsov

This is the code I mean :

{
        byte[] bytes = image.AsJPEG(1.0f).ToArray();
        var rgba32image = SixLabors.ImageSharp.Image.Load(bytes); // This lines takes around 10-11 sec.
        return new Rgba32Image(rgba32image);
 }

But, the peace of code is really fast:

public static UIImage ToUIImage(this IImage image)
        {
            using (var stream = new MemoryStream())
            {
                var rgba32image = image.ToRgba32Image();

                rgba32image.Save(stream, ImageFormats.Jpeg);

                var bytes = stream.ToArray();

                var imageData = NSData.FromArray(bytes);

                return UIImage.LoadFromData(imageData);
            }
        }

[antonfirsov]

@Lapinou42 the NSImage.AsJpeg() -> ImageSharp jpeg decoding step seems unnecessary.

If you are familiar with the underlying UIImage + CGImage + CGContext... API-s, you can try extracting the pixel data into a byte[] array of Rgba32 format or into an Rgba32[] array directly. Then you can use our Image.LoadPixelData<Rgba32>(data) to construct an ImageSharp image in a more efficient way.

Using .AsPNG() should be also faster. Jpeg is a super expensive stuff if your .NET runtime doesnt support SIMD.

[tocsoft]

you want to access the raw byte array from the UIImage directly instead of converting back and fourth in jpeg format.

This looks like some sample code that can help you access image data https://forums.xamarin.com/discussion/comment/60985/#Comment_60985
_disclaimer: not been tested, not an IOS developer._

Then as @antonfirsov suggests call var img = Image.LoadPixelData<Rgba32>(rawData, width, height);

That will prevent new overhead of any encoding/decoding and will be the fastest way of gettign data from UIImage->Image