Aws-lambda-dotnet: Does AWS Lambda support System.Drawing invocations with libgdiplus ?
Hey there,
I have recently implemented Captcha generation in my AWS Lambda .NET Core 2 API. I could have sworn that it was running at one time, but now it is simply failing out. I have looked into the current System.Drawing implementation of .NET Core; however, from what I'm reading, the explicit dependencies on GDI+ make it so that this library is not cross-platform compatible. Upon further investigation, there is a CompatCore.System.Drawing that was originally developed by the Mono team. Supposedly, this is a cross-platform version of System.Drawing that is capable of using the Linux based implementation of GDI+; however, attempting to use it simply raises the following error / stack trace:
System.TypeInitializationException: The type initializer for 'System.Drawing.GDIPlus' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'gdiplus': The specified module or one of its dependencies could not be found.
(Exception from HRESULT: 0x8007007E)
at System.Drawing.GDIPlus.GdiplusStartup(UInt64& token, GdiplusStartupInput& input, GdiplusStartupOutput& output)
at System.Drawing.GDIPlus..cctor()
--- End of inner exception stack trace ---
at System.Drawing.GDIPlus.GdipCreateBitmapFromScan0(Int32 width, Int32 height, Int32 stride, PixelFormat format, IntPtr scan0, IntPtr& bmp)
at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
at Ivy.Captcha.Services.CaptchaGenerationService.GenerateCaptchaImage(Int32 captchaCharLength, Int32 width, Int32 height)
at IAGE.Api.Data.Services.CaptchaApiService.GenerateCaptcha() in D:\Workspaces\iam-global-education\src\Api\IAGE.Api.Data\Services\CaptchaApiService.cs:line 63
at lambda_method.lambda_method(Closure , Object , Object[] )
at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Builder.RouterMiddleware.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elmah.Io.AspNetCore.ElmahIoMiddleware.d__6.MoveNext()
According to the following thread: https://github.com/CoreCompat/CoreCompat/issues/3
I should have been able to include the libgdiplus DLL ref by adding the runtime.linux-x64.CoreCompat.System.Drawing NuGet package; however, that was to no avail for me. I have been trying various combinations of library references hoping that this would be a similar issue many people have seen with the System.Cryptography.x509Certificates; but I've yet to find a combination that is allowing this to work.
Has anybody else been able to get System.Drawing functionality up and running on AWS Lambda?
ibuchan72390
All 7 comments
I have heard about using libgdiplus for System.Drawing on Linux but I haven't tried it out. For image manipulation in Lambda I have been using the latest version of Magick.NET-Q16-AnyCPU. I haven't done any drawing with it which I assume you would need for Captcha but I see it does have some drawing capabilities.
normj
on 18 Apr 2018
My problem appears to be that libgdiplus is not a default installation on the Lambda Linux instance. Supposedly, the runtime.linux-x64.CoreCompat.System.Drawing was a NuGet package that simply executed scripts to install this library via apt-get on application startup; however, after digging into the code for this project over at the link below, I've been unable to understand exactly how this works or why it might be failing for me.
I'll continue to dig around for a solution for now; but, that Magick.NET link appears to be a valid method of generating some Captcha images, so I'll be sure to keep that as a fallback. I've looked into using the native System.Drawing since it appears that's been developed for better cross-platform compatibility, but It still appears that they're in some way referencing the installed GDI+ library though,
You have seen this Magick.NET package used successfully on Lambda?
ibuchan72390
on 18 Apr 2018
Yes Magick.NET was updated a couple months a go to work cross platform out of the box. All I had to do was add it to the project. Below is a Lambda function I wrote to create image thumbnails with it.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.0</TargetFramework>
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Amazon.Lambda.Core" Version="1.0.0" />
<PackageReference Include="Amazon.Lambda.Serialization.Json" Version="1.1.0" />
<PackageReference Include="Amazon.Lambda.S3Events" Version="1.0.2" />
<PackageReference Include="AWSSDK.S3" Version="3.3.16.2" />
<PackageReference Include="AWSXRayRecorder.Handlers.AwsSdk" Version="2.1.0-beta" />
<PackageReference Include="Magick.NET-Q16-AnyCPU" Version="7.4.2" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="Amazon.Lambda.Tools" Version="2.1.1" />
</ItemGroup>
</Project>
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.Lambda.S3Events;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.XRay.Recorder.Core;
using ImageMagick;
// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]
namespace ImageResizer
{
public class Function
{
IAmazonS3 S3Client { get; set; }
/// <summary>
/// Default constructor. This constructor is used by Lambda to construct the instance. When invoked in a Lambda environment
/// the AWS credentials will come from the IAM role associated with the function and the AWS region will be set to the
/// region the Lambda function is executed in.
/// </summary>
public Function()
{
Amazon.XRay.Recorder.Handlers.AwsSdk.AWSSDKHandler.RegisterXRayForAllServices();
S3Client = new AmazonS3Client();
}
/// <summary>
/// Constructs an instance with a preconfigured S3 client. This can be used for testing the outside of the Lambda environment.
/// </summary>
/// <param name="s3Client"></param>
public Function(IAmazonS3 s3Client)
{
this.S3Client = s3Client;
}
HashSet<string> SupportedImageTypes { get; } = new HashSet<string> { ".png", ".jpg", ".jpeg" };
/// <summary>
/// This method is called for every Lambda invocation. This method takes in an S3 event object and can be used
/// to respond to S3 notifications.
/// </summary>
/// <param name="evnt"></param>
/// <param name="context"></param>
/// <returns></returns>
public async Task FunctionHandlerAsync(S3Event evnt, ILambdaContext context)
{
foreach (var record in evnt.Records)
{
var bucket = record.S3.Bucket.Name;
var key = record.S3.Object.Key;
if (key.StartsWith("thumbnails"))
{
context.Logger.LogLine($"Object s3://{bucket}/{key} is already a thumbnail");
continue;
}
if (!SupportedImageTypes.Contains(Path.GetExtension(key.ToLower())))
{
context.Logger.LogLine($"Object s3://{bucket}/{key} is not a supported image type");
continue;
}
context.Logger.LogLine($"Processing s3://{bucket}/{key}");
MemoryStream resizedImageStream;
using (var response = await this.S3Client.GetObjectAsync(bucket, key))
{
AWSXRayRecorder.Instance.BeginSubsegment("Magick Image Resize");
try
{
using (MagickImageCollection collection = new MagickImageCollection(response.ResponseStream))
{
foreach (MagickImage image in collection)
{
image.Resize(200, 200);
}
context.Logger.LogLine($" Image resized");
resizedImageStream = new MemoryStream();
collection.Write(resizedImageStream);
resizedImageStream.Position = 0;
}
}
catch(Exception e)
{
AWSXRayRecorder.Instance.AddException(e);
throw;
}
finally
{
AWSXRayRecorder.Instance.EndSubsegment();
}
}
var index = key.LastIndexOf('/');
var thumbnailKey = "thumbnails/" + (index != -1 ? key.Substring(index + 1) : key);
await this.S3Client.PutObjectAsync(new PutObjectRequest
{
BucketName = bucket,
Key = thumbnailKey,
InputStream = resizedImageStream
});
context.Logger.LogLine($" Thumbnail saved to s3://{bucket}/{thumbnailKey}");
}
}
}
}
I'm just getting into Magick and I think it will work beautifully. Not only do you have access to a relatively diverse image manipulation API, it appears to be well supported cross-platform. Thank you so much for the suggestion, a little bit more development effort and I should be able to clear this hurdle without issue!
I'm still curious how some people seem to have been able to get the GDI+ implementation up and running. I do not see any way in which one could get the appropriate libgdiplus library installed into the Lambda prior to runtime execution.
ibuchan72390
on 18 Apr 2018
It seems that the ImageMagick Drawables API may not properly be Cross-Platform compatible with AWS Lambda from what I'm seeing. Upon implementing the library and executing the Draw() function, I'm raising the following error:
ImageMagick.MagickDrawErrorException: NonconformingDrawingPrimitiveDefinition `text' @ error/draw.c/DrawImage/3402
at ImageMagick.MagickExceptionHelper.Check(IntPtr exception)
at ImageMagick.DrawingWand.NativeDrawingWand.Render()
at ImageMagick.MagickImage.Draw(IEnumerable`1 drawables)
at Ivy.Captcha.Magick.Services.MagickCaptchaGenerationService.GenerateCaptchaImage(Int32 captchaCharLength, Int32 width, Int32 height)
at IAGE.Api.Data.Services.CaptchaApiService.GenerateCaptcha() in D:\Workspaces\iam-global-education\src\Api\IAGE.Api.Data\Services\CaptchaApiService.cs:line 63
at lambda_method.lambda_method(Closure , Object , Object[] )
at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Builder.RouterMiddleware.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Elmah.Io.AspNetCore.ElmahIoMiddleware.d__6.MoveNext()
No matter what I do, I can't seem to get any drawing functionality to work in Lambda. I've attempted to use System.Drawing (Microsoft), CoreCompat.System.Drawing (Mono), and ImageMagick (3rd Party), but all seemingly to no avail. Every image generation library seems to have some major system dependencies that I'm having a hard time getting around with AWS Lambda. I'll continue to experiment and report back if I find success.
Is there any chance we'll ever be able to customize our Lambda runtime in the future similar to how we can customize Docker containers? All of this could be solved with a simple execution to install libgdiplus into my Lambda instance.
ibuchan72390
on 19 Apr 2018
I'm an idiot and never should have been generating Captcha images on my AWS Lambda server when I could have simply used Google ReCaptcha. This way my AWS Lambda only handles the verification code and never has to reference any form of drawing functionality.
I'd still like to see a method of implementing some form of drawing functionality on AWS Lambda in the future; however, I'm going to put this issue on the back burner and chalk this up to a terrible design decision on my part for the time being.
ibuchan72390
on 20 Apr 2018
Perhaps Lambda's deployment layers could remedy this issue until proper support is available in .Net Core at some point.
https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
tofalck
on 13 Apr 2019
Related issues
01binary
路
7Comments
matsola
路
4Comments
ljacobsson
路
7Comments
Xocix
路
7Comments
Kralizek
路
3Comments
Most helpful comment
Yes Magick.NET was updated a couple months a go to work cross platform out of the box. All I had to do was add it to the project. Below is a Lambda function I wrote to create image thumbnails with it.