Runtime: Server-side generation of images from a Canvas does not work in IIS but does in IIS Express

[jagbarcelo]

Issue Title

Server-side generation of images from a Canvas does not work in IIS (Release) but does in IIS Express (both Debug & Release)

General

This is a controller class used to generate a PNG file on the server side and return it to the client (browser). The image is made through the creation of a System.Windows.Controls.Canvas object first (along with some operations on it), which is then rendered to a System.Windows.Media.Imaging.RenderTargetBitmap object, finally to a System.Drawing.Bitmap in PNG format and then streamed to the client.

This piece of code is a migration to .NET Core from another project that was originally made for .NET Framework 4.5.2 and worked fine in both the development computer and the production server (though it was a Windows Server 2012R2 instead of a Windows Server 2019).

Sample Code

using System.IO;
using System.Threading;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Shapes;
using Microsoft.AspNetCore.Mvc;

namespace myNameSpace.Controllers
{
    public partial class drawController : Controller
    {
        private const string MEDIA_TYPE_PNG = "image/png";

        [RequireHttps]
        [HttpGet()]
        public ActionResult image()
        {
            byte[] theImage = CreateImage(96, 320);
            if (theImage != null)
            {
                var stream = new MemoryStream(theImage);
                FileStreamResult result = new FileStreamResult(stream, new Microsoft.Net.Http.Headers.MediaTypeHeaderValue(MEDIA_TYPE_PNG));
                return result as FileResult;
            }
            else
            {
                return new BadRequestObjectResult("Error");
            }
        }

        private byte[] CreateImage(double screen_dpi, double image_size)
        {
            byte[] outByteArr = null;

            ThreadStart ts = new ThreadStart(delegate () { CreateImageSTA(screen_dpi, image_size, out outByteArr); });
            Thread thread = new Thread(ts);
            thread.IsBackground = true;
            if (thread.TrySetApartmentState(ApartmentState.STA)) // Since we will use UI components in the server side
            {
                thread.Start();
                thread.Join();
            }
            return outByteArr;
        }

        [System.STAThread]
        private static void CreateImageSTA(double screen_dpi, double image_size, out byte[] outByteArr)
        {
            double OuterDiameter = 100;
            double Thickness = 3;
            // Measure and arrange the surface
            System.Windows.Controls.Canvas canvas = CreateCanvas(OuterDiameter, Thickness);

            double fake_dpi = screen_dpi * (image_size / OuterDiameter);

            Size size = new Size(image_size, image_size);
            canvas.Measure(size);
            canvas.Arrange(new Rect(size));

            // Create a render bitmap and push the part to it
            RenderTargetBitmap renderBitmap = new RenderTargetBitmap((int)size.Width, (int)size.Height, fake_dpi, fake_dpi, PixelFormats.Pbgra32);
            Rect bounds = VisualTreeHelper.GetDescendantBounds(canvas);
            DrawingVisual dv = new DrawingVisual();
            using (DrawingContext ctx = dv.RenderOpen())
            {
                VisualBrush vb = new VisualBrush(canvas);
                ctx.DrawRectangle(vb, null, new Rect(new Point(), bounds.Size));
            }
            renderBitmap.Render(dv);

            // Create a file stream for saving image
            using (MemoryStream outStream = new MemoryStream())
            using (MemoryStream tempStream = new MemoryStream())
            {
                var encoder = new PngBitmapEncoder(); // Use png encoder for our data
                encoder.Frames.Add(BitmapFrame.Create(renderBitmap)); // push the rendered bitmap to it
                encoder.Save(tempStream); // save the data to the stream

                using (System.Drawing.Bitmap bitmap = (System.Drawing.Bitmap)System.Drawing.Image.FromStream(tempStream))
                {
                    using (System.Drawing.Bitmap newBitmap = new System.Drawing.Bitmap(bitmap))
                    {
                        double scale = System.Math.Max(size.Width, size.Height) / 1200D; // regular size
                        newBitmap.SetResolution((float)screen_dpi, (float)screen_dpi);
                        newBitmap.Save(outStream, System.Drawing.Imaging.ImageFormat.Png);
                    }
                }
                outByteArr = outStream.ToArray();
            }
        }

        private static System.Windows.Controls.Canvas CreateCanvas(double ShellOuterDiameter, double ShellThickness)
        {
            Canvas myCanvas = new Canvas();

            double NormalizedThickness = ShellOuterDiameter * (0.2 / 100D);
            double ShellInnerDiameter = ShellOuterDiameter - (2 * ShellThickness);

            // The real Canvas is much more complex. For the sake of simplicity, let's just draw a Circle

            if ((ShellOuterDiameter > 0) && (ShellInnerDiameter > 0))
            {
                Ellipse ShellOuterFill = new Ellipse();
                ShellOuterFill.StrokeThickness = NormalizedThickness;
                ShellOuterFill.Stroke = Brushes.Black;
                ShellOuterFill.Fill = Brushes.LightGray;
                Canvas.SetLeft(ShellOuterFill, -(ShellOuterDiameter / 2));
                Canvas.SetTop(ShellOuterFill, -(ShellOuterDiameter / 2));
                ShellOuterFill.Width = ShellOuterDiameter;
                ShellOuterFill.Height = ShellOuterDiameter;
                myCanvas.Children.Add(ShellOuterFill);
            }

            return myCanvas;
        }
    }
}

Behaviour

When the url "/draw/image" is called from both Debug/Release within Visual Studio 2019 and IIS Express, the resulting image is as expected (a PNG with a circle in it):

However, when this is deployed to a Windows Server 2019 with IIS, the resulting image is an empty PNG file (full transparency, no circle there):

The development is done in a Windows 10 v1909 (18363.592).
Microsoft Visual Studio Community 2019 Versión 16.4.3
.NET Core 3.1.1

The production server is a Windows Server 2019 hosted in Azure.
Also with .NET Core 3.1.1

[carlossanlop]

cc @safern @maryamariyan are the owners of System.Drawing.
cc @Pilchie @wtgodbe since this problem is arising using ASP.NET.
I transferred it to dotnet/runtime, but if this belongs to the ASP.NET repo, feel free to move it there.

[anurse]

What comes to mind is that IIS runs as a special user (something like IIS APPPOOL\[App Pool Name]) whereas IIS Express runs as the user who launched it. @safern @maryamariyan could user permission issues cause problems with System.Drawing? The IIS user often doesn't have a User Profile directory (it's an option that can be enabled).

@jagbarcelo is there an exception occurring? Can you post a runnable sample application that reproduces the problem?

[safern]

@safern @maryamariyan could user permission issues cause problems with System.Drawing?

I don't believe so. We write files down, so if the user doesn't have permisions it would fail, but it is writing the PNG file but in a wrong format... I don't know if maybe the version of GDI+ in that Windows Server 2019 machine is busted, but I wouldn't expect that to be the case.

[jagbarcelo]

The IIS user often doesn't have a User Profile directory (it's an option that can be enabled).

In IIS, Application Pools -> Advanced settings, we have changed the option Load User Profile to True, restarted IIS, with same results.

@jagbarcelo is there an exception occurring? Can you post a runnable sample application that reproduces the problem?

There are no exceptions thrown at the server.

Regarding a sample project/solution, I've created this simple solution that illustrates the issue.

canvas-test.zip

When run via VS and IIS Express (debug and release) works as expected. When deployed in release, it does not.

I hope you could have a look at it and spot the problem easily. Thanks

[anurse]

The development is done in a Windows 10 v1909 (18363.592).
Microsoft Visual Studio Community 2019 Versión 16.4.3
.NET Core 3.1.1

The production server is a Windows Server 2019 hosted in Azure.
Also with .NET Core 3.1.1

@safern Does System.Drawing behave differently in Windows Server 2019? I seem to remember there are issues with using the Server OS SKU with System.Drawing

[anurse]

@safern Any thoughts? The difference between Client and Server SKU seems likely to cause problems with GDI APIs...

[safern]

The difference between Client and Server SKU seems likely to cause problems with GDI APIs...

That would be the most likely explanation to this as all our imaging APIs PInvoke into GDI+ and we just process the result of those APIs. I tried to find a documented difference in behavior in between Client and Server but couldn't find anything. @JeremyKuhne who has played longer than me with GDI+ might have some insights here.

[JeremyKuhne]

I'm unaware of any Server specific issues. It would be good to see if the stream you're giving System.Drawing.Bitmap is different between your two scenarios. I would guess the problem you're seeing happens before you're giving it to System.Drawing.

I'd try to mix this up a little bit. Does saving as a different image format work (bitmap, jpeg, etc.)? Perhaps there is a codec issue? I don't know how WPF's drawing/imaging is implemented, so that is a wild guess.

[jagbarcelo]

I've just tested to change the output to Jpeg, Gif and Tiff without any major differences. All of them, when run locally with IIS Express, render the circle correctly (considering the the lack of transparency for Jpg and Gif). When all the same tests were done in release with IIS, Tiff & Png returned a whole transparent file, whereas Gif and Jpeg returned a whole black square.

The 8 combinations shown up there are attached in the compressed file, along with the drawController.cs updated code to alternate among the different formats. The image method of the controller can now be called with a format parameter (passed in the query):

• ~/draw/image?format=png
• ~/draw/image?format=tiff
• ~/draw/image?format=gif
• ~/draw/image?format=jpg

canvas-test-update.zip

[jagbarcelo]

I've been doing some more tests in order to check if the stream given to System.Drawing.Bitmap was in fact the same or not in both scenarios. It seems it is not.

I have added a piece of code to calculate a Hash for the BitmapFrame that is then passed along to the encoder (via encoder.Frames.Add). So, before reaching that point now we have this:

    ...
    using (DrawingContext ctx = dv.RenderOpen())
    {
        VisualBrush vb = new VisualBrush(canvas);
        ctx.DrawRectangle(vb, null, new Rect(new Point(), bounds.Size));
    }
    renderBitmap.Render(dv);

    {
        // This block calculates a Hash function for the BitmapFrame object to 
        // see if there are any differences among IIS and IIS Express executions
        BitmapFrame aux = BitmapFrame.Create(renderBitmap);
        // The former line behaves differently in IIS and  IIS Express, returning completly different BitmapFrames

        // Now we just calculate the Hash for the BitmapFrame object
        var stride = ((aux.PixelWidth * aux.Format.BitsPerPixel + 31) / 32) * 4;
        byte[] pixels = new byte[(int)aux.PixelHeight * stride];
        aux.CopyPixels(pixels, stride, 0);
        using (MD5 sec = new MD5CryptoServiceProvider())
            hash = GetHexString(sec.ComputeHash(pixels));
        //hash = GetHexString(pixels); // Return the whole object as hex string
    }

    // Create a file stream for saving image
    using (MemoryStream outStream = new MemoryStream())
    using (MemoryStream tempStream = new MemoryStream())
    {
    ...

That hash value is then added to the response in a HTTP header so that it could be checked (using Fiddler for instance) while still returning the image and not changing much the whole behaviour of the method.

Response.Headers.Add("BitmapFrameHash", hash);

My tests show that the hash values are the same for both IIS Express in Release and Debug modes, but are not the same when this code is deployed to IIS.

    IIS Express (Debug)             IIS Express (Release)               IIS (Release)
PNG EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     A5DC-382D-75EC-4643-4B31-3E28-9C28-1D8C
GIF EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     A5DC-382D-75EC-4643-4B31-3E28-9C28-1D8C
TIFF    EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     A5DC-382D-75EC-4643-4B31-3E28-9C28-1D8C
JPG EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     EAAF-7EC2-A110-DE36-D4D8-819D-5A30-BE81     A5DC-382D-75EC-4643-4B31-3E28-9C28-1D8C

Continuing with the tests, instead of returning a Hash for the pixels byte[], I simply returned the whole byte[] converted to hexadecimal string within the response headers. The IIS version was returning a byte[] completely filled with zeros (in IIS Express there were quite some non-zero values).

drawController-with-hash-in-response-header.zip

[anurse]

My tests show that the hash values are the same for both IIS Express in Release and Debug modes, but are not the same when this code is deployed to IIS.

Are you using different machines with different Windows SKUs (Client vs Server) or are all these tests on the same machine? As mentioned above, I can't think of any reason the IIS version would affect the output, but I can think of ways the Client/Server version of the OS could. Can you isolate this behavior into a console app that you can try on the Client and Server OS?

[jagbarcelo]

Are you using different machines with different Windows SKUs (Client vs Server) or are all these tests on the same machine? As mentioned above, I can't think of any reason the IIS version would affect the output, but I can think of ways the Client/Server version of the OS could. Can you isolate this behavior into a console app that you can try on the Client and Server OS?

Yes, they are different machines. Developing and testing is done in a Windows 10 1909 (Build 18363.628), with Visual Studio 2019 (16.4.4). The release is done in a virtual machine in Azure running Windows Server 2019 1809 (Build 17763.973). I do not know how to retrieve the SKU you mentioned.

I have created a console app and reused most of the code in the web app controller. With minor changes the console app compiles and surprisingly it runs fine both in the development machine and in the server. So, the issue must be something related to the app being run through IIS, or to the user running the app being headless or not (IIS APPPOOL[App Pool Name] in the web app vs the admin user in the console app).

As I mentioned in one of my first replies, I had enabled the option for loading the user profile in IIS.

I have attached both the web app and the console app. You can just compare them side by side and see that the changes are minimal. However the web app does not work as expected and the console does.

Web app: canvas-test.zip
Console app: canvas-test-console.zip

[jkotalik]

@anurse do you think this issue should be transferred to the runtime repo at this point?

[jagbarcelo]

Hi, guys. Is there any chance of moving this issue from Backlog to a Milestone with higher priority? We really need this issue to be fixed so that we could release our client/server application into production. I will be available to do any test you might need. Thanks.

[anurse]

We can transfer it over to dotnet/runtime to see if someone with experience with System.Drawing can help out here. The IIS host process is a different environment but I can't think of a way ASP.NET would be corrupting the image content so I think we'll need someone from System.Drawing to investigate further.

[jagbarcelo]

Anyone? (•_•) @anurse ? @maryamariyan ? I understand these times are harder than ever, but I simply do not want this issue to be bot-closed due to inactivity. Thanks.

[jagbarcelo]

Hi all,

Here's a little update on this issue. We've just updated to .NET Core 3.1.3 and the problem persists:

• IIS Express runs as expected on the developing machine (Win 10), both in debug and release versions (now with Visual Studio 2019 v16.5.4).
• IIS on Windows Server 2019 (Azure) in release shows empty images (which is wrong).
• The console-only version of this this test works fine in release in the same Windows Server 2019.

This means that it is something related to IIS or to how IIS integrates with the client .NET Core runtime library (windowsdesktop-runtime-3.1.3-win-x64.exe). We've tested with/without enabling Load User Profile in IIS manager, with the same wrong results (empty images).

I've attached the updated test apps:

• Web app: canvas-test-3.1.3.zip
• Console app: canvas-test-console-3.1.3.zip

Any help with this issue is greatly appreciated. Thanks.

[trykyn]

Hi,
I am facing the same issue. I migrated an app to netcore which generated pictures in a webapi.
everything works smoothly in IISExpress but I get empty images in IIS. Both run locally in windows 7 environment. The self-hosted version with Kestrel also works.

If IIS starts the process by "dotnet dllpath" command, it does not work. If we run "dotnet dllpath" from console, it works as expected

[trykyn]

I found the solution to the problem.

Github compatibility
Service/Non-interactive Window Stations
Rendering will continue by default, irrespective of the presence of display devices. Unless the WPF API's being used are short-lived (like rendering to a bitmap), it can lead to a CPU spike. If an application running inside a service would like to receive the 'default' WPF behavior, i.e., no rendering in the absence of display devices, then it should set to true

In short, add a new file "runtimeconfig.template.json" to your project with the content

{
"configProperties": {
  "Switch.System.Windows.Media.ShouldRenderEvenWhenNoDisplayDevicesAreAvailable": true
  }
}

[jagbarcelo]

Thanks a lot @trykyn for your solution. How did you find it? I wouldn't say it's an undocumented feature (because it is documented) but it is quite far-fetched to me. Anyway, thanks again.

[stevetemple]

The caution in this MSDN article says that

Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions. For a supported alternative, see Windows Imaging Components.

Should this be in the Future milestone? A lot of effort went in to finding a solution but nobody explained that this isn't a supported use case.