Magick.net: Is there a thread-safe way to consume GhostScript?

[thebaysix]

Prerequisites

• [x] I have written a descriptive issue title
• [x] I have verified that I am using the latest version of Magick.NET

System Configuration

• Magick.NET version: 7.15.5
• Environment (Operating system, version and so on): Windows, 64-bit, NuGet (https://www.nuget.org/packages/Magick.NET-Q8-AnyCPU/7.15.5)

Question

I have a thread up on SO about this here: https://stackoverflow.com/questions/60823468/is-there-a-thread-safe-way-to-consume-ghostscript-via-imagemagick-net

I am using MagickImageCollection and calling "Read" where I notice a lack of thread-safe behavior. Concurrent requests result in FailedToExecuteCommand. I am able to hack together a working scenario by wrapping "Read" in a lock, but this is not sufficient as performance takes a large hit.

Some commenters have suggested this could be a problem with out-of-the-box Magick.NET and how it handles GhostScript. For instance, running multiple copies of GS is possible in Windows but Magick.NET can't seem to handle it. Is this a Magick.NET issue? Is there a thread-safe way to consume GhostScript via Magick.NET?

Thanks!

P.S. An additional related question:

From a GS Dev (source https://stackoverflow.com/a/6482178/8104403):

"Most people that want to use current GS from applications that need multiple instances spawn separate processes for each instance so that GS doesn't need to be thread safe."

Is this the way Magick.NET approaches GS?

[dlemstra]

The reason that this behaviour is happening is due to the fact that the "library" calls to the Ghostscript library can only be excuted in a single thread. The ImageMagick library will only allow a single thread to access the library (in memory) api. Other threads will need to use the command line API. But it look that it is not possible for the ImageMagick library to find the file gswin64c.exe.

Have you tried setting the Ghostscript directory (https://github.com/dlemstra/Magick.NET/tree/1dd29b578b89cab1ff04b74cfb5c0e1aa2e73cbe/docs#ghostscript)?

[thebaysix]

Thanks for the response! Yep! I already do that like so:

            var exePath = System.Reflection.Assembly.GetExecutingAssembly().Location;
            var ghostScriptPath = Path.Combine(Path.GetDirectoryName(exePath), "GhostScript");
            var ghostScriptFile = Path.Combine(ghostScriptPath, "gsdll64.dll");

            MagickNET.SetGhostscriptDirectory(ghostScriptPath); 

...as I mention in the SO thread the problem only occurs when I allow concurrent access to MagickImageCollection.Read. So it's not really that the file gswin64c.exe can't be found, but rather that it is inaccessible to a second thread when already being used (at least, this is my understanding based on the behavior I see - if it really was that GS couldn't be found at all I would expect it to always fail instead of the intermittent failures I see).

I am curious what you mean by "Other threads will need to use the command line API." I know there is a cli-based way to use GS. Is there a way to achieve this via Magick (perhaps a flag or setting)? Or will I have to spawn other threads from C# and have them call GS themselves? Any other suggestions?

[dlemstra]

Let me try to explain it with more details. Lets say we have three threads that want to read a PDF file at the same time. The first thread will use the gsdll64.dll and call the API of that library because there is a performance benifit when doing this. But due to issues inside the Ghostscript library only a single thread can use gsdll64.dll. The second thread will have to use the command line to read the PDF file and will do a command line call to the gswin64c.exe file. If the first thread is already done the third thread will be able to use gsdll64.dll. because the lock that ImageMagick has is now open again. If the first thread is still busy the third thread will also need to use gswin64c.exe to read the file.

Does your ghostScriptPath folder contain both the gsdll64.dll and the gswin64c.exe file?

[thebaysix]

Ah! Interesting. No, it currently only contains the .dll file. Are you saying that if I add gswin64c.exe Magick will then be able to handle multiple threads calling Read at the same time?

I'll definitely try adding the exe there and see if that helps me.

[dlemstra]

That is indeed what I am trying to tell you 😄

[thebaysix]

Very cool, thanks very much for the guidance! I'll report back to this thread after I make sure this addresses the issue :)

[thebaysix]

After some testing, I'm still seeing an issue, though it is a different one than before (which I'm hoping means I'm closer!). Still getting FailedToExecuteCommand, but this time it is able to find the file and the details say error/ghostscript-private.h/InvokeGhostscriptDelegate/143.

Advice appreciated!

ImageMagick.MagickDelegateErrorException
  HResult=0x80131500
  Message=FailedToExecuteCommand `".../GhostScript/gswin64c.exe" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=4 -dGraphicsAlphaBits=4 "-r300x300" -dFirstPage=1 -dLastPage=1 "-sOutputFile=.../temp/magick-10252ZvAOzMMOsCXz%d" "-f...temp/magick-10252gRfuzynqR0Pe" "-f...temp/magick-10252Bp3xSsSeS1Ol"' (1) @ error/ghostscript-private.h/InvokeGhostscriptDelegate/143
  Source=Magick.NET-Q8-AnyCPU
  StackTrace:
   at ImageMagick.MagickImageCollection.NativeMagickImageCollection.ReadBlob(MagickSettings settings, Byte[] data, Int32 offset, Int32 length)
   at ImageMagick.MagickImageCollection.AddImages(Byte[] data, Int32 offset, Int32 count, MagickReadSettings readSettings, Boolean ping)
   at ImageMagick.MagickImageCollection.AddImages(Stream stream, MagickReadSettings readSettings, Boolean ping)

[dlemstra]

This might be an access issue. Please check if your application has the rights to execute the gswin64c.exe file and is able to read or write to the ../temp folder. Another reason could be that the file that you are reading is corrupt. Does this happen for every image?

[thebaysix]

This does not happen for every image (some pdfs successfully rasterize into images but others fail with the above error, but that could be that the dll is able to handle some of the images and all the ones handed off to the exe end up failing). I am following up on a potential rights issue with the exe.

[dlemstra]

The exe and the dll will execute the same code so it is very unlikely that they will result in a different image. Have you tried converting the failing files in a single thread?

[thebaysix]

I don't believe anything is resulting in "different images." Perhaps I wasn't clear enough. In order to handle multiple simultaneous Reads, Magick.NET will delegate some Reads to the dll and some to the exe (this is my understanding based on your explanation above). The difference I'm theorizing is that the ones that go to the dll succeed, but the ones that go to the exe fail - perhaps due to a permission issue.

The service I'm implementing by nature is asynchronous and handles multiple simultaneous requests coming in through threads. What I have tried is locking the Read call, but that solution is what I'm trying to improve upon :).

[dlemstra]

You could possible test this with a parallel for loop using the same PDF file as input?

[thebaysix]

I wrapped all the code (see the StackOverflow thread for exact code) in a big Parallel.For and I have each iteration start with getting a PDF from the file system. I get the pretty much same result:

ImageMagick.MagickDelegateErrorException
  HResult=0x80131500
  Message=FailedToExecuteCommand `".../GhostScript/gswin64c.exe" -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 "-sDEVICE=pngalpha" -dTextAlphaBits=4 -dGraphicsAlphaBits=4 "-r300x300" -dFirstPage=1 -dLastPage=1 "-sOutputFile=C:/SfDevCluster/Data/_App/_Node_0/Microsoft.Canvas_App3/temp/magick-40552LJ9_Y4L9ivrn%d" "-f.../temp/magick-40552C7ylUAk98jIe" "-f.../temp/magick-40552ZBHXFAt6No4f"' (1) @ error/ghostscript-private.h/InvokeGhostscriptDelegate/143
  Source=Magick.NET-Q8-AnyCPU
  StackTrace:
   at ImageMagick.MagickImageCollection.NativeMagickImageCollection.ReadStream(MagickSettings settings, ReadWriteStreamDelegate reader, SeekStreamDelegate seeker, TellStreamDelegate teller)
   at ImageMagick.MagickImageCollection.AddImages(Stream stream, MagickReadSettings readSettings, Boolean ping)

Error still is error/ghostscript-private.h/InvokeGhostscriptDelegate/143 but this time it's going through ReadStream instead of ReadBlob.

I'm still trying to understand why the exe is acting this way. Thanks for all your help so far!

[dlemstra]

Maybe you could create a small sample console project that demonstrates the issue? This might also help you find the problem.

[thebaysix]

I have built such a standalone console application and it seems to be able to leverage the dll and exe just fine in the context of a Parallel.For loop, which above I demonstrated to yield the error.

From this I conclude that the issue is either permissions-based or a corrupted exe file. Either way the issue is not with Magick.NET. This issue can be closed if you have nothing further to add.

I appreciate your help!