ImageMagick creates huge temporary files

[serghei-topor]

The trouble is described in this thread https://www.imagemagick.org/discourse-server/viewtopic.php?t=23649

I have a cron job which removes magick files every 1 minute as a short term solution
find /tmp -maxdepth 1 -type f -name "magick-*" -delete

Does this trouble have solution? Thanks in advance

My current environment:
ImageMagick 6.8.9-9 Q16 x86_64
Ubuntu 16.04 LTS

[mikayla-grace]

ImageMagick requires temporary disk space, for example, when you utilize a delegate program or if you are processing a large file that exceeds the memory resources of your system. When the command completes, there should not be any magick-* files left over. If there is, its a bug-- unless ImageMagick fails due to some unrecoverable error. An example is, if you only have a small amount of free space in your /tmp folder and ImageMagick memory maps the file and the /tmp partition fills up. For that scenario, the OS issues an unrecoverable signal so the magick-* file remains.

If you do find magick-* files remain, first try ImageMagick 6.9.7-10, the latest release. If that leaves orphan temporary files, post here with your command so we can reproduce the problem.

[serghei-topor]

@mikayla-grace thanks, I'll try

I understand the reason of using tmp files and I have just one problem with magick temporary files: theese files have very huge size and image magick doesn't automatically remove theese files

Just for additional information:

• I use ImageMagick from command line for cropping

• all cropping operations are organized as background jobs (web service is written in RoR, for background operations I use DelayedJob framework)

• Everage image file size is 1 Mb

• System always gets in /tmp folder 2 magick- files; each file has constant size in 232 GB, so total size is 464 Gb

[jrothrock]

Currently having this issue as well, ROR 5, background process with sidekiq, but I'm using ffmpeg to create a snapshot of a video, and optimize the image with minimagic (carrierwave).

Image ends up being a couple KB, but the file in the temp is 32GB.

I have a 40GB DO droplet - 16.04 Ubuntu - it seems like it's just taking up all the remaining available space.

The file does clear when ffmpeg, or carrierwave, is finished, but sometimes the process will fail due to not having enough space, and the files won't delete

ImageMagic: 8:6.8.9.9-7ubuntu5.6

[jrothrock]

Okay so I figured out what was going on, on my end. I had minimagic running in my video uploader for carrierwave, when it shouldn't have been, I think it then tried to run imagemagick over the video. I just had to remove the minimagic process in the video uploader

[d4vidmiller]

This seems to be happening to us also. We are running ImageMagic 6.7.8.9

Any help is appreciated.

[serghei-topor]

@d4vidmiller Probably it may help you - in ImageMagick in configuration file you can strictly limit the size of the temporary files. https://www.imagemagick.org/script/resources.php#configure

In my case I had 2 dump files and each file had 250 GB and 400 Gb as the total size of /tmp folder, and these magick dumps were hanging the system

[terion-name]

Same issue

[mikayla-grace]

See . https://www.imagemagick.org/script/security-policy.php. Set policies to limit the amount of resource ImageMagick is permitted to consume. See https://www.imagemagick.org/script/architecture.php to estimate the amount of resources ImageMagick requires to read, process, and write an image or image sequence.

[unhammer]

Sorry, I've been reading the links posted by @mikayla-grace and I'm still not sure how to interpret the policy.xml settings. Am I correct in assuming that when I've added

  <policy domain="resource" name="memory" value="2GiB"/>                                                                                                                                       
  <policy domain="resource" name="disk" value="1GiB"/>

(which gives me

$ identify -list policy

Path: /etc/ImageMagick-7/policy.xml
  Policy: Resource
    name: disk
    value: 1GiB
  Policy: Resource
    name: memory
    value: 2GiB

Path: [built-in]
  Policy: Undefined
    rights: None 

) then ImageMagick will not create tmp-files over 1GiB (per file) and at most malloc 2GiB?

Or does disk add a limit to the user-specified output file?

[Erudition]

Same problem. likely due to inadvertently processing a video. Problem is Now my disk is so full I can't even delete the file...

sudo rm -rf magick-20592fpKOr42kW0ma.pam 
rm: cannot remove 'magick-20592fpKOr42kW0ma.pam': Read-only file system

[urban-warrior]

Your disk got an error and it was set to read-only to help prevent further errors. Try unmounting and remounting the file system or perhaps rebooting.

[marcelogrsp]

Hi guys, how did you sort it?

[ghareesh]

While trying to convert pdf to tiff image, the solution gets stuck in one place and generates lot of
magick-xxxxxx files in temp.. Seems to repeat at same point while re executing..How can I troubleshoot this?

[fmw42]

Where is the PDF file? We need that to test. Also what is your ImageMagick version and platform?

[unkn0wn-developer]

How did you guys resolve this?

[unkn0wn-developer]

See . https://www.imagemagick.org/script/security-policy.php. Set policies to limit the amount of resource ImageMagick is permitted to consume. See https://www.imagemagick.org/script/architecture.php to estimate the amount of resources ImageMagick requires to read, process, and write an image or image sequence.

Can this cause other problems when something that big actually needs to be processed uncompressed?

[urban-warrior]

If you read the architecture document carefully, you'll see you can limit memory and image pixels are forced to disk. The processing is slower but it helps prevent ImageMagick from thrashing your system. You can even limit your disk by policy so that if the limit is exceeded, ImageMagick will exit. See https://imagemagick.org/script/security-policy.php for a good starting point for setting polices for your environment.

[unkn0wn-developer]

If you read the architecture document carefully, you'll see you can limit memory and image pixels are forced to disk. The processing is slower but it helps prevent ImageMagick from thrashing your system. You can even limit your disk by policy so that if the limit is exceeded, ImageMagick will exit. See https://imagemagick.org/script/security-policy.php for a good starting point for setting polices for your environment.

Thank you so much for your quick reply!! :)
I'm going to edit "/etc/ImageMagick-6/policy.xml" and uncomment/edit the following line:
from

  <!-- <policy domain="resource" name="disk" value="16EiB"/> -->

to

<policy domain="resource" name="disk" value="5GB"/>

to avoid ending up with 64gb temp file causing disk problems in the worst case scenario.

We don't mind something slowing down as long as it still works.
We definitely don't want this setting to crash the uploads because temp file is too small.

• Also, how do I determine the maximum disk size I need for optimal performance?

I couldn't find any guides regarding that.
I saw people randomly commenting on other gitlab issues "oh this much gb worked for me, that much gb worked for us" but it looks like they find that by trial and error.

• Is there a concrete fact/calculation that I can use?
  For example, what caused 64gb temp file that we got? Was it a 20000000x20000000px image file? or how big of an image file that 64gb can utilize in realistic terms?
  For example, We don't care anything bigger than a thumbnail created from an 8k video (7680 x 4320).
  How much GB/MB that would be when it's uncompressed and processed by ImageMagick?

• Also, let's say ImageMagick crashed and it left a temp file behind? Would I still need to delete it manually?

• Or would ImageMagick clean that up when it runs next time for something else?

[urban-warrior]

How much GB/MB that would be when it's uncompressed and processed by ImageMagick?.

Read the architecture guide carefully. It tells you exactly how to calculate the answer to your question-- that is for the pixel cache. Some image processing algorithms consume operational memory. Even then, we have memory methods to mitigate large requests, e.g. memory-mapping the request on disk instead of heap memory.

For example, what caused 64gb temp file that we got?

Likely a very large image file.

Also, let's say ImageMagick crashed and it left a temp file behind?

ImageMagick intercepts whatever signals it can and cleans up temporary files automagically. For signals it cannot catch, like SIGBUS, some files might be left behind. However, if you set the cache synchronize policy to true, SIGBUS should never be thrown. In our online studio, the script checks for any temporary files that are over some threshold hours old and automatically deletes them. The online studio has been running for over a decade and we have never had any downtime other than routine maintenance thanks to this security policy (some parts obfuscated):

<policymap>
  <policy domain="resource" name="temporary-path" value="/always-use-something-other-than/tmp"/> 
  <policy domain="resource" name="memory" value="256MiB"/> 
  <policy domain="resource" name="list-length" value="32"/> 
  <policy domain="resource" name="width" value="8KP"/> 
  <policy domain="resource" name="height" value="8KP"/> 
  <policy domain="resource" name="map" value="512MiB"/> 
  <policy domain="resource" name="area" value="16KP"/>
  <policy domain="resource" name="disk" value="1GiB"/> 
  <policy domain="resource" name="file" value="768"/> 
  <policy domain="resource" name="thread" value="2"/>
  <policy domain="resource" name="time" value="120"/> 
  <policy domain="system" name="precision" value="6"/>
  <policy domain="cache" name="memory-map" value="anonymous"/>
  <policy domain="cache" name="synchronize" value="true"/>
  <policy domain="cache" stealth="true" name="shared-secret" value="that would be telling."/>
  <policy domain="coder" rights="write" pattern="{HTTP,HTTPS,MVG}" />
  <policy domain="filter" rights="none" pattern="*" />
  <policy domain="path" rights="none" pattern="@*"/>
  <policy domain="system" name="font" value="/somewhere/my-unicode-font.ttf"/>
</policymap>

[unkn0wn-developer]

Also, let's say ImageMagick crashed and it left a temp file behind?

ImageMagick intercepts whatever signals it can and cleans up temporary files automagically. For signals it cannot catch, like SIGBUS, some files might be left behind. However, if you set the cache synchronize policy to true, SIGBUS should never be thrown. In our online studio, the script checks for any temporary files that are over some threshold hours old and automatically deletes them. The online studio has been running for over a decade and we have never had any downtime other than routine maintenance thanks to this security policy (some parts obfuscated):

Thank you so much for providing a very detailed answer. That was super helpful!
Okay, so I've been reading the architecture guide in the past couple days. It answered most of my questions.
So, if we have <policy domain="cache" name="synchronize" value="true"/> where it's value set to true, it's impossible for imagemagick to crash and leave orphan files like 64gb file that we got. Probably, the system would run out of space or have another problem but ImageMagick would recover from it and it would clean up that temp file before the process stopped.

However, considering that we don't want 64gb file anyway, I will also set this line<policy domain="resource" name="disk" value="1GiB"/> and voila. I believe that everything should work as expected. Right?

I hope that I understood everything right. It's time to implement all that now 🎉

[urban-warrior]

it's impossible for imagemagick to crash and leave orphan files like 64gb file that we got

Let's say its improbable.

and voila. I believe that everything should work as expected. Right?

Well it depends on your expectations. But if your goal is to ensure your online system can't be taken down by some nefarious or accidental means while utilizing ImageMagick, the security policy we provided should go along way toward that goal. However, we of course, as a provider of free software, can not make any guarantees. To help ensure stability, with the security policy, you are limiting the amount of memory any one ImageMagick process can consume, how much disk, how many threads, which image types are permitted to be read or written, how much processing time, etc. If any of these policies are violated, ImageMagick will throw an exception. However, don't rely on ImageMagick to keep everything secure. If your app download's files off the Internet, you likely want to limit the filesize. You may want to sanitize any parameters an online user is setting. You may want to track your users to identify any nefarious behaviors and block them, etc.