Cms: Add support for image post-processing tools like jpegoptim and pngcrush

[angrybrad]

Created by: Anonymous ([email protected]) on 2016/01/25 13:32:12 +0000
Votes at time of UserVoice import: 40

---

Craft3 suggestion: please allow image transforms to create progressive JPEGs/interlaced GIFs/PNGs… and also allow for support of image post-processing tools like jpegoptim, pngcrush, etc, etc. This is all invaluable for “client-proofing” websites, and ensuring optimal images. Also adding support for percentage-based origins as generated by the FocusPoint plugin: https://github.com/smcyr/Craft-FocusPoint

Right now, I eschew Craft’s transforms completely, and install the Imager plugin, because the Craft transforms don’t support the above.

[angrybrad]

> Posted by Ryan Masuga ([email protected]) on 2016/09/16 16:15:29 +0000

We use Imager for all image transformations (and Focus Point...). Basically, if Imager was core transformation functionality, that would be a major upgrade for image handling in Craft.

[angrybrad]

> Posted by Daniel Howells ([email protected]) on 2015/11/23 13:00:18 +0000

For me, this would be the single best improvement to Craft I can think of right now. Standard transformations result in huge files.

[angrybrad]

> Posted by idol chan ([email protected]) on 2016/04/25 03:40:41 +0000

Also add 'transforms on upload' options so images can be delivered faster since user don't have to wait the transform to be executed when 1st request. This is useful when content is highly dynamic.

[angrybrad]

> Posted by Jeremy Daalder ([email protected]) on 2015/08/17 22:59:59 +0000

I agree but would add sharpening as being pretty much essential too - Craft's default transforms all result in very soft images, the unsharp mask control in Imager makes it essential for that alone...

[MattWilcox]

+1 for this; I'd love to not have to rely on Imager, which I use ATM only because that's how we can get focus point cropping and image optimisations (which Page Speed always clouts us for when using Crafts built in image manipulations).

[andris-sevcenko]

I'd just like to chime in that Craft 3 has Focal Point in the image editor. Admittedly it does not store percentage but absolute coordinates that can be converted to percentages with little work.

Also if you have Imagick installed then the images coming out of Craft should be pretty optimized (As of Craft 2.5)

Image editor is still not 100% done and there's more stuff coming, but I'm against adding code that will serve to bloat the core and can be done easily by a plugin.

[khalwat]

@andris-sevcenko It'd go a long way if the focal point was stored as a percentage, and thus could be used with arbitrarily-sized images. Storing the values as absolute coordinates is not particularly useful, because they are only meaningful for one size of the image.

Yes, you can translate this into a percentage with more work, but I do think this would be useful as part of the core functionality, so that once a focal point is picked, it can be applied to images of all sizes when an image is transformed ala:

^ from the FocusPoint plugin, which stores the point as a % (as does the Focal Point plugin from @aelvan)

Refs:

https://github.com/smcyr/Craft-FocusPoint
https://github.com/aelvan/FocalPointField-Craft
https://nystudio107.com/blog/creating-optimized-images-in-craft-cms

[khalwat]

As for the image optimization part of it, what we really need is just a way to run arbitrary executables on an image after it's been transformed, with a configurable options string.

This is somewhat critical, as we need a way to run image optimization tools on images after they have been uploaded by the client, and after they've been transformed.

If this is just done with an array in volumes.php on a per-asset volume basis that's fine; maybe like:

'imagePostProcessors' => [
    'jpegOptim' => [
        'path' => '/usr/bin/jpegoptim',
        'options' => '-s -P',
        'types' => ['.jpg', '.jpeg']
    ]
]

You wouldn't be adding any bloat really, it'd just check for a given Asset Volume whether there are any post processors, match the file types each should be run on, and execute them.

For the work that I do, I unfortunately can't use Craft's built-in image editor or native transforms if they lack the ability to do post-transform and post-client upload processing on the images.

[khalwat]

@andris-sevcenko One comment I want to follow up on is:

Also if you have Imagick installed then the images coming out of Craft should be pretty optimized (As of Craft 2.5)

I wish that were true. It's definitely better than it was before, but tools like jpegoptim and mozjpeg do a really nice job losslessly optimizing the images that come out of Imagick, as do optipng, and obviously, pngquant if you need it.

There are also tools to optimize svg files, and other types. It's hard to predict what a given project will need, and what will be the new hotness in the future, what with webp and other advances in image optimization.

Thus having a generic-ish method of calling post-processors would be nice to have, both for now, and for future-proofing.

refs:
https://nystudio107.com/blog/installing-mozjpeg-on-ubuntu-16-04-forge

[steverowling]

I agree with @khalwat, it would be great to be able to control how images are optimised after transforms using a post processor like jpegoptim, optipng or others.

After educating clients about the importance of optimising their sites for performance, it's frustrating not being able to use the best tools available to reduce image sizes.

[mildlygeeky]

Seems like a real issue if Craft 3 ships (as Craft 2 does) with image resizing / processing functionality that outputs larger-than-necessary file sizes and no way to fix that without non-CMS third-party tools.

Today, I have to continually run an npm process to reduce file size from images after-the-fact on client uploads and the resulting transforms.

Would be nice if it were built into core, but if not, this should be something that a plugin could do as @khalwat suggests.

[mildlygeeky]

Looking at how focal point is being done, what is the use case for someone ever just using the X and Y coords? Are we forcing everyone to do one-off math PHP and/or Twig code themselves manually rather than it being programmed once, correctly and consistently, in the CMS?

[michaelrog]

Hearty +1 for focal points being accessible as percentage values. (I don't care how they're _stored_ as long as I can _get_ a % without having to calculate it myself.)

I don't necessarily agree with image post-processors being core feature, since it requires more consistency about server environment than we can probably assume OOTB, but it'd be cool to add that as a 3P integration off a post-upload/post-save hook. (Will add this to the list of things to explore for _ImageSmith_.)

It'd also be badass if we could define Transforms via config, and have Transforms include other operations such as sharpening in addition to resizing. (This is also something I'm looking at for _ImageSmith_, but it'd be way cooler if it was core.)

[khalwat]

@michaelrog What I proposed is that Craft 3 ships with no presets for image post-optimizations at all, thus no assumptions are made about the server environment OOTB at all.

It's up to the developer / admin who _does_ know about the server environment to add that imagePostProcessors array, and fill in the appropriate info for their server environment _if they want to_.

I just need the ability to do it to be in place.

[michaelrog]

To be clear, I don't think it's a _bad_ idea to handle image workflow OOTB, if there's a way to make it consistent/elegant.

I see two questions here:

1. Should ___ be implemented as a 1P or 3P solution?
2. _How_ should ___ be implemented?

Craft has always tried to be philosophically focused on "managing content", and allowing the dev to bring whatever processing/view/distribution workflows to the equation that they wanted. So, the answer to "1P vs 3P" is basically "Is this a content problem, or is it a website-building problem?" If the answer is that it should be 1P, then Craft potentially has to care about pipes and error handling for a bunch of external binaries that it knows nothing about. It comes down to what P&T wants to support, and on that topic, I'm not in a position to be opinionated, but I see merit to @andris-sevcenko's argument about keeping core code tight.

On the answer to _how_ this all gets implemented, I'm more opinionated: I think only Transforms (pre-configured or via the editor) should ever modify an image. If something's going to happen to an image on a per-Volume basis, it should be by somehow attaching default/hooked Transforms to a Volume config, rather than directly specifying manipulation settings on a Volume.

i.e. Rather than _"When I upload a JPG to Volume X, resize the image to 1000px before storing it"_, I would want: _"When I upload an image to Volume X, run Transform Y on the image before storing it."_

(If those Transforms somehow hooked into external libraries, hooray. If they just had more access to Imagine's functions, that'd be swell too.)

I wonder if we should split this Issue into multiple topics — focal points, more ops on Transforms, pre-processors, etc. — since each is really a separate FR.

[andris-sevcenko]

@khalwat

It'd go a long way if the focal point was stored as a percentage, and thus could be used with arbitrarily-sized images. Storing the values as absolute coordinates is not particularly useful, because they are only meaningful for one size of the image.

Seeing as how focal point coordinates are defined for a particular image that has a size, they are pretty meaningful. There's very little (if any) difference in whether you multiply a set of coordinates by the scale factor or multiplying a dimension by a percentage. It's just splitting hairs at this point, but I'm open to storing that as a percentage. It's just the format of the exactly same data.

As for the image optimization part of it, what we really need is just a way to run arbitrary executables on an image after it's been transformed, with a configurable options string.

Sound like an event in the AssetTransforms service that a plugin would use to accomplish just that. Our philosophy on this can be inferred from the fact that we removed external Volumes and moved them to plugins, I think. If you're advanced enough to know what image post-processing is and what options are available to you, you're advanced enough to install the right plugin for that. What you're proposing here sounds to me like a support ticket that starts with "I found this config setting for optimizing images, so I enabled it and now I can't upload images and I promised client to launch in 2 hours. Can I get urgent help here?". And, don't get me wrong - there's nothing wrong with support, but, while it's provided for free, it has a hidden cost and that cost is development time for other features. Deferring this to plugins is better for all parties involved in the long run.

It's hard to predict what a given project will need, and what will be the new hotness in the future, what with webp and other advances in image optimization.

This just underlines everything I tried to say about not complicating the core code. It paints us in the corner in the long run for backwards compatibility because of that one hypothetical guy who still believes in .bmp optimization and is really vocal about it.

---

Bottom line: Craft is extensible. Why not take advantage of that?

@mildlygeeky

Looking at how focal point is being done, what is the use case for someone ever just using the X and Y coords?

You'd be surprised what deadlines do to your code :) But, if we're serious here, the intention is that transforms are performed by Craft. Craft performs then on a discrete Asset so coordinates vs percentages is nothing more than presenting the same data in a different format. Now that the deadline has passed, it definitely seems like percentages would work better and I'm not against that.

[khalwat]

Sounds like it's not happening. I am sad.

[andris-sevcenko]

In the core I would say no. There's no place for that there. It's perfectly feasible to do with plugins (well, we'd need to add transform events), but I think it's a similar discussion as Macbook and USB-C ports. You either see dongles everywhere or you see USB-C ports that you can tailor to be used however you want with a dongle.

[khalwat]

In the core I would say no. There's no place for that there. It's perfectly feasible to do with plugins (well, we'd need to add transform events), but I think it's a similar discussion as Macbook and USB-C ports. You either see dongles everywhere or you see USB-C ports that you can tailor to be used however you want with a dongle.

lol that's not a good example to pick, imo... having to carry dongles around is a really sore point for a number of MBP owners. But I guess the analogy is apt, in terms of having to install a plugin to get the desired functionality.

I do consider this basic functionality for anything that produces images: the ability to optimize them. But I also realize that everyone has different priorities in terms of what they consider important.

I just thought that with all of the focus on Craft 3 being performant, that functionality like this which helps frontend developers make performant sites would fit right in.

If it's not going to be in core, can we at least get the requisite events to make creating a plugin to make up for this deficiency possible? https://github.com/craftcms/cms/issues/905

[andris-sevcenko]

Of course you'll get them. We're still in beta, after all. There are quite a few things to tackle, still.

As for focal point coordinates - the way I see it there's no difference, so if people feel strongly about having it as percentages then that's coming too.

[MattWilcox]

My personal opinion is that no CMS should be outputting non-optimised images in 2017. It's just bad practice, whether it's the norm or not. If the server has the software, it should just happen.

As for Focus Point - I'd prefer %. Why bloat templates with code to work that out every single time you need it? Percentage is simply more convenient to use in a template, and will continue to work if assets are replaced with different resolution versions. You can think of it as a DRY unit.

[andris-sevcenko]

@MattWilcox If you have Imagick, you'll get pretty optimised images.

There's a distinct line between relying on a PHP extension and relying on a 3rd party tool that has to be installed and then called via command line. That line should be bridged by plugins. There's a reason why we removed external Volumes and are moving them to plugins - remove bloat and allow you to customize your install. If you think about it's the same about not making any assumptions about your content. We're not assuming anything about where you want to store your Assets or what you want to do with them.

Craft 3 will ship with a plugin store. If your environment supports it, enabling further image optimization will be one click away.

[MattWilcox]

Yeah, I get where you're coming from, but I also think given the increasing importance of optimised assets and their impact on things like SEO, that it's a very important part of a CMS's responsibility. Whether that means structurally putting such stuff in core or whether its architected as a plug-in that's shipped by default is less relevant - but the idea that a CMS is creating un-optimised assets by default, even if the software is installed on the server, seems wrong.

I really want Craft to (continue to) avoid the issues that Expression Engine had: that a "realistic" starting place actually had to have half a dozen plugins installed, because the actual core product didn't do extremely common things. I'd class image optimisation as that.

Right now the only plugins our agency use that are no-brainer 'always there for every build' are SEOmatic and Imager.

Still, it's good to know that at least it'll only be a click away for those of us that know to do it.

[andris-sevcenko]

That's a good point, Matt, and we do welcome the feedback. For now, though, we think a good starting point is to enable plugins to provide this and then we can circle back and re-visit this if that feels cumbersome.

[khalwat]

Yeah all I'm looking for is an entirely _optional_ (and omitted by default) configuration section in volumes.php that presumably only people who know what they are doing will muck with.

And it would be agnostic, you just give it the path and the options and it will just run whatever the post-processors whenever the types match.

While installing a plugin to do this would always be an option, it'd be nice to _not_ have to install a plugin just to use Craft's Assets. I realize it may seem extreme, but I have long eschewed Asset Transforms for the Imager plugin just because I need this type of optimization on the frontend.

[angrybrad]

FWIW, it sounds like what @khalwat is asking for is similar to what we're doing with the new C3 backupCommand config setting.

* Craft will use the command line libraries `pg_dump` and `mysqldump` for backing up a database
* by default.  It assumes that those libraries are in the $PATH variable for the user the web server is
* running as.
*
* If you want to use some other library, or want to specify an absolute path to them,
* or want to specify different parameters than the default, or you want to implement some other backup
* solution, you can override that behavior here.
*
* There are several tokens you can use that Craft will swap out at runtime:
*
*     * `{path}` - Swapped with the dynamically generated backup file path.
*     * `{port}` - Swapped with the current database port.
*     * `{server}` - Swapped with the current database host name.
*     * `{user}` - Swapped with the user to connect to the database.
*     * `{database}` - Swapped with the current database name.
*     * `{schema}` - Swapped with the current database schema (if any).
*
* This can also be set to `false` to disable database backups completely.

i.e. we provide a sensible default (mysqldump or pg_dump), but the entire command is completely over-writable for people with more specific business requirements.

[khalwat]

Yep, same idea @takobell , different data. Images are sexier than databases.

[aelvan]

@MattWilcox It's not like the images you get by using PHPs built-in functionality are "non-optimized". :)

Also, the notion that it's possible to optimize an image 100 % is just wrong. All compression is a trade-off between the level of compression, and the amount of time (resources) you're willing to use. I think it's fair that Craft sticks to what's supported by PHP, and what an average webserver can be expected to cope with.

What I really would love is if all of Imager's functionality could be implement in Craft3 so @andris-sevcenko can enjoy the support requests! ;P

Seeing as that's not likely to happen, I would love to see more features for extending the transform process. Maybe as config settings as @khalwat suggests, or hooks that plugins can take advantage of. I think well-placed hooks would be better, but I guess it depends on the case. Say you want to do post-processing on an image, it works great to do that after the transform process is done if the files are in a local volume. But if it's on a remote one, you'd ideally want to do it after the image has been transformed, but before it's uploaded to the cloud. So, yeah, hooks everywhere, before, during, after. :)

(And of course, % in focus points!)

[khalwat]

Right @aelvan most of what I'm discussing here is the ability to run _lossless_ image optimization tools that remove the unneeded cruft. Let's call it minimization/ugilification for images.

We can already set things like the compression level and resize images as we see fit with transforms. The only piece of the puzzle that is left is the lossless optimization of them for frontend delivery.

I also tend to agree that going forward, Imager hooking into Craft to provide some of its awesome capabilities would be the way to go. Leverage your tech debt by utilizing Craft core. :)

[andris-sevcenko]

@aelvan ❤️❤️❤️ I love that you've gotten a taste of support requests that involve image transforming in a gazillion different environments and know what's coming ;)

[aelvan]

LOL. As a fun-fact (for everyone else than me) I can add that the most common support request by far is: "I downloaded your plugin, used all the features at once, trying to create a gazillion transforms, and now my server has died. Help.". I have a reply that I copy-paste.

[angrybrad]

@aelvan it's pretty incredible the things people take for granted and expect their lowly web server to do.

[khalwat]

@aelvan send them @andris-sevcenko 's personal cell number

[aelvan]

@khalwat I read in Craft's new "Code of conduct" that doxing is frowned upon, so I'm not doing that anymore... :-/

[andris-sevcenko]

Closed via 2da21d1 (focal point as decimal coordinates) and ba8fa7f (transform events).

[khalwat]

ImageOptim is a Craft 3 plugin that automatically optimizes images after they've been transformed https://github.com/nystudio107/craft3-imageoptim

[mark-chief]

Still cant specify image processor other than GD and Imagick off the shelf? Craft Version 3.6.8