Cms: feature: option to force transform generation on getUrl() call

[narration-sd]

Description

@andris-sevcenko Andris, I wonder if you'd considrer adding a param, so that we would have Asset::getUrl($transform, $generateImmediately = false)?

This would make something cleaner in a plugin I'm building, whose abilities you might sometime enjoy, and quite possibly for others. because it would mean the url was real without taking a page reload, for situations where the use isn't directly on a page. Think Element Api, for one example.

It would be or'd with generateTransformsBeforePageLoad, so would need to be passed through to Assets..php line 578.

I don't _think_ it would affect any normal operation from magic getters such as with Twig, but that thought would need to be validated -- maybe you already know.

Steps to reproduce

1. n/a
   2.

Additional info

PHP version 7.1.7-1+ubuntu16.04.1+deb.sury.org+1
Database driver & version MySQL 5.7.19-0ubuntu0.16.04.1
Image driver & version GD 7.1.7-1+ubuntu16.04.1+deb.sury.org+1
Craft edition & version Craft Pro 3.0.0-beta.33
Yii version 2.0.13
Twig version 2.3.2
Guzzle version 6.2.1
Imagine version 0.7-dev
Plugins: yes

[khalwat]

Try something like:

        // Force generateTransformsBeforePageLoad = true to generate the images now
        $generalConfig = Craft::$app->getConfig()->getGeneral();
        $oldSetting = $generalConfig->generateTransformsBeforePageLoad;
        $generalConfig->generateTransformsBeforePageLoad = true;

...do your stuff...

        $generalConfig->generateTransformsBeforePageLoad = $oldSetting;

[narration-sd]

yeah, would work, and thanks -- still thinking you are a config-meddler :)

but wouldn't this param be more elegant -- and people could use it from Twig, probably, inverse of my concern check above....

[andris-sevcenko]

I have to side with Andrew here - for sites in general there’s the config setting and, if you need that setting overriden, then Andrew’s code is he way to go.

Also, I don’t think it’s a good idea to have a parameter that overrides a config setting.

[narration-sd]

Well, regret you got kind of stuck on this, Andris.

I don't think Andrew was making a 'side', just offering a dependably intelligent suggestion, which I went ahead and used. We had a nice conversation later, where I got a chance to say something appreciatively about all he contributes.

For myself, I've made some much more complex interactions with and alterations to Craft's internal behaviors, so this one seemed straightforward. As well, I guess it's something we come to with experience, that a primary honor in a design is how accomodating you can be in it.

It's not what I'm doing, but I think the first time you made a React or such UX running off an api and found that none of the images showed until you not just changed spa screens, but entirely reloaded the site, possibly even several of its views -- and that the same would of course occur for every one of your client's viewers -- you'd recognize why it would be useful to have a defaulted-off parameter as proposed, not dependant on how more internal matters might get moved around.

When I was musing over this a moment last night, somehow I thought again of a poet and philosopher from your corner of Europe, who did get a Nobel, but is even better (and showing entirely human) in being read again, as I have done in chasing down an image that stuck with me, about discovering openable doors into a future.

This is Czeslaw Milosz, 'Visions from San Francisco Bay' the book of several read, and of course I'd made up my own version of the image in earlier years, which turns out truly complex, nuanced, and extensible as well as capable in the original -- creatively non-finished., there for us to do something with.

I didn't intend on a metaphor, but one or so might be there, and useful, as to learn an Asian language, perhaps just enough to recognize what it is about those and their cultures, to touch another point in recent mention. Mine primarily was Korean, though the entomology and 'strokes' of just a few characters used across many countries doesn't go amiss either, and there are quite fascinating books for this.

Btw, I listen to the Bangles also, as at the moment...and think Milosz was a bit this way as well....

Ok, @andris-sevcenko , don't think intended more than a tidbit here, but you got several, including a kind of apology for maybe being a bit direct in paying attention to making confidence in how another contributor yesterday felt appreciated, by his own lights...

Cheers,
Clive

[khalwat]

@narration-sd Seems like the case for your SPA website, you should just globally set 'generateTransformsBeforePageLoad' => true config, in general.php no?

[narration-sd]

@khalwat, yes, I knew I should have addressed that probably.

The point is that what I'm doing might be used by many others, or even a site I worked on, where the consideration for the larger situation would be to keep the default lazy transform generation.

I've often enough felt that to be a bit twitchy, and these days we see the problems, people taking outside solutions, and even this global behavior switch, which has its own issues in scale circumstances.

So -- I can't force the global to be set a certain way, the places my arrangement would need it would often be of workable small scalele, and if it weren't, the early or external means of getting transforms would be in use. Then my regime is simple, can afford generation times, and needs to avoid the dead on first arrival problem, needs to force generation.

Doing that by setting an otherwise unused, not very visible, defaulted call parameter seemed a clean solution, especially as it would modify in simple fashion about three lines in Craft. I've used the same in a pull request, for similar reasons.

But nusuth, doesn't matter, as one of my favorite individuals in a cultural fiction says in his language.

Life goes on, happily, and thanks again for the means I hadn't yet gotten to think of, which as you know I used. What I said here yesterday is really in support of some other hope, and we are allowed those, I think :).

[narration-sd]

p.s. I just happened to check in a moment at Slack, Andrew, and saw your latest announcement -- so it's you, unsurprisingly, one of those prominent in the alternative solutions.

On a scan, ImageOptimize looks like a very powerful and useful tool - which precreates the transforms. Q.E.D., sort of, as the adversary of Moriarity used to like to say...

This is one more thing a client can't be required to use, but right there now when they want to.

[michaelrog]

I'll toss in that I bump up against this all the time, when I'm mapping data to Algolia indexes, ElementAPI transformers, etc. — places where I need the actual hard URL of the transformed image, rather than a 'promise' URL that goes through a controller.

I use getConfig()/setConfig() to get around it, and that works... but I'll admit that having to write those 3 lines of extra code repetitively is annoying. Seems like it's a common enough use case (for me, anyway) to merit adding a param to getUrl() that says "no matter what the config happens to say, please transform _this one_ now if needed."

Granted, this whole decision tree is tied to the assumption that Craft is performing these transforms locally in its own native way, and I feel we're moving further from that assumption in Craft 3 [because we can]. Still given the frequency with which Craft's internal transform engine will still be used, and given the rising popularity of using Craft as a data provider to external systems _as well as_ to Twig templates... I think it'd be convenient to allow transforms to be escalated _ad hoc_ using a param on getUrl as @narration-sd suggested.

[brandonkelly]

Wouldn’t be opposed to adding a new $generateNow argument to craft\services\Assets::getAssetUrl(), which is what craft\elements\Asset::getUrl() calls internally. So if a plugin has a need to get the URL right away, they could just call that method rather than going through getUrl().

[narration-sd]

Seems a pretty perfect creative solution, which might be a model for other things, @brandonkelly -- answers the need, while keeping any tendency for unsensible use in check. Thanks!

[maxstrebel]

@brandonkelly – is this also possible to do in twig?

If you want to generate favicons or og:images in head, the transforms will never be triggered. There are plugins for that - but I have projects, where I'd rather solve this with core functionality. I tried to make it work, but I believe this is not exposed?

[brandonkelly]

@maxstrebel Good question. I just added a new $generateNow argument to craft\elements\Asset::getUrl(), so you’ll be able to start setting that to true on a case-by-case basis as of the next release.

[maxstrebel]

Thank you so much! This is very helpful.

[khalwat]

I think it's awesome that this option is added, but @maxstrebel I'd think the pending image transforms should be generated no matter where they are in the document?

I mention it just because it's possible there might be something else wrong here as well?

[maxstrebel]

Hi @khalwat, that's a good point. I honestly need to investigate. Is this unusual behavior?

[brandonkelly]

It’s a little complicated. By default if a transform isn’t generated yet, getUrl() will give you a URL to the controller action that generates the transform, which will end up redirecting the browser to the generated transform once it’s done. It will also add a “Generating pending image transforms” job to the queue, to ensure that the transform gets generated regardless of whether the transform generator URL ever actually gets requested.

However in the case of social images like og:image, that URL won’t actually be requested until the page is shared on a social network, which may never happen. And by default the queue will only get run automatically on control panel requests, so there’s a chance that even the background job doesn’t get around to it for a while (unless you’ve set up an alway-running queue worker).

So with those combined, it’s certainly possible for social image transforms (or various srcset-sized image transforms) to not get generated for a long time, all the while preventing the page from being cached properly.

[maxstrebel]

Thank you for the detailed clarification. Can't wait to test drive the next release 🤘🏻.

[brandonkelly]

Craft 3.4.26 is out now with that change.