October: Backend render blocking gravatar image and plugin images

[ghost]

How to reproduce this issue

1. Don't add a custom avatar thumbnail - October will each time try to load an image from the website:

//www.gravatar.com

This gets even worse because October sends this website a Query String to get back a specific image size.

The other no-no is that October is trying to load the website with a // this means both the HTTP and HTTPS connections are being loaded. October should be loading HTTPS only external files.

That code can be found here: https://github.com/octobercms/october/blob/26173486d39cdf4312535ecf247e3d7a161713b4/modules/backend/models/User.php#L94-L117

1. If you have a lot of plugins all the images will render block the loading of the web page. This gets worse when the svg or png file sizes are big and not compressed.

Screenshot of performance

In the above waterfall image we can see three problems.

1. The avator image - blocking the rendering.

2. The plugin images - blocking the rendering.

3. All the plugin images are being double loaded?

Solutions

1. October should not need to load via an external connection to grab a default avatar thumbnail it should be saved in the October folder.

2. The avatar image should not need to have any query string's needed to work out the image size! The image size range is wide enough to be able to contain all it's image sizes without the need to set up srcset.

As per comment found here: https://github.com/octobercms/october/issues/4510#issuecomment-517287002

Avatar account image sizes in the top navigation menu are now fixed to the following sizes: 45px, 60px and 78px.

Want to mention something like min-size for account avatar's should be at least 78px in size and in a square format.

Therefore the range would be 45px - 78px meaning a default avatar image of 78px will cover all possible outcomes.

See a previous PR here: https://github.com/octobercms/october/pull/4511

So to fix points 1 and 2 October should load a default avatar image of 78px via a local connection. You can remove all that useless code and replace it with the following logic: Does the user's setup have an avatar? (yes or no) Yes ok use their image, no ok use the default image. As for sizes it's fixed with a max-width and max-height of 78px.

1. Loading all the plugin main icon images before the web page is rendered is causing me big headaches!

The correct solution would be to load a template at render time and then when the web page has loaded fill in the plugin images.

For example:

In this example the web page would be loaded with an empty container with the dimensions of the images (I have drawn a white square to give an example - but in real life that would be transparent). Then when the web page has rendered October then fills in all the images.

I'm not sure if the images could be preloaded as they need to be loaded within one second and October is very slow. But it maybe a good idea to preload them as all the images are above-the-fold.

1. There is another issue with regards to the javascript for the top navigation slowing down the rendering of the web page. But that is a different issue and I want this issue to just focus on the loading of all these images. I strongly feel that script should be loaded async as currently the list of plugins are being loaded and then that script is only showing the left hand side and hiding the right hand side and then showing all the plugins and then the page is allowed to fully render by which time that script has added a huge amount of not needed time to render a backend page.

October loads like this:

Then adjusts and fills the whole width - you may either see it with a large delay, or like a FOUT (a quick flash when the web page loads). That was discovered in this issue: https://github.com/octobercms/october/issues/4052

1. October should start using the Native Lazyload API - Let's say a user has 200 plugins and so there is a huge amount of images out side the visible screen area - all those images should be lazyloaded and not tried to be loaded when the web page is being rendered.

Code line: https://github.com/octobercms/october/blob/a67ccfe993a6c34abc420d01bdd1cc9332b18cae/modules/backend/layouts/_mainmenu.htm#L27

<img role="img" loading="lazy" class="svg-icon" src="<?= Url::asset($item->iconSvg) ?>">

Above example added: role="img" and loading="lazy"

Code line: https://github.com/octobercms/october/blob/a67ccfe993a6c34abc420d01bdd1cc9332b18cae/modules/backend/layouts/_mainmenu.htm#L62

<img role="img" loading="lazy" src="<?= $this->user->getAvatarThumb(90, ['mode' => 'crop', 'extension' => 'png']) ?>" class="account-avatar" />

Above example added: role="img" and loading="lazy"

1. Default avatar can be loaded via DATA URI image - it's gonna be so small, so why waste time loading another connection.

2. Stop all the plugin images being double loaded.

3. Might as well preload this as well, as it's also slowing down the web page load:

modules/system/assets/ui/font/fontawesome-webfont.woff?v=1.0.1

It's always going to be needed due to the preview button using a font awesome ison.

1. Found yet another issue with this setup:

modules/backend/assets/images/favicon.png

Can be preloaded as well.

[ghost]

I wait for the admins and other people's input in to this issue. For me this issue is the most annoying thing currently in October CMS and can delay the rendering of the backend web pages between 2 - 30 seconds depending on network congestion. Which is totally pointless as October shouldn't even need to use an external connection for this part.

[Samuell1]

I agree that gavatar is useless in backend and should be removed and changed to just simple name + icon?

For fontawesome we can use font-display:swap; but idk how it will look before it swaps to correct font.

[bennothommo]

I disagree - I like having the Gravatar image there.

@ayumihamsaki whilst I can definitely see that the rendering is blocked by the image being loaded, would you be able to find out why and see if there's a way around it?

[ghost]

@bennothommo

To me it looks like the avatar only chooses Mystery man for default and no other choice for default. Meaning a hard-coded data-uri image might work fine.

When testing the pr - I did by swaping the default avatar from:

        // Default is "mm" (Mystery man)
        $default = array_get($options, 'default', 'mm');
        return '//www.gravatar.com/avatar/' .
            md5(strtolower(trim($this->email))) .
            '?s='. $size .
            '&d='. urlencode($default);

into this:

        // If no custom avatar then display the default (Mystery man) image
        return 'data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAgICAgJCAkKCgkNDgwODRMREBARExwUFhQWFBwrGx8bGx8bKyYuJSMlLiZENS8vNUROQj5CTl9VVV93cXecnNEBCAgICAkICQoKCQ0ODA4NExEQEBETHBQWFBYUHCsbHxsbHxsrJi4lIyUuJkQ1Ly81RE5CPkJOX1VVX3dxd5yc0f/CABEIAFoAWgMBIgACEQEDEQH/xAAbAAEAAgMBAQAAAAAAAAAAAAAAAQYDBAUCB//aAAgBAQAAAAD6KBEiAmCSDJYujrVzUElk6pipkAuWwKbrgtPQPFKggz23P5q/OJh7s3SNerasxKz9MMNO8G3bwOBxDvdwDQqhYuukNamv/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/aAAgBAhAAAAAAAAAAD//EABQBAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQMQAAAAAAAAAA//xAAyEAACAQEGAwYDCQAAAAAAAAABAgMEAAURICExEiJREBMwQWFxcoGhIyQyMzRAU5Gx/9oACAEBAAE/AP3ccbyuqIuLG1NdUEYBl53+gsIYAMBEgHThFp7upJVPIFbqmlqujlpX4W1U/hYbHwLppwkJmI5nOnooyVNOk8DxtuRy+hsQQSCNc9IAKWnw/jXLVhRVVHDt3jf7nuucS0qoTrGcD7HJNKsMbu2yizMWZmO5OJz09RJTyiRPmOotBMJ4g4Vl9GHYTgCddOmtq+teduAKVRTsdyfXPGjyOqIuLG1Jd0UADOA8nXyGSppIKhMHXm8n87VdJJSylH1Hkw2Oa7aQQRCRh9o/0HTsxyVFPHUwNG2+4PraSNo3ZGGDA4HJQwd9Uxqdhq3sPAvqALIkq+Y4W9xkuSMfeJD0Cj5+BeaB6OTquDD+8ly/ky/HY9nSwyVv6Of4T2//xAAUEQEAAAAAAAAAAAAAAAAAAABQ/9oACAECAQE/ABP/xAAUEQEAAAAAAAAAAAAAAAAAAABQ/9oACAEDAQE/ABP/2Q==';

The same test loaded 2 seconds faster. Everytime I got a lightning fast response.

For a person on a localhost the only file being externally loaded is the //www.gravatar.com/avatar/ which means October can only run on a working network connection. While switching to a data-img the same user doesn't need to have a connection to the internet to run October as everything is local.

From running a test in Firefox there is the new:

Which does label gravatar.com as a bad tracking website - but this may not cause any problems in rendering. As I've tested this feature on and off.

I feel the issue is with four things:

1. External connection - October doesn't pre-connect this website - so has an extended delay, it would need to be pre-connected and preloaded. (Which will cause a delay in front of the asset - seen in the waterfall timing diagram).

2. Using //www.gravatar.com instead of https://www.gravatar.com creates a delay, two connections where the browser needs to choose - the pre-connect time is more than double. (Which will cause an increase to the delay in front of the asset - seen in the waterfall timing diagram).

3. Query string added, another delay needed to adjust the avatar image size to be brought back from the request. (Which will cause a delay after the asset - seen in the waterfall timing diagram, to recalculate everything).

4. Network congestion - As an unnecessarily external connection. This can cause October to have a delayed (up to) 2 mins in loading time. October has no built in Network connection logic, so we would just end up seeing part of the web page loaded or a spinner.

Please give more reasons why you still want it and I can research into this issue more.

[bennothommo]

@ayumihamsaki My understanding is that once its loaded once, it should be cached by the browser, so it shouldn't affect any further loads. Same as on Gravatar's side, once they have generated the image once with our specific size, it shouldn't have that same delay again because they'll just use their cached version when we make any further requests.

We can definitely change the URL to use https:// - no need for the // any more.

The reasons I want to keep it are simply that it's been there for at least 4 years and this is the first time, from what I can tell from the issues/PR history, that someone has raised an issue with it being there.

[ghost]

@bennothommo

I believe the answer is from the history of that file, it was added into October from day one. It's the sort of thing you do as a temp thing when you are building a pltform and then when everything is being finalized and you add some performance optimizations, you would then remove it. Not being rude - but October has never been optimized (that is quite obvious).

The reasons I want to keep it are simply that it's been there for at least 4 years and this is the first time, from what I can tell from the issues/PR history, that someone has raised an issue with it being there.

It's being reported as something else, the delayed loading of this avatar is causing this to happen:

That issue is being reported in several places in this repo.

My understanding is that once its loaded once, it should be cached by the browser, so it shouldn't affect any further loads. Same as on Gravatar's side, once they have generated the image once with our specific size, it shouldn't have that same delay again because they'll just use their cached version when we make any further requests.

If you look at the timing diagram you will see this:

We can see that it's being loaded by cache and 304 - yet there is a huge delay being created between the previous asset and the next asset. This is because October is using an unoptimised external connection.

End of the day these are some results:

1. With external connection

1. With local connection

---

Really makes no sense to keep it.

[bennothommo]

@ayumihamsaki I'm just not getting the same test results as you - on a website hosted locally and a separate one hosted externally, I'm seeing around 200-300ms for the initial load of a Gravatar, and then once it's cached, it like 30-60ms to load it up on subsequent requests, and it is having little to no bearing on the load time of the site or any further assets.

If we're going to remove it unilaterally for all October CMS installs, I think I would want to see some more compelling evidence that it is widely affecting people's load times.

[ghost]

@bennothommo I've given up on this issue...

You have basically said roughly 400mS is being added to the load time on every backend web page rendering - then said a conflicting comment "no bearing on the load time" so adding half a second to the load time has no bearing ???

Plus you haven't factored in people using a 3G device which would increase this delay even more.

Plus you haven't factored in network congestion and a timeout error - which October doesn't have.

Then lastly you said:

I think I would want to see some more compelling evidence that it is widely affecting people's load times.

I don't think anyone is going to be bothered to create a timing disgram and add to this github issue.

Hence, why I said I've given up! I fully understand why October takes forever to sort out anything now.

[bennothommo]

@ayumihamsaki I don't want to come across as combative, if I do, I apologise - I, and I'm sure the other maintainers, are very appreciative of the work you are doing. I am just simply saying that if we are to go ahead with removing a feature that has been in there for a long time, that we do it for the right reasons. I'm simply saying that in the scenario of it slowing down "local" installs - this is, I would say, an uncommon scenario as most people use the website for web-accessible sites and systems.

The other suggestions you've made in this issue, like with the plugin icons, are likely good ideas that we can investigate further on.

Regarding:

I fully understand why October takes forever to sort out anything now.

One of the things that initially convinced me on using this project was the measured and careful approach the maintainers took in implementing new features and changes, and ensuring that the breaking changes that were introduced by third-party libraries were mitigated as much as possible. Now that I am part of that team, I want to continue that. It probably comes across as slow, but I feel that's an admirable goal to achieve.

[ghost]

@bennothommo

I don't want to come across as combative, if I do, I apologise

No, just negative all the time! Maybe you scared of change?

Plus the issue is not about local installs it's about reducing the load time of a web page. Removing 400mS to a rendering of a webpage is pretty black and white to most developers! It's up to you if you enjoy slow loading performance.

Maybe try running October of a mobile device with a 3G connection. Oh wait it can't be done as October doesn't work on below 768px screen sizes. But wait no one is complaining about it so it must be fine.

The truth is many people can't be bothered to make an issue in this repo! Luke will tell them to not create a github issue unless they are going to create a pr - so that scares off many non-technical people.

Github issues get closed down all the time here and so many things don't get fixed. That's not how to do things and you know it. You don't close down issues due to lack of staff and available commitment time.

Many github issues take years here and even when a pr fix is done just sits there for ages. There are loads of pr's sitting there for over 15 months. What an insult to the people who coded them.

When I open a github suggesting how to speed up all these issues and 20 - 30 people have given it their thumbs up and many people say they see the same thing! Then Luke closes the post after a month is kind of rude and just says to people we have no intention of trying to improve.

Seriously do this:

1. Stop closing github issues - just because no one is fixing the problem.

2. Stop saying no to everything! Also stop changing your mind all the time! One minute you say you want to make October easy for people - next minute you are doing the complete opposite! e.g. the .apng format. Does that make it easy for non-techie people what you said. No it's the complete opposite saying to create a config file. So now one of my clients has to learn to write a config file to use the media section - surely you must see that's the complete opposite of you saying to make October simple for people?

3. This project is all messed up! Any enhancements the admins have the only say. It doesn't matter about the community! Personally I wouldn't be suprised if I hard-forked this project and then said everyone you can submit any bug issue - it won't get closed down. You can submit any enhacement idea - it won't get stone-walled by admins! Instead the people can vote for enhacement ideas they like and everything is running from a de-centralized community.

4. This project is almost dead, there is not one full-time person or any team properly developing it. It more of a taking liberty where the admins try and get other people to fix the problems.

[LukeTowers]

@ayumihamsaki I think you need to take a deep breath and calm down.

In regards to this specific issue, I have run into it myself before and I agree the current situation can be improved. We won't be removing gravatars outright because there are other solutions that can work better without being a breaking change in behaviour. For example, we could easily just attempt to retrieve the user's Gravatar in the User model's afterSave() event and pull it down locally and attach it as a regular avatar image at that point. We could also make it so that if no local avatar is found we use the data URI (or even just include that MM image as a static file in the repo). This would solve all of your issues with the avatar as it currently is (performance, poor / no network connection, tracking, etc) while retaining its functionality.

In regards to your specific comments:

No, just negative all the time! Maybe you scared of change?

Nobody here is scared of change. We have a responsibility to ~50,000 users (end users and developers) to maintain a stable product that they can count on to serve their businesses and goals reliably. As such, any change that is made to existing behaviour will require discussion and agreement amongst the maintainers before we decide to implement it and push it out to all of our users. Yes, this means that we will push back on suggestions we feel could be improved or don't fit the project's goals, and it means that progress will be slower than you might want; but we are never going to reject changes because we're "scared of change".

The truth is many people can't be bothered to make an issue in this repo! Luke will tell them to not create a github issue unless they are going to create a pr - so that scares off many non-technical people.

I only tell people to not create an issue without a PR if it is for a feature request, as there is simply too many other requests and legitimate issues already for us to ever consider a random feature request in an issue for inclusion. If someone wants a feature included in October I encourage them to put their money where their mouth is (so to speak) by either offering to sponsor the development of the feature or develop it themselves. As for my specific requests for your issues, you are a developer and there is no need for you to be opening issues and PRs within minutes of each other when you already have a proposed solution to the problem.

Github issues get closed down all the time here and so many things don't get fixed. That's not how to do things and you know it. You don't close down issues due to lack of staff and available commitment time.

Closing issues is a tool we use to focus our time. Just because an issue has been closed doesn't mean that it will never get addressed or looked at, it just means that it's not going to get dealt with at the moment (usually due to a lack of interest from anyone).

Many github issues take years here and even when a pr fix is done just sits there for ages. There are loads of pr's sitting there for over 15 months. What an insult to the people who coded them.

There are 252 open issues and 61 open PRs right now, out of a total of ~4,600. This is just in this one repo, we also maintain all of the RainLab plugins, the Library repo, the docs repo, and our own personal open source projects. This also doesn't account for all the time spent on user support in Slack, IRC, the forums, and StackOverflow. We currently have three active maintainers (Myself, @bennothommo, & @w20k) and @daftspunk to release builds. Not one of us is paid for the time we spend on this project (I do get a share of the marketplace revenues, which over the past year has amounted to $3,337 total - however given that I put a minimum of 10 hours a week into this project that works out to roughly $6/hr at most, when my regular rate is $150/hr), and all of us have our own full time jobs to support us.

As such, we operate on the principal that "the squeaky wheel gets the grease". If issues / PRs are just sitting open and nobody has checked in on them, then we take that to mean that the issue isn't important enough for someone to spend any of their time pushing through so why should we bother spending any of our limited time reviewing it when there are so many other issues clamouring for attention.

When I open a github suggesting how to speed up all these issues and 20 - 30 people have given it their thumbs up and many people say they see the same thing! Then Luke closes the post after a month is kind of rude and just says to people we have no intention of trying to improve.

20 - 30 people giving 2 seconds of their time to click a button on a website doesn't mean that we're obligated to drop everything and look into whatever it is that's being reported. If it's truly an issue that affects your business (and this goes to all 30 people interacting with the post) then do something about it. Either pay us for support or contribute to the solution, but just sitting there grumbling that we're not solving your problems for you is not going to get anything done. October is free, you are earning an income off of the countless hours and hard work that has been put into this project, so if something is a problem for you pitch in to get it fixed. I say this not just to you (you have been pretty good about investing your own time providing solutions to problems that you have) but to those 30 people clicking the button.

Stop closing github issues - just because no one is fixing the problem.

Not going to happen. Like I said before, closing issues is not a permanent discarding of the issue, it's a way for us to focus our time and attention on issues that matter enough to people for them to get involved. If an issue becomes active again then we have no problem reopening it.

Stop saying no to everything!

No! 😝 In all seriousness though, there are a lot of factors that we have to consider when making our decisions. In this specific example (apng) that you are complaining about here, we're not going to commit to maintaining such a change in the core when a) the support / adoption of the feature is very limited right now, and b) there exists a simple work around for anyone that actually needs the feature right now.

So now one of my clients has to learn to write a config file to use the media section - surely you must see that's the complete opposite of you saying to make October simple for people?

No, your client does not have to learn to write a config file. That's your responsibility to do for them to deliver a product that's customized to their needs. That's why they pay you instead of just using October by themselves.

This project is all messed up! Any enhancements the admins have the only say. It doesn't matter about the community!

Yes, we the maintainers are the BDFLs and it's going to stay that way for the forseeable future. Why should I bow to the demands of the "community" (really a vocal minority) when they aren't the ones that are going to be stuck dedicating their time to supporting the features they demanded? They don't have any skin in the game so to speak, it's easy for someone to add their ideas and opinions to something (which I do appreciate), but it will still come down to us to make the decision whether something gets included or not because we are the ones that it will affect down the road when the people clamouring for it have moved on but we're stuck supporting it. I say again, if someone wants something, then contribute towards making that a reality. Either contribute code, your time, or your money; but any way you look at it we're not obligated to implement every request that comes our way.

Personally I wouldn't be suprised if I hard-forked this project and then said everyone you can submit any bug issue - it won't get closed down. You can submit any enhacement idea - it won't get stone-walled by admins! Instead the people can vote for enhacement ideas they like and everything is running from a de-centralized community.

The project is open source, you're welcome to do whatever you want to do. I think that would probably be a good learning experience for you to figure out what's actually involved with being a maintainer of a popular open source project. People can vote all they like, but unless work is actually put in by the people that care about the issue it's just empty discussions.

This project is almost dead, there is not one full-time person or any team properly developing it. It more of a taking liberty where the admins try and get other people to fix the problems.

This project is most certainly not dead. You seem to be under the impression that the support level received should be on par with paid products provided by companies. If you want that level of support, then pay for it. We're more than happy to offer official support contracts to those that are willing to pay for it. This is an open source project built and maintained by volunteers, that's just how it is right now.

[SebastiaanKloos]

I have to say I understand what @LukeTowers is saying. And also understand some of @ayumihamsaki points.

I think most people think that an issue is rejected when an issue is closed.

I’m still learling about OctoberCMSs core. When I think I’m good enough to help resolve issues, I will help fixing bugs and help improving the core.

For everybody who benefits from OctoberCMS and is technical enough to contribute, I would like to invite to take a look at this website:
https://opensourcefriday.com

[ghost]

@LukeTowers @bennothommo Thanks Luke for taking the time to go through my points and sorry if I was rude in anyway.

I currently have 368 commerical websites running October CMS and ever since 2015 I've had a huge amount of complaints from my clients about October CMS. The worst thing Daftspunk did was removing the beta flag. October is fine for people wanting to create personal websites but in a commerical enviroment it totally sucks.

October is very buggy right now! You may not think it is, but all the feedback and constructive complaints I get from my clients all the time, tells a completely different picture!

One example, the CSP issue found here: https://github.com/octobercms/october/issues/3608

The clients website is in europe! Websites in europe have a GDPR law. The GDPR plugin we built uses October AJAX. The problem is all our websites have high security features - we have totally spent many hours hardening October! Due to October AJAX using eval. The CSP blocks the GDPR plugin and a user can not give a consent option. So we have a choice either to remove the security features or the privacy features. Of course we choose to keep the security features! The goverment made a complaint against that website company and contacted me and I wrote them a legal letter showing them proof that it's October's fault and the bug is not fixed.

Both myself, my company I work for and the clients we been waiting months for this to get fixed and nothing happens!

The reason why I report so many bugs is not from me - but from my clients. I have said to them there are no paid admins here and everything is slow. I told them I would file the issues they raise and try and do the pr's myself. Hence why you see so many issues I have patched.

You need understand our clients and other clients to other companies using October are all saying the same thing! Either fix these problems or remove October and switch to another CMS. This is what we hear everyday year after year.

It's highly annoying trying to patch October to a stable level that's worth my clients time and money.

So understand where I am coming from, please make a plan to speed up these issues and help out professional companies that are trying their best to use your product in a commerical enviroment.

Truthfully I got around another 30 major issues I need to file and fix in October. I'm trying to space them out to help you guys out. But it would be nice if their was more support testing these pr's and proper feedback so I can adjust them quickly.

[LukeTowers]

@ayumihamsaki if you have 368 clients running October with high demands of the platform then I highly recommend you either get them or your company to pay us for official support. A mere $100 / year from each site would add up to $36,800 / year for me to dedicate to solving those issues for your clients. Changing platforms is expensive, so is dealing with these sorts of issues. I would say that your specific case is one that would greatly benefit from a paid support contract. Shoot me an email at [email protected] if you'd like to discuss it further.

I can guarantee you that if your clients or company are willing to pay for our official support then your issues would be solved in a timely matter. Otherwise, it's entirely reliant on us volunteering our time to solve your client's problems for free.

[ghost]

@LukeTowers Thanks I will recommend to them and my manager about that, I didn't even think we could do that. Definitely the way to go!

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see action on it, please respond and we will get the ball rolling.

[LukeTowers]

@ayumihamasaki2019