Prestashop: Cart / Checkout performance issue for multiple products in basket

[groomershop]

Todo

• [ ] Analyze where the problem comes from
• [ ] Find the appropriate solution and re-qualify the issue

Describe the bug
After adding for example 50 diffrent products to cart (we are a B2B wholesaler, so it's typical for us) the load time of any site (with product, catalog, adding next product, going to checkout) increases significantly (from 200-300ms to about 800ms - on clean install at empty 6-core server).

We have turned profiling and the problem is with amount of MySQL queries, which for 50 products raise to over 2000 and query time about 250ms.

To Reproduce

1. Clean install PrestaShop 1.7.6.
2. Put sample products.
3. Add diffrent products to basket - from 20-30 the speed goes down, about 50 it's slow, when more is going dramitaly slow.
4. Turn on profilling and you will see how much the queries number has increased.

Screenshots
Screenshot from profiler in attachment

Additional information
PrestaShop version: 1.7.6
PHP version: 7.2.20

[khouloudbelguith]

Hi @groomershop,

Thanks for your report.
I manage to reproduce the issue with PS1.7.6.0 & PS1.7.5.1.
With PS1.7.6.0

• Load Time | 2767 ms
• Stopwatch SQL - 1796 queries

With PS1.7.5.1
Load Time 1491
Stopwatch SQL - 2157 queries

I’ll add this to the debug roadmap so that it’s fixed. If you have already fixed it on your end or if you think you can do it, please do send us a pull request!
Thanks!

[groomershop]

Ok, great. We are rather unable to solve this - we are working on migration from another platform to PrestaShop and this was one of our tests before migration. We plan to migrate in September, so there is time to solve this before our production start.

[colinegin]

Hello @PrestaShop/prestashop-contributors , what do you think of this issue ? Can it be fixed easily ?

[matks]

@colinegin It requires to explore the queries being done (= needs technical analysis) but it's not an easy pick. PrestaShop is resources-hungry so queries pile up when a lot of data is required.

[colinegin]

Hello @matks , thanks for your feedback :)
Alright then, i add it to the to do pile, but it won't be prioritized for 1.7.7 as we lack time.

[mnowakpcgs]

I would strongly disagree that this is not a priority.

Please note you are seeing this degradation on a clean install and a dev server with no traffic.

Add in some modules and overrides and 100 active users, and it is amplified significantly.

I've been dealing with this ever since migrating to 1.7.x and was shocked that nobody else had experienced this problem. I tried hiring Bruno Leveque to help fix it but his solution was upgrading from 1.7.2.x to 1.7.6.x. After having to spend a lot of time fixing modules and overrides that broke with the upgrade, the problem remained.

At this point I would be happy to pay a dev myself to finally fix this nonsense. Prestashop is a shopping cart. This is literally its primary job. This is what it should do best, and ever since 1.7 it is failing spectacularly. Please help fix this.

[kpodemski]

Hey @mnowakpcgs

could you tell me more about your issue?

how big are customer carts?
is your store with a lot of combinations?
have you tried to profile performance issues like people before you in this thread?

[mnowakpcgs]

Most customers only order a handful of items, but the cart is sluggish after only a few items.

We have a few wholesale customers and that becomes unbearable. If they add say 30 different items, it is incredibly slow. It gets to the point it times out and they can't even checkout.

Yes, I have tried.

[mnowakpcgs]

At this point I am investigating backporting everything to 1.6

[arirangz]

Hello,

I don't know if it's related but I have a similar problem on :
Presashop 1.7..5
Tested on both PHP 7.1 or PHP 7.3
All module disabled
Override disabled
Default theme

As soon as I have one product in the cart, the pages take much more time to load : from 300ms to 3000ms+

Before adding a product :

After adding a product :

Do you have any idea from where this can come from?

Thanks

[mnowakpcgs]

It was even worse for me, it would increase exponentially like that for
each additional item to the point that 20+ items in cart would start to
time-out and checkout was impossible.

I don't know. The advice I've received for years was to go back to 1.6. I
had skipped straight from 1.4.4.0 to 1.7.1.2 so its been a lot of work
migrating overrides, modifications, and modules, but I'm 90% done and
its MUCH faster. I do not think I will regret it.

Mark Nowak
President
Purecigs LLC
3135 Skyway Circle
Melbourne, FL 32934
m: 321.292.2220
e: [email protected]
www.purevape.com http://www.purevape.com | www.eliquid-depot.com
http://www.eliquid-depot.com | www.blaze-cbd.com
http://www.blaze-cbd.com
https://www.purevape.com http://www.blaze-cbd.com

On Sat, May 30, 2020 at 3:57 PM arirangz [e-mail] wrote:

Hello,

I don't know if it's related but I have a similar problem on :
Presashop 1.7..5
Tested on both PHP 7.1 or PHP 7.3
All module disabled
Override disabled
Default theme

As soon as I have one product in the cart, the pages take much more time
to load : from 700ms to 3000ms+

Before adding a product :
[image: Capture d’écran 2020-05-30 à 22 52 38]
https://user-images.githubusercontent.com/1193727/83338879-b5793580-a2c8-11ea-9d69-946f1d9f8a65.jpg

After adding a product :
[image: Capture d’écran 2020-05-30 à 22 53 14]
https://user-images.githubusercontent.com/1193727/83338881-bf9b3400-a2c8-11ea-938e-fcd52223f174.jpg

Do you have any idea from where this can come from?

Thanks

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-636384588,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZX2NTJDRNJJCQ7552TRUFXMXANCNFSM4IJDO6KQ
.

[arirangz]

Thanks for your feedback. From my side it's an update from 1.7.3 to 1.7.6.

I double check and adding more products is not increasing the slowdown.

The difference is only from 0 product in cart to 1 or more product in cart.

Any progress from Prestashop team on this?

Thanks

[marcelosi]

Hello, is there a solution to this problem?

in my clothing store, the normal is 40/50 online users and the consumption of the 8-core server is minimal (attached image)

At times of new releases, there may be 150/200 users online. For wholesalers, the minimum order is 20 units. At that moment the server consumption shoots up, even collapsing the server with only 12 users with orders of more than 20 units (attached image). I have had many times the web without answer because of these few carts of many units.

Prestashop 1.7.6.3
PHP 7.2.31
8-core 32gb ram, SSD and Litespeed VPS server

DEBUG PROFILING TRUE
No products in cart 126 queries
cart with 1 product, 222 queries
cart with 10 different products 556 queries

Is there any improvement to reduce the number of database queries with carts of many products?

[mnowakpcgs]

What shipping modules do you have?

Although I am in the middle of downgrading to 1.6 which itself has helped
immensely, I did finally find a "fatal flaw". PrestaShop team persued
functionality to be able to split ship orders, but the end result is that
it loops through shipping options for every item in cart. Caching results
from the shipping API helps and the modules mostly all do that, but what
they don't all do is cache when a shipping option is NOT available. Let's
say FedEx Ground Home Delivery is available, but regular FedEx Ground is
not. Both the "fedexpro" and presto-changeo modules will not cache the
unavailable FedEx ground option and thus the cart will loop and request
from the FedEx API dozens of times every time a page is loaded.

On Sun, Jun 14, 2020, 3:57 PM marcelosi [e-mail] wrote:

Hello, is there a solution to this problem?

in my clothing store, the normal is 40/50 online users and the consumption
of the 8-core server is minimal (attached image)

At times of new releases, there may be 150/200 users online. For
wholesalers, the minimum order is 20 units. At that moment the server
consumption shoots up, even collapsing the server with only 12 users with
orders of more than 20 units (attached image). I have had many times the
web without answer because of these few carts of many units.

Prestashop 1.7.6.3
PHP 7.2.31
8-core 32gb ram, SSD and Litespeed VPS server

DEBUG PROFILING TRUE
No products in cart 126 queries
cart with 1 product, 222 queries
cart with 10 different products 556 queries

Is there any improvement to reduce the number of database queries with
carts of many products?
[image: cpu-day]
https://user-images.githubusercontent.com/66922989/84605042-805d0d80-ae9a-11ea-93bc-49e3ced70b8e.png
[image: cpu-daynormal]
https://user-images.githubusercontent.com/66922989/84605044-818e3a80-ae9a-11ea-8931-e51c2d64ba86.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-643827525,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZQDLUAR5XFRMI27UEDRWVBUFANCNFSM4IJDO6KQ
.

[mnowakpcgs]

Forgot to mention.. for the excessive queries this may also be the same
problem. Some of the shipping modules cache rates in SQL but not with the
available built in caching functions. So again it loops through shipping
rates for each product in cart and can result in a ton of SQL queries for
the cached results what could easily have been stored in a global variable
instead.

Solving both these issues have brought my cart load time down to 1-2s max
on my current dev server whether it's 1 item or 100 items in cart.

On Sun, Jun 14, 2020, 3:57 PM marcelosi [e-mail] wrote:

Hello, is there a solution to this problem?

in my clothing store, the normal is 40/50 online users and the consumption
of the 8-core server is minimal (attached image)

At times of new releases, there may be 150/200 users online. For
wholesalers, the minimum order is 20 units. At that moment the server
consumption shoots up, even collapsing the server with only 12 users with
orders of more than 20 units (attached image). I have had many times the
web without answer because of these few carts of many units.

Prestashop 1.7.6.3
PHP 7.2.31
8-core 32gb ram, SSD and Litespeed VPS server

DEBUG PROFILING TRUE
No products in cart 126 queries
cart with 1 product, 222 queries
cart with 10 different products 556 queries

Is there any improvement to reduce the number of database queries with
carts of many products?
[image: cpu-day]
https://user-images.githubusercontent.com/66922989/84605042-805d0d80-ae9a-11ea-93bc-49e3ced70b8e.png
[image: cpu-daynormal]
https://user-images.githubusercontent.com/66922989/84605044-818e3a80-ae9a-11ea-8931-e51c2d64ba86.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-643827525,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZQDLUAR5XFRMI27UEDRWVBUFANCNFSM4IJDO6KQ
.

[arirangz]

From my side, I tested with all modules disabled and the issue is the same.

[marcelosi]

Thank you mnowakpcgs for your answers.

I have done a new installation of Prestashop 1.7.7 without installing any module and with the default data and the problem still persists:

Prestashop 1.7.7 Beta, new installation
PHP 7.2.31
8-core 32gb ram, SSD and Litespeed VPS server

Blank page - DEBUG PROFILING TRUE
No products in cart 79 queries
cart with 1 product, 141 queries
cart with 10 different products 427 queries

the queries are multiplied with each product in the cart.

How can this be optimized?

[mnowakpcgs]

I think unfortunately there is no answer. Prestashop just makes excessive
queries. It is a significant endeavor to correct this, which I myself lack
the time for. I am running on AWS and my database clusters are
significantly disproportionate to my EC2 instances.

Mark Nowak
President
Purecigs LLC
3135 Skyway Circle
Melbourne, FL 32934
m: 321.292.2220
e: [email protected]
www.purevape.com http://www.purevape.com | www.eliquid-depot.com
http://www.eliquid-depot.com | www.blaze-cbd.com
http://www.blaze-cbd.com
https://www.purevape.com http://www.blaze-cbd.com

On Sun, Jun 21, 2020 at 11:49 AM marcelosi [e-mail] wrote:

Thank you mnowakpcgs for your answers.

I have done a new installation of Prestashop 1.7.7 without installing any
module and with the default data and the problem still persists:

Prestashop 1.7.7 Beta, new installation
PHP 7.2.31
8-core 32gb ram, SSD and Litespeed VPS server

Blank page - DEBUG PROFILING TRUE
No products in cart 79 queries
cart with 1 product, 141 queries
cart with 10 different products 427 queries

the queries are multiplied with each product in the cart.

How can this be optimized?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-647153021,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZWTT6OWN2ZA73S2VLLRXY2Y7ANCNFSM4IJDO6KQ
.

[matks]

Hi there,

I just wanted to say that the fact this issue is not fixed yet (or in a fixed timeframe) does not mean that it won't be fixed, or that we don't care. It doesn't mean that the issue is NOT important either. It just means that other issues have been considered, for whatever reason, _more_ important, so they were (or are being) tackled first. There's an immense amount of work to be done (784 issues right now, and growing!), and tasks take time.

Developer time is limited, so if the amount of issues opened in a certain timeframe exceeds the amount of issues that can be processed using the available developer time, tasks will simply pile up. This is common to ALL projects. Due to the number of pending issues, sometimes they may get lost in the pile and be forgotten about, so it's not a bad idea to ping us there from time to time to remind us about it, even if it's been a long while 😄

_But here's the upside:_ PrestaShop is an open source, community project 🚀 . If you find that any issue is critical for you, and it's important to have it fixed ASAP, then you can _invest_ into having it done just like @mnowakpcgs said . You can hire a developer to fix it, or if you are a developer yourself, you can try doing it on your own. The most important thing is to share that fix with everyone by submitting a Pull Request -- that's what the open source spirit is all about.

This project grows because everybody contribute to it, not only PrestaShop company 😉 . Let's say that we consider the 20 most important known bugs in the backlog. If 20 companies using PrestaShop decide to pick one, and hire a dev to fix it, then submit the Pull Request, then at the end of this process each companies gets 20 bug fixes ... for the cost of fixing one ! That would be an awesome demonstration of the strength of open source.

The same idea works for features by the way 😄 if 20 companies using PrestaShop decide to implement one amazing feature, each of them, and every company submits a Pull Request, at the end of the process the project has grown with 20 features but each of the participants only invested time/money for one. That's quite a good ROI don't you think 😉 ?

If however no PR is done by the community, it is likely this issue will be fixed by PrestaShop company at some point. However it will be fixed following the priority decided by the Product team who analyze and triage all incoming bug reports.

[mnowakpcgs]

Hah, I wouldn't cite me when my solution was listening to the advice I've
been given repeatedly and backport to 1.6 codebase.

Don't get me wrong, I've been a prestashop supporter since 2010 and started
on 1.3 and 1.4, then migrated directly to 1.7 from there. But the switch to
symfony has been terrible. I basically learned to code only because of
prestashop, and symfony has made it significantly more difficult to work
with. At some point after I get this project stable and in production I
will try to chip away at the excessive queries issue, but on my 1.6
codebase it would not save and significant load time and might just save a
few hundred dollars using a slightly smaller RDS instance.

If you want people like me contributing again, give up this symfony
bullshit.

Mark Nowak
President
Purecigs LLC
3135 Skyway Circle
Melbourne, FL 32934
m: 321.292.2220
e: [email protected]
www.purevape.com http://www.purevape.com | www.eliquid-depot.com
http://www.eliquid-depot.com | www.blaze-cbd.com
http://www.blaze-cbd.com
https://www.purevape.com http://www.blaze-cbd.com

On Mon, Jun 22, 2020 at 7:26 AM Mathieu Ferment [e-mail]
wrote:

Hi there,

I just wanted to say that the fact this issue is not fixed yet (or in a
fixed timeframe) does not mean that it won't be fixed, or that we don't
care. It doesn't mean that the issue is NOT important either. It just means
that other issues have been considered, for whatever reason, more
important, so they were (or are being) tackled first. There's an immense
amount of work to be done (784 issues right now
https://github.com/PrestaShop/PrestaShop/issues?q=is%3Aopen+is%3Aissue+label%3ABug,
and growing!), and tasks take time.

Developer time is limited, so if the amount of issues opened in a certain
timeframe exceeds the amount of issues that can be processed using the
available developer time, tasks will simply pile up. This is common to ALL
projects. Due to the number of pending issues, sometimes they may get lost
in the pile and be forgotten about, so it's not a bad idea to ping us there
from time to time to remind us about it, even if it's been a long while 😄

But here's the upside: PrestaShop is an open source, community project
🚀 . If you find that any issue is critical for you, and it's important
to have it fixed ASAP, then you can invest into having it done just
like @mnowakpcgs https://github.com/mnowakpcgs said . You can hire a
developer to fix it, or if you are a developer yourself, you can try doing
it on your own. The most important thing is to share that fix with everyone
by submitting a Pull Request -- that's what the open source spirit is all
about.

This project grows because everybody contribute to it, not only PrestaShop
company 😉 . Let's say that we consider the 20 most important known bugs
in the backlog. If 20 companies using PrestaShop decide to pick one, and
hire a dev to fix it, then submit the Pull Request, then at the end of this
process each companies gets 20 bug fixes ... for the cost of fixing one !
That would be an awesome demonstration of the strength of open source.

The same idea works for features by the way 😄 if 20 companies using
PrestaShop decide to implement one amazing feature, each of them, and every
company submits a Pull Request, at the end of the process the project has
grown with 20 features but each of the participants only invested
time/money for one. That's quite a good ROI don't you think 😉 ?

If however no PR is done by the community, it is likely this issue will be
fixed by PrestaShop company at some point. However it will be fixed
following the priority decided by the Product team who analyze and triage
all incoming bug reports.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-647484579,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZXR7OQFVNDR4Z6A7ITRX5EW7ANCNFSM4IJDO6KQ
.

[matks]

@mnowakpcgs Do you know we are fully aware of the impression that "Symfony is bullshit" people have ? 😉

Yes, check out this article we wrote: https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-3-the-future-architecture/
=> "many people are convinced that the Symfony migration was a waste of time and money"

All the answers you are looking for (why Symfony ? why does it seem so complex ? why the issues ?) are provided in this series of article. I strongly recommend reading them.

• https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-1-current-architecture/
• https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-2-pain-points/
• https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-3-the-future-architecture/

[mnowakpcgs]

I have read that before. While the reasoning sounds compelling "on paper",
in practice prestashop has spent years working on that while doing nothing
to improve the actual user experience. This excessive query issue that
slows down the cart for example. Or the very user unfriendly checkout that
remains unchanged since 2009. The one thing I'll miss from 1.7 is the
product admintab. I do not miss symfony one bit. I hope you guys realize
you should put some focus on improving the cart's actual main purpose, and
that this theory sounded great but perhaps symfony was not the way to do
it. If you think the code is unclean, then clean it up. You do not always
have to throw everything away and start over, especially building it on a
generalize framework when you had customized code.

Mark Nowak
President
Purecigs LLC
3135 Skyway Circle
Melbourne, FL 32934
m: 321.292.2220
e: [email protected]
www.purevape.com http://www.purevape.com | www.eliquid-depot.com
http://www.eliquid-depot.com | www.blaze-cbd.com
http://www.blaze-cbd.com
https://www.purevape.com http://www.blaze-cbd.com

On Tue, Jun 23, 2020 at 2:06 AM Mathieu Ferment [e-mail]
wrote:

@mnowakpcgs https://github.com/mnowakpcgs Do you know we are fully
aware of the impression that "Symfony is bullshit" people have ? 😉

Yes, check out this article we wrote:
https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-3-the-future-architecture/
=> "many people are convinced that the Symfony migration was a waste of
time and money"

All the answers you are looking for (why Symfony ? why does it seem so
complex ? why the issues ?) are provided in this series of article. I
strongly recommend reading them.

-
https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-1-current-architecture/
-
https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-2-pain-points/
-
https://build.prestashop.com/news/prestashop-in-2019-and-beyond-part-3-the-future-architecture/

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-647952499,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZQJFYAIWT3KCS42MJLRYBH6PANCNFSM4IJDO6KQ
.

[colinegin]

Hello @mnowakpcgs ,

Thanks for your feedbacks. I'm a bit surprised with your comment saying that the checkout remains unchanged since 2009, while the whole front office has been fully reworked with 1.7.
If you have some time, I would be very interested to get more information about it with a comment on our checkout EPIC : https://github.com/PrestaShop/PrestaShop/issues/17034.

Thanks !

[arirangz]

@matks Thanks for your feedback.
Do you have any idea from where this query increase would be coming from?
This could help us (developers) to debug and try to improve the code.
I think it's quite a big issue as if a customer is adding even only one product and continue shopping, the client is experiencing a much slower navigation which can lead to cart abandon.
Thanks

[mnowakpcgs]

I'm not sure how you say that, its barely a different checkout process in
1.7 than 1.4. I'd be happy to share what I did if you have somewhere to
send screenshots.

I took inspiration from shopify for a more streamlined experience, and to
remove unnecessary customer inquiries. For example, dropdowns to select
addresses which fill the form fields. if customer changes anything, update
or create new address seamlessly when they click button for next step. Do
not show billing address or checkbox for different billing address, it
seems to just confuse some people. Shipping page includes a map showing
where its going for customers to better understand estimated delivery. I
also have the carrier modules passing along actual estimated shipping times
and the shipping step shows estimated delivery date. Payment page has cc
form embedded and dropdown selection for billing address as well. If they
edit the form fields that would at that point be pre-fileld with the
shipping address, it will create a new address. Order confirmation page
overridden to be viewed repeatedly at any time to see updates. Linkn
directly to that confirmation page is sent in the confirmation email so
they can come back and easily see updates without logging in or anything
(secure key in link). The email is updated to resemble the confirmation as
well, which both again take cues from shopify with the delivery route map
again, etc.

Mark Nowak
President
Purecigs LLC
3135 Skyway Circle
Melbourne, FL 32934
m: 321.292.2220
e: [email protected]
www.purevape.com http://www.purevape.com | www.eliquid-depot.com
http://www.eliquid-depot.com | www.blaze-cbd.com
http://www.blaze-cbd.com
https://www.purevape.com http://www.blaze-cbd.com

On Wed, Jun 24, 2020 at 2:57 AM colinegin [e-mail] wrote:

Hello @mnowakpcgs https://github.com/mnowakpcgs ,

Thanks for your feedbacks. I'm a bit surprised with your comment saying
that the checkout remains unchanged since 2009, while the whole front
office has been fully reworked with 1.7.
If you have some time, I would be very interested to get more information
about it with a comment on our checkout EPIC : #17034
https://github.com/PrestaShop/PrestaShop/issues/17034.

Thanks !

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-648660904,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APD3JZWRXFFC3KW736A3EMTRYGWV5ANCNFSM4IJDO6KQ
.

[ghost]

I can confirm that the direct Checkout button on PS 1.7.6.5 is taking forever to create the checkout forms and write the abandoned cart/ order (30-60seconds) Also, I get another problem which I don't know if it is related: PayPal official module leaves the status "Awaiting for PayPal Payment" instead of "Payment Completed"

[ghost]

Just learned that using Chaching (CacheApc) was slowing down the Checkout page a lot (30s) as opposed to 15s without caching enabled

[arirangz]

Hi,
All the shop navigation is slower as soon as there is one product in the shopping cart.
For example, when switching from one combination to another, the stock message is updated after 0.5sec but as soon as there is one product in cart, it takes at least twice (1.2sec).

Is Prestashop team working on this performance issue?

Thanks