Pim-community-dev: Completeness taking too long on save

[wakqasahmed]

I'm reporting a Bug

CE 1.6.6

Product saving takes quite a lot of time and specifically "completeness computation" takes around 12.5 sec on an average, as shown in the attached screenshot.

[LaureBrosseau]

Hello @wakqasahmed,
Thanks for reporting this.
This issue is similar to #3771, and the good news is that we are currently working on a fix (from 1.5 version for ODM and ORM).
I'll keep you updated, our internal ticket reference is PIM-6429.
Thanks!

[ahocquard]

Hello @wakqasahmed,

Unfortunately, I don't think that the fix on the completeness widget will resolve your problem.

1. Could you tell me:
2. your volumetry

3. orm/mongo

4. Also, did you take a look at the status of the running request ? Do you have any row lock or something like that that could explain that time.

The best thing to do would be to isolate the request causing the problem and then try to understand why it takes so much time.

1. Did you have that problem in a previous version or is it since a patch ?

Thanks.

[wakqasahmed]

[Edition]
CE

[Version]
1.6.6

[Environment]
prod

[Install time]
2016-11-16T14:28:53+04:00

[Server version]
Apache/2.4.7 (Ubuntu)

[Number of channels]
10

[Number of locales]
2

[Number of products]
3768

[Number of attributes]
2541

[Number of families]
60

[Number of users]
13

[OS version]
Linux prod-pim 4.4.0-64-generic #85~14.04.1-Ubuntu SMP Mon Feb 20 12:10:54 UTC 2017 x86_64

[PHP version]
7.0.19-1+deb.sury.org~trusty+1

[PHP extensions]
Core
date
libxml
openssl
pcre
zlib
filter
hash
Reflection
SPL
session
standard
apache2handler
imagick
mysqlnd
PDO
xml
apcu
apc
calendar
ctype
curl
dom
mbstring
fileinfo
ftp
gd
gettext
iconv
intl
json
exif
mcrypt
mongodb
mysqli
newrelic
pdo_mysql
Phar
posix
readline
shmop
SimpleXML
sockets
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xmlreader
xmlwriter
xsl
zip
Zend OPcache

[Registered bundles]
APY\JsFormValidationBundle\APYJsFormValidationBundle
Akeneo\Bundle\BatchBundle\AkeneoBatchBundle
Akeneo\Bundle\BufferBundle\AkeneoBufferBundle
Akeneo\Bundle\ClassificationBundle\AkeneoClassificationBundle
Akeneo\Bundle\FileStorageBundle\AkeneoFileStorageBundle
Akeneo\Bundle\MeasureBundle\AkeneoMeasureBundle
Akeneo\Bundle\StorageUtilsBundle\AkeneoStorageUtilsBundle
Dglab\Bundle\DigitalAssetBundle\DglabDigitalAssetBundle
Dglab\Bundle\ExtendedAttributeBundle\DglabExtendedAttributeBundle
Dglab\Bundle\ExtendedMeasureBundle\DglabExtendedMeasureBundle
Dglab\Bundle\ExtendedProductBundle\DglabExtendedProductBundle
Dglab\Bundle\ExtendedUIBundle\DglabExtendedUIBundle
Dglab\Bundle\ExtendedUserBundle\DglabExtendedUserBundle
Dglab\Bundle\ExtendedVariantGroupBundle\DglabExtendedVariantGroupBundle
Dglab\Bundle\InstallerBundle\DglabInstallerBundle
Dglab\Bundle\UniversalConnectorBundle\DglabUniversalConnectorBundle
Doctrine\Bundle\DoctrineBundle\DoctrineBundle
Doctrine\Bundle\FixturesBundle\DoctrineFixturesBundle
Doctrine\Bundle\MigrationsBundle\DoctrineMigrationsBundle
Escape\WSSEAuthenticationBundle\EscapeWSSEAuthenticationBundle
FOA\CronBundle\FOACronBundle
FOS\JsRoutingBundle\FOSJsRoutingBundle
FOS\RestBundle\FOSRestBundle
Hoathis\SymfonyRulerBundle\HoathisSymfonyRulerBundle
JMS\SerializerBundle\JMSSerializerBundle
Knp\Bundle\MenuBundle\KnpMenuBundle
Liip\ImagineBundle\LiipImagineBundle
Oneup\FlysystemBundle\OneupFlysystemBundle
Oro\Bundle\AsseticBundle\OroAsseticBundle
Oro\Bundle\ConfigBundle\OroConfigBundle
Oro\Bundle\DataGridBundle\OroDataGridBundle
Oro\Bundle\FilterBundle\OroFilterBundle
Oro\Bundle\FormBundle\OroFormBundle
Oro\Bundle\NavigationBundle\OroNavigationBundle
Oro\Bundle\RequireJSBundle\OroRequireJSBundle
Oro\Bundle\SecurityBundle\OroSecurityBundle
Oro\Bundle\TranslationBundle\OroTranslationBundle
Oro\Bundle\UIBundle\OroUIBundle
Oro\Bundle\UserBundle\OroUserBundle
Pim\Bundle\AnalyticsBundle\PimAnalyticsBundle
Pim\Bundle\CatalogBundle\PimCatalogBundle
Pim\Bundle\CommentBundle\PimCommentBundle
Pim\Bundle\ConnectorBundle\PimConnectorBundle
Pim\Bundle\DashboardBundle\PimDashboardBundle
Pim\Bundle\DataGridBundle\PimDataGridBundle
Pim\Bundle\EnrichBundle\PimEnrichBundle
Pim\Bundle\FilterBundle\PimFilterBundle
Pim\Bundle\ImportExportBundle\PimImportExportBundle
Pim\Bundle\InstallerBundle\PimInstallerBundle
Pim\Bundle\JsFormValidationBundle\PimJsFormValidationBundle
Pim\Bundle\LocalizationBundle\PimLocalizationBundle
Pim\Bundle\NavigationBundle\PimNavigationBundle
Pim\Bundle\NotificationBundle\PimNotificationBundle
Pim\Bundle\PdfGeneratorBundle\PimPdfGeneratorBundle
Pim\Bundle\ReferenceDataBundle\PimReferenceDataBundle
Pim\Bundle\UIBundle\PimUIBundle
Pim\Bundle\UserBundle\PimUserBundle
Pim\Bundle\VersioningBundle\PimVersioningBundle
Pim\Bundle\WebServiceBundle\PimWebServiceBundle
Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle
Sentry\SentryBundle\SentryBundle
Symfony\Bundle\AsseticBundle\AsseticBundle
Symfony\Bundle\FrameworkBundle\FrameworkBundle
Symfony\Bundle\MonologBundle\MonologBundle
Symfony\Bundle\SecurityBundle\SecurityBundle
Symfony\Bundle\SwiftmailerBundle\SwiftmailerBundle
Symfony\Bundle\TwigBundle\TwigBundle
Symfony\Bundle\WorkflowBundle\WorkflowBundle

[Storage]
doctrine/orm

[MySQL version]
mysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $

[ahocquard]

@wakqasahmed

1.6.16 have been tagged today with the fix on the completeness for the widget.

After thinking about your problem, maybe this fix will resolve it. The long insert queries are maybe just locked by the request launched by the widget that is taking too much time, and locking the rows. Can you update in 1.6.16 and tell me if it resolves your problem ?

Thanks.

[wakqasahmed]

sure, i will apply this patch in a while and comment back soon.

[momoss]

Hi @wakqasahmed,

Did you have time to apply the patch ?
Is there any changes?

Thanks!

[ahocquard]

@wakqasahmed , a PR has been done on the calculation of the completeness. Maybe it will resolve your problem because it reduces the level of locking of the table in SQL requests to calculate completeness.

[wakqasahmed]

@ahocquard Fortunately, i didn't get that long time in inserting completeness afterwards (it was for half an hour and restarting apache server did the trick). However, i did apply the patch to speed up widget opening process.

Thanks