Magento2: After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root

[Jilco]

Preconditions

1. Magento 2.2.6
2. Ngingx
3. Redis

Steps to reproduce

1. Upgrade to 2.2.6
2. Switch on reports (or don't switch of if you already switched it on in earlier version)

Expected result

1. I don't realy know, i think the files are needed for advance reports

Actual result

1. In the /tmp/analytics folder in de root of the webserver an huge (1.7 Gb) .tar (~tmp-1537867209.3073tar.tar) file is generated every 24 hours (happend 2 times now after upgrade and exact the same time). Because it is in the tmp folder of the root/webserver Nginx cant make new sessions and the site goes down.

When i look in the .tar (i downloaded it) i see a lot of the same files and the same size. Maybe generating the file goes in a loop somewhere?

[magento-engcom-team]

Hi @Jilco. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• [ ] Summary of the issue
• [ ] Information on your environment
• [ ] Steps to reproduce
• [ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me $VERSION instance

where $VERSION is version tags (starting from 2.2.0+) or develop branches (for example: 2.3-develop).
For more details, please, review the Magento Contributor Assistant documentation.

@Jilco do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• [ ] yes
• [ ] no

[Jilco]

@magento-engcom-team give me 2.2.6 instance

[magento-engcom-team]

Hi @Jilco. Thank you for your request. I'm working on Magento 2.2.6 instance for you

[magento-engcom-team]

Hi @Jilco, here is your Magento instance.
Admin access: https://i-18243-2-2-6.engcom.dev.magento.com/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[Jilco]

Settings in the profided instance are the same as in my situation, but i can't see what happens when generation the report files

[magento-engcom-team]

Hi @engcom-backlog-nazar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• [ ] 6. Add label Issue: Confirmed once verification is complete.

• [ ] 7. Make sure that automatic system confirms that report has been added to the backlog.

[ghost]

HI @Jilco seems like this bug is acknowledged -> https://github.com/magento/magento2/issues/11543

[Jilco]

Thanks! But this is not related, system backup is switched off in my environment

---

Van: engcom-backlog-nazar [e-mail]
Verzonden: woensdag 26 september 2018 09:20
Aan: magento/magento2
CC: Jilco; Mention
Onderwerp: Re: [magento/magento2] After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root (#18243)

HI @Jilcohttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJilco&data=02%7C01%7C%7C90c08bdfa93a48907bad08d623809749%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735432524690309&sdata=%2B0RhrjvU5sB%2F7PS5FUA9m10d4rrc%2BGkrJvwTu2U%2FneE%3D&reserved=0 seems like this bug is acknowledged -> #11543https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F11543&data=02%7C01%7C%7C90c08bdfa93a48907bad08d623809749%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735432524690309&sdata=iLQGJYllfICXYriN%2FWjUqAY4T1b9DLXFQq6%2FAgxjyKw%3D&reserved=0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F18243%23issuecomment-424611780&data=02%7C01%7C%7C90c08bdfa93a48907bad08d623809749%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735432524690309&sdata=fNBTHIZlzChmC%2Fz%2B%2BOoGLzRk3HvKAiOwNIyh3zmAOk4%3D&reserved=0, or mute the threadhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGpw46lMrnK1R6LnQPcPvNEniALuTks5ueyrTgaJpZM4W5L7t&data=02%7C01%7C%7C90c08bdfa93a48907bad08d623809749%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735432524690309&sdata=lRUKky0oB4Pgb3xixn4XIioFNzxqKkvmpvBv1kxTeE8%3D&reserved=0.

[snyderm]

This issue is 2.2.6 specific, and appears to be generated by magento advanced analytics. This issue did not occur in 2.3.

[ghost]

@Jilco As a temporary solution try disable advanced reporting.

[Jilco]

I did and the file is still generated....

Exactly 11.21

---

Van: engcom-backlog-nazar [e-mail]
Verzonden: woensdag 26 september 2018 11:22
Aan: magento/magento2
CC: Jilco; Mention
Onderwerp: Re: [magento/magento2] After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root (#18243)

@Jilcohttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJilco&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=RXUfOhFJuh0Rvrzsf6kruWjxaIX8v%2FN2Sk1%2BHuDYU5E%3D&reserved=0 As a temporary solution try disable advanced reporting.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F18243%23issuecomment-424646813&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=WX0prxdDyJYdgQihbcRSWUN38f1kPCu2FjlQpjfl6sY%3D&reserved=0, or mute the threadhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGomrQzaE3lOJxVpoaeQSs-MiT4aVks5ue0dsgaJpZM4W5L7t&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=0xh6QebGQMe%2BhU%2FmVQoTeOuob1mcBex9yRmYT0Kz4bI%3D&reserved=0.

[Jilco]

Sorry, i disabled reporting not advance reporting. I now did

---

Van: engcom-backlog-nazar [e-mail]
Verzonden: woensdag 26 september 2018 11:22
Aan: magento/magento2
CC: Jilco; Mention
Onderwerp: Re: [magento/magento2] After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root (#18243)

@Jilcohttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJilco&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=RXUfOhFJuh0Rvrzsf6kruWjxaIX8v%2FN2Sk1%2BHuDYU5E%3D&reserved=0 As a temporary solution try disable advanced reporting.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F18243%23issuecomment-424646813&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=WX0prxdDyJYdgQihbcRSWUN38f1kPCu2FjlQpjfl6sY%3D&reserved=0, or mute the threadhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGomrQzaE3lOJxVpoaeQSs-MiT4aVks5ue0dsgaJpZM4W5L7t&data=02%7C01%7C%7C3d29391c223e455b433f08d62391a38f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735505744841578&sdata=0xh6QebGQMe%2BhU%2FmVQoTeOuob1mcBex9yRmYT0Kz4bI%3D&reserved=0.

[ghost]

@Jilco Problem in this file

But i can't understand why this method uses in loop.

[Jilco]

i changed the time and refreshed cache to see if it reoccurs.

Now i see a file of 5 kb and its not growing

---

Van: engcom-backlog-nazar [e-mail]
Verzonden: woensdag 26 september 2018 11:37
Aan: magento/magento2
CC: Jilco; Mention
Onderwerp: Re: [magento/magento2] After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root (#18243)

@Jilcohttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJilco&data=02%7C01%7C%7C627896895e5a4ca47b9008d62393a544%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735514364501917&sdata=VOnFTUchuGMZhA8qABEdHZLepEd156WzyDyF3BfEhjg%3D&reserved=0 Problem in this file
[selection_138]https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F40756439%2F46071438-ae5f6300-c188-11e8-8668-10ca21baead3.png&data=02%7C01%7C%7C627896895e5a4ca47b9008d62393a544%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735514364501917&sdata=Fi8cxwwFEi5DS6gnQDDMuQV4Wpd2jmotx9Gs3Sv%2BRDU%3D&reserved=0
But i can't understand why this method uses in loop.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F18243%23issuecomment-424651183&data=02%7C01%7C%7C627896895e5a4ca47b9008d62393a544%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735514364501917&sdata=mwrV3nxGqytlNkCqcpv5dcRD3WBFqi8Yqxc9cKQvO0E%3D&reserved=0, or mute the threadhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGhSPCyZhhEyuDC4-ZX71mzUpUwEVks5ue0rKgaJpZM4W5L7t&data=02%7C01%7C%7C627896895e5a4ca47b9008d62393a544%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636735514364501917&sdata=rzseeMW0Dnpwus6zCTnnYIsswUpINpsm2%2F7Jzq5RVq8%3D&reserved=0.

[ghost]

@Jilco Problem in Cron for Analytics module this cron runs report to write file, if you turn off reports, they not be generate files.

[ghost]

Hi @Jilco This may be fix your issue -> https://github.com/magento/magento2/pull/18322

[ghost]

Hi @Jilco This issue was fixed by this commit -> https://github.com/magento/magento2/commit/8e1a5d342cbc63f58529b6c25be41c7d9a979a66

[Flowlance]

Same problem. A 83GB tar file in /tmp/analytics...

[ghost]

Hi @Flowlance This issue has been fixed by this commit -> https://github.com/magento/magento2/commit/8e1a5d342cbc63f58529b6c25be41c7d9a979a66

[maderlock]

Is this fixed in 2.2.7 then?

[Jilco]

Rob,

Just safe the settings again and the problem is gone.

---

Van: Rob Egginton [e-mail]
Verzonden: dinsdag 4 december 2018 11:19
Aan: magento/magento2
CC: Jilco; Mention
Onderwerp: Re: [magento/magento2] After 2.2.6 upgrade reports generates enormous .tar file in tmp/analytics folder in root (#18243)

Is this fixed in 2.2.7 then?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F18243%23issuecomment-444047785&data=02%7C01%7C%7C13b3b50175604707151a08d659d1ec39%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636795155471944875&sdata=Hvyj3O%2B9vzWagElqL%2F9cmQc52Gud0qb%2FPNIUVJ2m7lc%3D&reserved=0, or mute the threadhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGpXJ9WAvWgZQfHEbviYQY5T4yCRPks5u1kwZgaJpZM4W5L7t&data=02%7C01%7C%7C13b3b50175604707151a08d659d1ec39%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636795155471944875&sdata=uZdy82gWHVv3xMruEP4u8sIuLYvMTgGzj%2FYE8aL5fDQ%3D&reserved=0.

[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virusvrij. www.avast.comhttps://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail

[maderlock]

Sadly, saving the settings again has not helped. Two days later the disk filled up with 200Gb of rubbish. Going to have to turn this off in 2.2.7.

Would this fix be hard to back-port to 2.2?

[maderlock]

Apparently there is a back-port to 2.2 that got merged in October, but this has oddly not ended up in 2.2.7.

I've tried to override \Magento\Framework\Archive\Tar to patch this myself (pulling in everything by composer means I cannot just change the file), but sadly this has not worked because this class is pull in via a low-level "new" statement that bypasses DI.

Any advice on how this can be fixed in 2.2.7 when using composer would be gratefully received. Currently my only option looks to be to tell composer to skip the entire framework so I can copy in the framework locally and patch these few line :(

[DanielRuf]

@engcom-backlog-nazar not sure why you linked my clone and commit but this is clearly not the cause. Also this was not what we've merged in the end.

[DanielRuf]

@maderlock you can use composer-patches and / or the patch file of the commit.

[DanielRuf]

https://github.com/magento/magento2/commit/45f22aca2e455f2df3fdb0f7ae91563cf388e407.patch

[stu177]

I just had this occur on 2.3.0 on Ubuntu 18.04.1.

[Ctucker9233]

Not fixed as of 2.2.7.

[ghost]

@Ctucker9233 this fix will be available on 2.2.8 release and 2.3.1 release

[ghost]

@Ctucker9233 https://github.com/magento/magento2/pull/18322

[smadasam]

I am having the same issue on my 2.3.0 site. How do I patch it until the next release comes out?

:/tmp/analytics# ll -ah
total 62G
drwxrwxr-x 2 sw_temp psacln 4.0K Jan 25 10:02 .
drwxrwxrwt 11 root root 20K Jan 26 00:41 ..
-rw-rw-r-- 1 sw_temp psacln 101K Jan 22 10:00 ~tmp-1548151203.3161tar.tar
-rw-rw-r-- 1 sw_temp psacln 71M Jan 23 10:00 ~tmp-1548237603.0528tar.tar
-rw-rw-r-- 1 sw_temp psacln 41G Jan 24 10:02 ~tmp-1548324002.9252tar.tar
-rw-rw-r-- 1 sw_temp psacln 22G Jan 25 10:02 ~tmp-1548410402.865tar.tar

[DanielRuf]

https://github.com/magento/magento2/issues/18243#issuecomment-445789885

https://patch-diff.githubusercontent.com/raw/magento/magento2/pull/18322.patch

https://support.magento.com/hc/en-us/articles/360005484154-Create-a-patch-for-a-Magento-2-Composer-installation-from-a-GitHub-commit

[GreenGinkgo]

Just as a note, storing data in /tmp is misguided.

Shared servers that have a single /tmp would be co-mingling different user data together. I suggest that this data be stored within the Magento installation itself. Maybe in (magento)/var/analytics or similar.

[smadasam]

It is the default. If it isn't good, then hardcode a better default.

On Tue, Jan 29, 2019 at 9:22 AM Clark Adamczyk [e-mail]
wrote:

Just as a note, storing data in /tmp is misguided.

Shared servers that have a single /tmp would be co-mingling different user
data together. I suggest that this data be stored within the Magento
installation itself. Maybe in (magento)/var/analytics or similar.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/18243#issuecomment-458628550,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AeRy2GzBlTYo08aGOaDJfYqJhpru9n9Jks5vIINogaJpZM4W5L7t
.

--
Sam Adams (S)

[srenon]

Just upgraded to 2.2.7 and run into this issue with over 50gb of diskspace disappear in 2 days.

If you are not using this feature, you can disable "Advanced REporting Service" in admin and flush cache.

Stores > Configuration > General > Advanced Reporting

[DanielRuf]

@srenon see https://github.com/magento/magento2/issues/18243#issuecomment-424646813 ;-)

[DanielRuf]

I'm locking this issue as it is already fixed in the next releases and we have linked the patches and all needed steps to either disable Advanced Reporting or apply the patches.

[DanielRuf]

Shared servers that have a single /tmp would be co-mingling different user data together.

They have a much more critical issue - a security issue.