Rocket.chat: Error when using Download My Data or Export My Data

Created on 15 May 2018 · 17Comments · Source: RocketChat/Rocket.Chat

Description:

When attempting to export or download a user's data, the system errors out. No additional configuration has been done to User Data Download section, other than it is enabled.

Server Setup Information:

Version of Rocket.Chat Server:
Operating System: Ubuntu 18.04 LTS
Deployment Method(snap/docker/tar/etc): docker rocketchat/rocket.chat:latest
Number of Running Instances: 1

Rocket.Chat

| Version | 0.64.1 |
| ----------------------- | --------------------------------------------------------- |
| Database Migration | 115 |
| Database Migration Date | Mon May 07 2018 10:41:14 GMT-0500 (Central Daylight Time) |
| Installed at | Mon Jan 29 2018 16:02:11 GMT-0600 (Central Standard Time) |
| Uptime | 8 days, 2 hours, 28 seconds |
| Deployment ID | JAELT9iyADPbKEGbG |
| PID | 1 |
| Running Instances | 1 |
| OpLog | Enabled |

Commit

| Hash | 54738c011b1fededb198dbf64be11ba48889c762 |
| ------- | -------------------------------------------------------- |
| Date | Thu May 3 13:37:12 2018 -0300 |
| Branch | HEAD |
| Tag | 0.64.1 |
| Author | Rodrigo Nascimento |
| Subject | Merge pull request #10660 from RocketChat/release-0.64.1 |

Steps to Reproduce:

Go to User -> My Account -> My Data -> Download/Export My Data

Expected behavior:

The data should be made available for download.

Actual behavior:

System errors out with "Internal Server Error" message:

Relevant logs:

Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/*********'
    at Object.fs.mkdirSync (fs.js:885:18)
    at MethodInvocation.requestDataDownload (/app/bundle/programs/server/app/app.js:10356:10)
    at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
    at MethodInvocation.methodsMap.(anonymous function) (/app/bundle/programs/server/packages/rocketchat_lib.js:2186:28)
    at maybeAuditArgumentChecks (/app/bundle/programs/server/packages/ddp-server.js:1877:12)
    at DDP._CurrentMethodInvocation.withValue (/app/bundle/programs/server/packages/ddp-server.js:902:126)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
    at DDPServer._CurrentWriteFence.withValue (/app/bundle/programs/server/packages/ddp-server.js:902:98)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
    at Promise (/app/bundle/programs/server/packages/ddp-server.js:902:46)
    at new Promise (<anonymous>:null:null)
    at Session.method (/app/bundle/programs/server/packages/ddp-server.js:875:23)
    at /app/bundle/programs/server/packages/ddp-server.js:754:85

bug

Source

bkraul

All 17 comments

Can confirm this on 0.64.2-rc.2.

Download my data error

May 18 15:32:09 dev-chat RocketChat: #033[34mMeteor ➔ method#033[39m requestDataDownload -> userId: mFFfETbmt3gxcFk2G , arguments:  { '0': { fullExport: false } }
May 18 15:32:09 dev-chat RocketChat: Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/mFFfETbmt3gxcFk2G'
May 18 15:32:09 dev-chat RocketChat: at Object.fs.mkdirSync (fs.js:885:18)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.requestDataDownload (/opt/rocketchat/bundle/programs/server/app/app.js:10630:10)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
May 18 15:32:09 dev-chat RocketChat: at MethodInvocation.methodsMap.(anonymous function) (/opt/rocketchat/bundle/programs/server/packages/rocketchat_lib.js:2261:36)
May 18 15:32:09 dev-chat RocketChat: at maybeAuditArgumentChecks (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:1877:12)
May 18 15:32:09 dev-chat RocketChat: at DDP._CurrentMethodInvocation.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:126)
May 18 15:32:09 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:09 dev-chat RocketChat: at DDPServer._CurrentWriteFence.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:98)
May 18 15:32:09 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:09 dev-chat RocketChat: at Promise (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:46)
May 18 15:32:09 dev-chat RocketChat: at new Promise (<anonymous>:null:null)
May 18 15:32:09 dev-chat RocketChat: at Session.method (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:875:23)
May 18 15:32:09 dev-chat RocketChat: at /opt/rocketchat/bundle/programs/server/packages/ddp-server.js:754:85

Export my data error

May 18 15:32:26 dev-chat RocketChat: #033[34mMeteor ➔ method#033[39m requestDataDownload -> userId: mFFfETbmt3gxcFk2G , arguments:  { '0': { fullExport: true } }
May 18 15:32:26 dev-chat RocketChat: Exception while invoking method 'requestDataDownload' Error: ENOENT: no such file or directory, mkdir '/tmp/userData/mFFfETbmt3gxcFk2G'
May 18 15:32:26 dev-chat RocketChat: at Object.fs.mkdirSync (fs.js:885:18)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.requestDataDownload (/opt/rocketchat/bundle/programs/server/app/app.js:10630:10)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.methodMap.(anonymous function) (packages/rocketchat_monitoring.js:2731:30)
May 18 15:32:26 dev-chat RocketChat: at MethodInvocation.methodsMap.(anonymous function) (/opt/rocketchat/bundle/programs/server/packages/rocketchat_lib.js:2261:36)
May 18 15:32:26 dev-chat RocketChat: at maybeAuditArgumentChecks (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:1877:12)
May 18 15:32:26 dev-chat RocketChat: at DDP._CurrentMethodInvocation.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:126)
May 18 15:32:26 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:26 dev-chat RocketChat: at DDPServer._CurrentWriteFence.withValue (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:98)
May 18 15:32:26 dev-chat RocketChat: at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:12)
May 18 15:32:26 dev-chat RocketChat: at Promise (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:902:46)
May 18 15:32:26 dev-chat RocketChat: at new Promise (<anonymous>:null:null)
May 18 15:32:26 dev-chat RocketChat: at Session.method (/opt/rocketchat/bundle/programs/server/packages/ddp-server.js:875:23)
May 18 15:32:26 dev-chat RocketChat: at /opt/rocketchat/bundle/programs/server/packages/ddp-server.js:754:85

Cheers
Thomas

TwizzyDizzy on 18 May 2018

@RocketChat/core This is an important piece of our GDPR and seeing as it is broke on instances, we need to get it fixed asap.

graywolf336 on 19 May 2018

👍1

Should probably be re-opened, still broken in 0.74.3 and 1.0.0-beta.0. 69 jobs in rockechat_export_operations with the oldest one from 2018-06-02. Cleared all old jobs, queued up a new one, same error.

This feature doesn't seem to have ever actually worked except for single server setups.

{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/7zfSTXWNPAdXaToex/partial'
    at Object.fs.mkdirSync (fs.js:885:18)
    at createDir (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:33:6)
    at continueExportingRoom (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:186:2)
    at exportOperation.roomList.forEach.exportOpRoomData (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:418:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:417:29)
    at cursor.forEach.exportOperation (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:467:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at Object.processDataDownloads [as job] (packages/rocketchat:user-data-download/server/cronProcessDownloads.js:462:9)
    at packages/littledata_synced-cron.js:251:26
    at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
    at packages/littledata_synced-cron.js:356:42
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/7zfSTXWNPAdXaToex/partial' }

TkTech on 12 Mar 2019

I can confirm this on the latest docker develop version.

bkraul on 13 Mar 2019

We are able to confirm this on Rocket.Chat version 0.74.3 too.

dhoeld on 1 May 2019

Anyone have a fix for this?
On thing i noticed in my setup (3 docker containers on 3 different hosts) - One container has /tmp/userData created and the others done. Do you need to bind a host directory to /tmp/userData?

Also running 0.74.3

mghicas on 3 May 2019

According to the logfiles the requestDataDownload Service is trying to create a "tempdir/partial" folder without creating the "tempdir" folder first (tempdir seems to be the userId?)

The file.server Source uses "mkdirp" instead of "fs" which creates parent directories as well. Switching to that method should solve at least the "mkdir" error

himpich on 27 May 2019

User Data export in 1.1.1 (Docker Image at least) also appears to be broken.
When you select the export sand the internal scheduler kicks it off:

The directories are created
The error below is thrown
Email is never sent (but email is tested and working)

Setup is 3 containers on different hosts with an external MongoDB

From inside the docker container on node1:

#cd /tmp/userData
#find .
./userData
./userData/eq5ka6HvRJnaNg3Rp
./userData/eq5ka6HvRJnaNg3Rp/full
./userData/eq5ka6HvRJnaNg3Rp/full/assets
./userData/eq5ka6HvRJnaNg3Rp/partial
./userData/eq5ka6HvRJnaNg3Rp/partial/assets

Error messages in the logs:

Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Starting "Generate download files for user data".
Jun  2 03:45:00 rchat01 dockerd-current[9150]: { Error: ENOENT: no such file or directory, mkdir '/tmp/userData/eq5ka6HvRJnaNg3Rp/full'
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Object.fs.mkdirSync (fs.js:885:18)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Array.forEach (<anonymous>)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Object.processDataDownloads [as job] (app/user-data-download/server/cronProcessDownloads.js:463:9)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/littledata_synced-cron.js:251:26
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at scheduleTimeout (packages/littledata_synced-cron.js:304:9)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/littledata_synced-cron.js:356:42
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at packages/meteor.js:620:25
Jun  2 03:45:00 rchat01 dockerd-current[9150]:     at runWithEnvironment (packages/meteor.js:1356:24)
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   errno: -2,
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   code: 'ENOENT',
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   syscall: 'mkdir',
Jun  2 03:45:00 rchat01 dockerd-current[9150]:   path: '/tmp/userData/eq5ka6HvRJnaNg3Rp/full' }
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Finished "Generate download files for user data".
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.
Jun  2 03:45:00 rchat01 dockerd-current[9150]: #033[34mSyncedCron ➔ info#033[39m Not running "Generate download files for user data" again.

mghicas on 6 Jun 2019

Yup, #14645 fixes this.

Hudell on 6 Jun 2019

The same issue with the official docker images (latest, develop, release-candidate) from here.
The server's error log:

{ Error: ENOENT: no such file or directory, mkdir '/tmp/userData/EvFWaNTxwXrBP25zc/full'                                                                                                         
    at Object.fs.mkdirSync (fs.js:885:18)                                                                                                                                                        
    at createDir (app/user-data-download/server/cronProcessDownloads.js:36:6)                                                                                                                    
    at continueExportingRoom (app/user-data-download/server/cronProcessDownloads.js:189:2)
    at exportOperation.roomList.forEach.exportOpRoomData (app/user-data-download/server/cronProcessDownloads.js:419:5)
    at Array.forEach (<anonymous>)
    at continueExportOperation (app/user-data-download/server/cronProcessDownloads.js:418:29)
    at cursor.forEach.exportOperation (app/user-data-download/server/cronProcessDownloads.js:468:3)
    at SynchronousCursor.forEach (packages/mongo/mongo_driver.js:1107:16)
    at Cursor.(anonymous function) [as forEach] (packages/mongo/mongo_driver.js:887:44)
    at processDataDownloads (app/user-data-download/server/cronProcessDownloads.js:463:9)
    at app/user-data-download/server/cronProcessDownloads.js:479:3
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1304:12)
    at packages/meteor.js:620:25
    at runWithEnvironment (packages/meteor.js:1356:24)
  errno: -2,
  code: 'ENOENT',
  syscall: 'mkdir',
  path: '/tmp/userData/EvFWaNTxwXrBP25zc/full' }

0l-l0 on 10 Sep 2019

I can confirm this. This seems to not have yet been fully corrected.

bkraul on 10 Sep 2019

My RocketChat server admin has confirmed that we are at the latest version yet cannot export. He pointed me to this ticket as indication that we are stuck. That is all the information I have right now but just wanted to add my suspicion that this issue is not actually fixed.

eeowaa on 23 Oct 2019

It is working on my end now. I configured the correct path and removed the export-jobs from the mongo db. New jobs ran through.

himpich on 23 Oct 2019

It is working on my end now. I configured the correct path and removed the export-jobs from the mongo db. New jobs ran through.

For others can you please post how you did that? Thanks.

reetp on 23 Oct 2019

start your mongo client and connect

use correct database:

rs0:PRIMARY> use rocketchat;
switched to db rocketchat

Verify there is some export operations data:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

remove it:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

re-verify that the table (or whatever it is called in mongodb) is now empty.

Good luck

himpich on 23 Oct 2019

👍1

remove it:

rs0:PRIMARY> db.rocketchat_export_operations.find( {} );

Not sure that is quite right !

reetp on 23 Oct 2019

👍1

Still not sure this is fixed. v 2.1.1 docker

First, it shouldn’t leave old exports blocking up the system but it clearly has.

Second, to try and get over this I deleted out the old keys as per the above comments. That throws errors as per this:

SyncedCron ➔ info Starting "Generate download files for user data". SyncedCron ➔ info Finished "Generate download files for user data". SyncedCron ➔ info Not running "Generate download files for user data" again. SyncedCron ➔ info Not running "Generate download files for user data" again. SyncedCron ➔ info Not running "Generate download files for user data" again. SyncedCron ➔ info Not running "Generate download files for user data" again. Exception on find Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment. at Object.Meteor._nodeCodeMustBeInFiber (packages/meteor.js:1186:11) at Meteor.EnvironmentVariable.EVp.get (packages/meteor.js:1199:10) at Object.collection.(anonymous function) [as findOne] (packages/matb33_collection-hooks.js:132:37)

Full log: https://pastebin.com/Fq7q3N7t

I then tried adding one html and one json export for my user.

It puts the new file paths in the settings but does not use appear to use them - see pics.

Screenshot_2019-10-23_18-11-49