Yarn: tmp directory filled by yarn (and not cleaned)

[nicolas-lecab]

Do you want to request a feature or report a bug?
bug

What is the current behavior?
Each "yarn run" will create a directory under "/tmp" directory which looks like

/tmp/yarn--1542302743598-0.6454897498245407

If the current behavior is a bug, please provide the steps to reproduce.
Let's say we have a script in package.json as follow:

    "scripts": {
        "dummy": "echo 'dummy'"
    }

If we play this dummy script and we check number of directories before and after in /tmp:

$> ls -dla /tmp/yarn* | grep $USER | wc -l
0

$> yarn run dummy
yarn run v1.12.3
$ echo 'dummy'
dummy
Done in 0.12s.

$> ls -dla /tmp/yarn* | grep $USER | wc -l
1

Running the dummy script will increment the result each time.
FYI:

• yarn is installed globally
• bug seems to appear somewhere between v1.10.0 and v1.11.0 (I can't reproduce with 1.10.0)

What is the expected behavior?
Tmp directory should not be filled indefinitely: it should reuse the already created temporary files (if tmp is really needed) or it should have some kind of cleaning mechanism to avoid overwhelmingly the tmp directory.

Please mention your node.js, yarn and operating system version.
node: v8.9.4
nvm: 0.33.2
yarn: 1.12.3
os: Debian 3.16.51-3

[ellioseven]

I am getting the same results on 1.12.3. Not really sure why this is happening?

I noticed each tmp entry has a shell script relating to node and yarn. I have tried running yarn run with different flags such as --link-folder, --global-folder and --cache-folder but I can't seem to prevent the directories from being created.

[arcanis]

Yarn creates a temporary directory each time a script is executed, in order to setup "fake" Node and "fake" Yarn binaries that point towards the exact same Yarn/Node than those currently being executed (to prevent the case where the global Node is different from the one used to run Yarn itself).

The problem is that they never get removed at the moment:
https://github.com/yarnpkg/yarn/blob/master/src/util/execute-lifecycle-script.js#L31-L48

[tomq42]

Yep, also see this.

[BookOfGreg]

My /tmp is 24gb almost all of which is the Yarn files described above.

me@me:~$ sudo du -hs /tmp
24G     /tmp
me@me:~$ uptime
 15:58:04 up 52 days,  5:30,  1 user,  load average: 0.97, 0.87, 0.50

Was this a regression? I don't remember my CI agents being filled by this in the past.

[lexaurin]

The same problem with yarn 1.15.2

[ZLima12]

Still happening for me with yarn 1.16.0.

[xmontero]

In the meantime.... Is it safe to manually remove all of them? Or the removal could break anything?

[JLarky]

I wrote crontab that does "mkdir -p /tmp/yarn & mv /tmp/yarn-* /tmp/yarn/"
(typing by memory, don't have my laptop to check). In Linux if you rename
directory it still can be used by program that already has file descriptor,
same goes for deletion I think. Never had issues with renaming via cron,
never had issues removing them manually, I would assume it's probably safe
to delete it from cron as well. Worst case scenario I would assume your
currently running "yarn run something" will crash.

On Mon, Jun 17, 2019, 12:33 Xavi Montero [e-mail] wrote:

In the meantime.... Is it sure to manually remove all of them? Or removal
could break anything?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/6685?email_source=notifications&email_token=AAABW4UKLTC7IRND2RCFOTDP27RIZA5CNFSM4GEERUT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX4G3BQ#issuecomment-502820230,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAABW4SVFYLVBE4F3H7ZRQ3P27RIZANCNFSM4GEERUTQ
.

[nicolas-lecab]

@xmontero We removed files older than 1 day on our dev server.
Never had issue since then.
10 4 * * * find /tmp/ -name "yarn*" -type d -mtime +1 -exec rm -rf {} \; > /dev/null

[arcanis]

There is no risk removing those files when Yarn isn't running. Even when running, it should be relatively safe as they're usually only used when the program boots.

Fwiw this problem will be fixed from the v2 onward; the artifacts will be properly discarded right after the commands return.

[lexaurin]

Running build in Docker image is also a good workaround. It protects you
from such leaks to CI workers system, by design.

On Tue, Jun 18, 2019, 11:37 Maël Nison [e-mail] wrote:

There is no risk removing those files when Yarn isn't running. Even when
running, it should be relatively safe as they're usually only used when the
program boots.

Fwiw this problem will be fixed from the v2 onward; the artifacts will be
properly discarded right after the commands return.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/6685?email_source=notifications&email_token=AADISDU4FFFFWZZI37RRY4TP3CUGXA5CNFSM4GEERUT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX5ZV4I#issuecomment-503028465,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADISDVU55SPQE54EYNMLVDP3CUGXANCNFSM4GEERUTQ
.

[dvdknaap]

Still getting this at the current version

[ClementNerma]

It's still not fixed at this time, even on latest 1.21.1 version :/

[djsiddz]

Is this issue persistent with yarn v2? If anyone has tried this, please let me know. I am giving it a shot today with v2 on one of our apps to check this in the mean time.

[snowl]

@djsiddz Did v2 fix it for you?

[dvdknaap]

De issue still exists in the latest version

sh yarn -version
1.22.4

sh ls -alF

drwxr-xr-x 2 username username 4096 Jun 9 11:57 yarn--1591696641892-0.05204308679429248/
drwxr-xr-x 2 username username 4096 Jun 10 14:55 yarn--1591793724850-0.4196722332063454/
drwxr-xr-x 2 username username 4096 Jun 10 15:15 yarn--1591794953501-0.3040314059514666/
drwxr-xr-x 2 username username 4096 Jun 10 15:26 yarn--1591795571216-0.8222359319740244/
drwxr-xr-x 2 username username 4096 Jun 11 09:33 yarn--1591860782625-0.15962323877699136/
drwxr-xr-x 2 username username 4096 Jun 11 09:36 yarn--1591860987612-0.0746955607161679/
drwxr-xr-x 2 username username 4096 Jun 11 11:03 yarn--1591866191467-0.6557044172705777/
drwxr-xr-x 2 username username 4096 Jun 11 11:04 yarn--1591866263355-0.7429486092878892/
drwxr-xr-x 2 username username 4096 Jun 11 11:07 yarn--1591866427043-0.3962468811805635/
drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872605318-0.8161204330414475/
drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872605322-0.48513192538032124/
drwxr-xr-x 2 username username 4096 Jun 11 12:50 yarn--1591872622244-0.4583333405082548/
drwxr-xr-x 2 username username 4096 Jun 11 13:59 yarn--1591876762200-0.05283671445181093/
drwxr-xr-x 2 username username 4096 Jun 11 14:09 yarn--1591877355444-0.7000002370403433/

[ClementNerma]

I can confirm this still happens on the very same version (1.22.4).
Platform: Debian 10 on WSL - Windows 10 x64 ver 19041 (2004).

[ezeqd]

Any update on this? we still have several problems with hard disk and "no left space" on Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-38-generic x86_64) with
yarn -v
1.13.0

[mohux]

still not fixed? :/ my server is filled with tmp yarn files