Symfony: cache:clear breaks with error: Warning: rmdir(...): Directory not empty in ...

[naitsirch]

Hi,
I am running Symfony 2.0.5 on Windows 7 with PHP version 5.3.8.
When I am calling cache:clear on the console tool, I get this error sometimes (I do not know why):

C:\projects\cnc_agency\trunk>php app\console cache:clear
Clearing the cache for the dev environment with debug true

[ErrorException]
Warning: rmdir(C:\projects\cnc_agency\trunk\app/cache/dev_old\twig\f8): Directory not empty in C:\projects\cnc_agency\trunk\vendor\symfony\src\Symfony\Component\HttpKernel\Util\Filesystem.php line 98

cache:clear [--no-warmup]

When I remove the cache dir, it works again. But this is annoying a little bit.

[stloyd]

Can you give an bigger stacktrace ? Seems you have an permissions error here.

[Seldaek]

I also get this every now and then, it just fails to delete something for no apparent reason and then that crashes the entire process.

[Tobion]

Same for me.

[naitsirch]

@ stloyd: I will try to get more info if it appears next time.
I am lucky that I am not the only one who has this problem ;)

[joke2k]

I think the problem is automatic indexing of folders run from Windows.
Enabling it can happen that you do not have permission to remove ... but disabling it does not solve the problem.
you have any news?

[naitsirch]

@joke2k: Maybe that's true.

I have not tested it again. Maybe I can do that tomorrow. But I think it is important to check, if the files are writeable, bevor removing or moving them, in every case. If they are not writeable Symfony could throw an Exception. This would make it easier to find the reason.

[naitsirch]

When I tried to clear cache today, I got the same error again.

C:\projects\cnc_agency\trunk>php app\console cache:clear
Clearing the cache for the dev environment with debug true

[ErrorException]

Warning: rmdir(C:\projects\cnc_agency\trunk\app/cache/dev_old\twig\e8): Directory not empty in C:\projects\cnc_agency\trunk\vendor\symfony\src\Symfony\Component\HttpKernel\Util\Filesystem.php line 98

Then I removed the folder app/cache/dev_old manually via Explorer and executed cache:clear again.
Then it worked, surprisingly. Any ideas? ...

[joke2k]

me too.
disabling windows indexing of cache path it is not enough.

do you use some IDE like netbeans o eclipse?
they also indexing cache dir, exclude it from project settings.. and retry.

[naitsirch]

do you use some IDE like netbeans o eclipse?
they also indexing cache dir, exclude it from project settings.. and retry.

That solved the problem. But I do not understand why the indexing leads to permission problems...

[joke2k]

it seems a protection for mass file deleting... or more probabily a concurrency locking error

[pminnieur]

you can "easily" reproduce this if you use a Linux VirtualBox on a Windows host. cache:clear is never able to remove app/cache/dev_old - but that may be an issue with the shared folder system provided by VirtualBox (read about similar issues on their forums).

[naitsirch]

Okay, Issue can be closed, because it's not a bug in Symfony. But may be this will help others who are searching for a solution for the Problem.

[fariazz]

I had this same issue in Ubuntu 11.10 and fixed it by doing:

cd app
chmod a+w cache -R

[ashton]

Had same problem in mac ox 10.8 mountain lion

[netmikey]

I have that same behavior, too on snow leopard, very weird stuff like:

root@vagrant-debian-squeeze-64:/var/www# rm -rf app/cache/dev_old/
rm: cannot remove `app/cache/dev_old/annotations': Directory not empty
rm: cannot remove `app/cache/dev_old/assetic': Directory not empty
rm: cannot remove `app/cache/dev_old/doctrine': Directory not empty
rm: cannot remove `app/cache/dev_old/jms_diextra': Directory not empty
rm: cannot remove `app/cache/dev_old/jms_security': Directory not empty
root@vagrant-debian-squeeze-64:/var/www# rm -rf app/cache/dev_old/
root@vagrant-debian-squeeze-64:/var/www# 

[netmikey]

It took me quite some time, but I have found a solution for my case, might be helpful to others as well, documented it here: http://forum.symfony-project.org/viewtopic.php?f=32&t=52241

[pborreli]

@netmikey this is a known bug fixed by VirtualBox in version 2.1.18, you have to upgrade VirtualBox Guest Additions https://github.com/composer/composer/issues/690#issuecomment-6527147
if you have no idea how to do it, I suggest https://gist.github.com/3293481.
On Windows it's the only way to go, on Unix system you are right, you'd better use NFS anyway.

[lenybernard]

Actually, you just have to add the user to the group vboxsf.
for example : sudo adduser YOURUSER vboxsf

[jenkoian]

Are you running IIS or apache? We had problems with IIS locking files so had to stop IIS, clear cache, then restart IIS to get around it.

[blickwerker]

same problem here.
dropbox syncing was blocking file deletion.

[Wirone]

I got similar problem since updating to Symfony 2.8

app/console cache:clear -e=prod -vvv

  [Symfony\Component\Filesystem\Exception\IOException]
  Failed to remove directory "app\cache\prod_old\annotations\f1".

Exception trace:
 () at vendor\symfony\symfony\src\Symfony\Component\Filesystem\Filesystem.php:165
 Symfony\Component\Filesystem\Filesystem->remove() at vendor\symfony\symfony\src\Symfony\Component\Filesystem\Filesystem.php:162
 Symfony\Component\Filesystem\Filesystem->remove() at vendor\symfony\symfony\src\Symfony\Component\Filesystem\Filesystem.php:162
 Symfony\Component\Filesystem\Filesystem->remove() at vendor\symfony\symfony\src\Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand.php:69
 Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand->execute() at vendor\symfony\symfony\src\Symfony\Component\Console\Command\Command.php:259
 Symfony\Component\Console\Command\Command->run() at vendor\symfony\symfony\src\Symfony\Component\Console\Application.php:857
 Symfony\Component\Console\Application->doRunCommand() at vendor\symfony\symfony\src\Symfony\Component\Console\Application.php:192
 Symfony\Component\Console\Application->doRun() at vendor\symfony\symfony\src\Symfony\Bundle\FrameworkBundle\Console\Application.php:98
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at vendor\symfony\symfony\src\Symfony\Component\Console\Application.php:123
 Symfony\Component\Console\Application->run() at app\console:41

I always need to manually remove whole cache.

[xrstf]

I am experiencing the same issues after upgrading from 2.3 to 2.8. These are my dependencies:

"require": {
    "php": ">=5.5.0",
    "doctrine/data-fixtures": "1.1.*",
    "doctrine/doctrine-bundle": "1.6.*",
    "doctrine/doctrine-fixtures-bundle": "2.3.*",
    "doctrine/doctrine-migrations-bundle": "1.1.*",
    "doctrine/orm": "2.5.*",
    "guzzle/guzzle": "~3.9",
    "jms/di-extra-bundle": "1.6.*",
    "jms/security-extra-bundle": "1.5.*",
    "sensio/distribution-bundle": "4.0.*",
    "sensio/framework-extra-bundle": "3.0.*",
    "symfony/assetic-bundle": "2.7.*",
    "symfony/monolog-bundle": "2.8.*",
    "symfony/swiftmailer-bundle": "2.3.*",
    "symfony/symfony": "2.8.*",
    "twig/extensions": "1.3.*"
}

On Windows x64 with PHP 5.6. Looks to me as if some cache filenames are absurdly long and cause issues. For example, I have a dev_old/annotations/f0/5b5b435d566973755265616c5c5765627365727669636542756e646c655c436f6e74726f6c6c65725c436f6e6e656374696f6e436f6e74726f6c6c657223757064617465456d61696c436f6e6e656374696f6e405b416e6e6f745d5d5b315d.doctrinecache.data file which I cannot even edit manually with Notepad anymore.

[xabbuh]

There have been recent fixes regarding your problem in doctrine/cache and Symfony. But there was no Symfony release after that change. So you will have to rely on ~2.8@dev for the moment.

[xrstf]

Thanks, I will give it a shot.

_Edit:_ Yes, using 2.8@dev of symfony/symfony fixes the issue for me.

[Wirone]

In my case it also relates to Windows x64, so probably it is what @xrstf said (too long pathnames) because in Unix-based instances everything works correctly. Thanks for pointing it out.

[ureimers]

This security change in the write privileges of Doctrine cache files could also be the problem.

tl;dr;
umask(0) doesn't work anymore when using a file cache for Doctrine.

[emudojo]

I'm having the exact same problem

[emudojo]

I get like a million of these [2016-04-21 15:25:54] php.DEBUG: rename(/var/www/app.local/app/cache/de_/annotations/a9/swaprP2px2,/var/www/app.local/app/cache/de_/annotations/a9/5b5b435d53656e73696f5c42756e646c655c446973747269627574696f6e42756e646c655c436f6e74726f6c6c65725c436f6e666967757261746f72436f6e74726f6c6c65722367657456657273696f6e405b416e6e6f745d5d5b315d.doctrinecache.data): Input/output error {"type":2,"file":"/var/www/app.local/vendor/doctrine/cache/lib/Doctrine/Common/Cache/FileCache.php","line":255,"level":28928} []

and the process never finishes stays here forever

[xabbuh]

Which Symfony version and which OS do you use?

[samayo]

And 6 Years later ...

[root@localhost blog]# php bin/console cache:clear
// Clearing the cache for the dev environment with debug true

[Symfony\Component\Filesystem\Exception\IOException]
Failed to remove directory "/var/www/blog/var/cache/de~/twig": .

[fabpot]

@samayo This is apparently a problem on Windows systems. Nobody in the core team is using Windows. We would be very happy to have a look at a PR fixing this issue though. Can you have a look?

[Freid001]

@fabpot - I am having the exact same problem with my (Windows/VBox/Linux) setup too. I have followed all the steps at: http://symfony.com/doc/current/setup/file_permissions.html but still cannot clear the cache directory.

[Symfony\Component\Filesystem\Exception\IOException]
  Failed to remove directory "/vagrant/project/var/cache/pro~/twig": rmdir(/vagrant/project/var/cache/pro~/twig): Directory not empty.

[samayo]

@Freid001 he has already answered you question, check his response to my post. No need to ask the same question twice

[Freid001]

So I assume there is no fixed? - "This is apparently a problem on Windows systems. Nobody in the core team is using Windows."

[samayo]

Yeah. I think I got around it by removing the var/cache folder repetedly. It looks like a good framework but the fact it is only tested on linux env makes it a hell to use via vbox or on windows

[Freid001]

Yeah, I did rm -rf var/* then run cache:clear again, but it still gives me the same exception. The only way it seems that I can re-run cache:clear is to restart the VM, which is really frustrating.

[samayo]

@Freid001 Let us know how you got rid of the issue, so it can be of help to new commers.

[litwicki]

I just wanted to add to the conversation that this is definitely not a windows only issue. Running Mac OS Sierra 10.12, Vagrant, Virtualbox (Ubuntu 16.04)

[Symfony\Component\Filesystem\Exception\IOException] Failed to remove file "myapp/var/cache/de~/profiler/ca/fd/f7fdca": .

Removing the files "locally" works fine if you just run something like rm -rf myapp/var/cache/* but executing it as vagrant in the virtualmachine fails regardless of permissions of any kind.

[cminovici]

It's a Vagrant issue with the permissions between host - guest. I'm using Homestead on Windows 10 and my way of clearing the cache is to run it with the help of Git Bash in the project folder. All other bin/console commands are running smoothly on linux box.

Windows: Didn't have much time lately to play with Vagrant options but I would try using Vagrant SMB file-sharing.
Linux/OSX: https://github.com/gael-ian/vagrant-bindfs might help setting the correct perms.

I know it's not the answer you guys are looking for but at least you don't have to reboot your VM lol

[melissabillias]

Just thought I would leave this here because this was the solution for me and I don't see any references to it. Basically, there is a performance issue with Symfony accessing the cache (and log) files on the shared system. So update the AppKernel so that caching (and logging) is done on the machine. Problem solved.

http://www.whitewashing.de/2013/08/19/speedup_symfony2_on_vagrant_boxes.html

[mchojrin]

@melissabillias I tried your solution and it worked thanks. I wonder why there isn't a way to configure this through config.yml (like framework.session.save_path, for which I had to apply the same criteria).

That would, in my opinion, be better, since it'd isolate the particulars of the dev/test environment without altering the app code.

[litwicki]

@melissabillias That _does_ work, however cache:clear now fails with permission issues on the new cache directory and needs to be manually set to 0775+ or it will not complete?

[J7mbo]

This is not a windows-only issue. Using Mac OS Sierra, latest virtual box, vagrant and Ubuntu 16.04, same problem. Did anyone actually fix this?

[samayo]

@J7mbo Nope. I finally stopped using Symfony, so there's always that :) I even disabled caching everywhere and Symfony still dumps cache files out of nowhere.

[kotbehemot53]

I'm using Windows 10 and run various Sf projects on various versions of Symfony from 2.3 to 3.2. I've had these problems in all of them since forever (also on Win 7). In some projects it's permanent, in others incidental - when clearing cache, things go bonkers - sometimes it can't rename a directory, sometimes it can't delete a file. Examples:

[Symfony\Component\Debug\Exception\ContextErrorException] Warning: file_put_contents(H:\_Moje_dydy\Documents\Projekty\atm\src/var/cache/dev/classes.map): failed to open stream: Permission denied

This one is very common (but only in some projects!):

[Symfony\Component\Filesystem\Exception\IOException] Cannot rename "H:\_Moje_dydy\Documents\Projekty\friendly_score\src_rainfin\app\cache\de_" to "H:\_Moje_dydy\Documents\Projekty\friendly_score\src_rainfin\app\cache\dev".

When I looked into NTFS permissions on some folders within the cache directory I saw complete chaos like this:

http://i.imgur.com/PSEGM72.png

Sometimes it needed very deep interventions to enable me to simply delete the cache dir (some subdirectories had so crazy permissions that even as admin user I was unable to delete them and had to mingle with permissions first).

Do you guys have any new insights on this?

[Wirone]

@kotbehemot53 _Moje_dydy, skisłem :D

[dsazup]

How is this a closed issue? This is happening on mac os as well, not only windows. And only when using Symfony 🤔

[litzinger]

The issue happens for me when running the app in a vagrant box. The error I get is

[Symfony\Component\Filesystem\Exception\IOException]
Failed to remove directory "/var/www/site.com/var/cache/de~/pools"

This is when running "php bin/console cache:clear" from within the vagrant guest. If I just do rm -rf var/cache/* from the host machine (OS X) then it works fine (and its faster).

[ptrm04]

Same here, macOS Vagrant:

[Symfony\Component\Filesystem\Exception\IOException]                         
  Failed to remove directory "/var/www/apps/appcom/app/cache/pro~/pools):   
  Directory not empty.

Looks like after deleting this pro~ directory and re-running cache:clear, the pro~ gets re-created so looks like it gets created as part of running cache:clear

Excluding the folder in PhpStorm did not help.

[ptrm04]

OK so upgrading Vagrant to 2.0.0 and mounting using NFS instead of "shared paths" has helped (macOS 10.12.6).

In your Vagrant file, mount as:
config.vm.synced_folder "/Users/you/PhpstormProjects/project", "/var/www/app", type: "nfs", :linux__nfs_options => ["rw,no_subtree_check,no_root_squash"]

Hope this helps someone!

[teeyo]

Same here on Vagrant homestead 4.0.0 and Symfony 4.0-DEV

[Symfony\Component\Filesystem\Exception\IOException] !! !! !! Failed to remove directory "/home/vagrant/code/teeyo/var/cache/de~/twig": r !! !! !! mdir(/home/vagrant/code/teeyo/var/cache/de~/twig): Directory not empty.

By the way, I had to remove the type: symfony from homestead, because apparently the server started downloading PHP scripts instead of executing them.

[axxox]

Executing php bin/console cache:clear --no-warmup did the thing for me.
If there is a problem in using composer require X in Flex (Symfony 4.1), because of "auto-script", just edit:

//composer.json
{
    "scripts": {
        "auto-scripts": {
-            "cache:clear": "symfony-cmd"
+            "cache:clear --no-warmup": "symfony-cmd"
        }
    }
}

[DHager]

Ran into this as well with Symfony 4, I suspect it's actually a VirtualBox mapped-folder problem, because the the host and container temporarily show different file listings. Symfony probably just happens to be doing something that triggers the latent bug.

Specifically:

1. I get an error saying that directory /var/www/demo-site/var/cache/de~/Container4eDXaZr cannot be removed when clearing the cache.
2. I set a breakpoint at the line in \Symfony\Component\Filesystem\Filesystem::remove for right before the rmdir() call.
3. While PHP is paused, I do an ls -al file listing for the directory on the container: Zero files!
4. I try to rmdir it from the command-line: "Directory not empty"
5. Back on the Windows host machine, I do a file listing for the directory: All the files are still there, they haven't been deleted yet!

When I copy all the files to a non-mapped folder, the problem doesn't occur, so I suspect some concurrency/race issue with the mapping. I'll see if I can boil it down to a minimal repro case before potentially disrupting things by upgrading VirtualBox.

[nicolas-grekas]

Can you try adding a few sleep before remove/rename calls in the command? Any better?

[nicolas-grekas]

But you should really upgrade before. We won't add code to deal with legacy setups.

[DHager]

@nicolas-grekas I don't have it solved, but I have some... some very interesting WTF-data to share.

First off, I analyzed what steps Symfony was trying to do and put them into a plain old PHP file with unlink() and rmdir() calls, and the problem still occurred. After a while I realized the problem was a even weirder:

• On the container, the PHP code attempts to remove files from the de~ folder
• On the container, the filesystem says both folders (de~ and dev) had the file removed!
• On the host, the filesystem says the only the dev folder lost a file -- the wrong folder!

So yeah, that's "question your sanity"-type behavior from a filesystem. As an experiment, I made a copy of de~ called de0, and changed my scripts to try to delete from de0 instead... and it worked normally. So now I'm thinking: "Is it the tilde character, or is it some leftover messup in Docker or Virtualbox caused by how those two folders get swapped around in normal usage?"

After each of these steps, I restored the "pre-test" file structure by doing a git checkout on the host, and then tried the repro script:

• Restarting the container: Same problem.
• Restarting docker-machine too: Problem disappears.
• Running cache:warmup: No problems.
• Running cache:clear: Problem comes back.

So my next step is to see at what the first point is in cache:clear where I can interrupt everything, reset the host filesystem, and get the error running the test script.

Summary

• There's some buggy state in VirtualBox and its shared filesystem.
• The buggy state causes two folders to get... very confused.
• Somehow the buggy state is triggered as a side-effect of cache:clear
• I highly doubt this is a Symfony bug, but if you don't mind I'll keep adding notes to this issue since the community seems to be affected.

[mvrhov]

DO NO USE SHARED FILESYSTEM, it's bound to cause a problems. There is a lot of bug reports here that some-sort of shared FS e.g causes a lot of weird problems,
Set up a samba in your vm environment (if you are connecting windows with linux) or NFS if you are connecting linux to linux

[DHager]

@mvrhov I'm certain that's part of the problem. However, the use-case I have here is local-development, where it's much harder to tell people "set up SMB/NFS in order to test the site." I'm also worried that if I don't really understand the bug, how will I know it's fixed if it happens to go away when I upgrade? :)

Minor additional findings:

• The problem is visible on the Docker-machine VM itself, not just in the individual container, so the image/container is probably irrelevant. (It's either Docker Machine or Virtualbox or both.)
• Unmounting and remounting the vboxfs share within the Docker-machine VM makes the problem go away -- no need to restart the entire VM.
• The mismatching occurs are the dev vs de~ tier, and not just inside the Container*/ folder.
• The bug does not cause duplicate inodes, which is too bad since it would have simplified things a bit.

[mvrhov]

This is the problem. It's the same with if you set up the vagrant with shared folder to do local dev linux to linux or vmware's shared folders. This thing is out of sync a lot of time. Just search the issues here and you'll se that this is the culprit most of the time.
I'm also running local development, where the linux server part is inside a VM but the IDE is running on Windows. And setting up the samba is piece of cake. If you get so many new developer each day then I recommend that you prepare a VM that each of them could just copy and be running in no time.

[bxct]

Got this issue during OROCommerce installation attempt using Homestead+Virtualbox. Definitely is some sort of IO bug of Virtualbox. Mounting folder as NFS helped me rather to get another type of permissions error, although might be a helpful solution for Symfony setups. Given I am not supposed to intrude in setup process, I just changed the VM provider, and finally had it to work.

[ultonic]

I had same problem. I use Vagrant on MacOs and configure it using puphpet. When I tried execute command:

sudo php bin/console cache:clear --env prod

I was getting error message:

[WARNING] Failed to remove directory "/var/www/doc/app/my_cache/pro~/pools":                                       
           rmdir(/var/www/doc/app/my_cache/pro~/pools): Directory not empty."

I was able to fix this trouble changed synced_folder type to nfs in puphpet config. Now, when I try to remove chache, I am not reciving errors.

Hope this helps someone.

[BaptWaels]

I'm using macOS. I've solved my problem using docker-machine-nfs. I was using docker-machine without nfs.

What i've done to solve the problem:

1 - Install docker-machine-nfs
2 - run once :

docker-machine-nfs <NAME_OF_YOUR_DOKCER_MACHINE> --shared-folder=<YOU_SHARED_FOLDER> --nfs-config="-alldirs -maproot=0" --mount-opts="async,noatime,actimeo=1,nolock,vers=3,udp" -f

3 - Happiness 🥇

[justincletus]

Same here on Vagrant homestead 4.0.0 and Symfony 4.0-DEV

[Symfony\Component\Filesystem\Exception\IOException] !! !! !! Failed to remove directory "/home/vagrant/code/teeyo/var/cache/de~/twig": r !! !! !! mdir(/home/vagrant/code/teeyo/var/cache/de~/twig): Directory not empty.

By the way, I had to remove the type: symfony from homestead, because apparently the server started downloading PHP scripts instead of executing them.

This is works for me.
Thanks