Magento2: Magento ver. 2.1.0 - Cron /bin/magento cron:run Permission denied in /vendor/colinmollenhour/cache-backend-file/File.php

[wired-circuit]

Magento ver. 2.1.0, Implementation of cron as per documentation on [(http://devdocs.magento.com/guides/v2.0/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-group-run)]

* * * * * /usr/bin/php /var/www/magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/magento2/var/log/magento.cron.log
* * * * * /usr/bin/php /var/www/magento2/update/cron.php >> /var/www/magento2/var/log/update.cron.log
* * * * * /usr/bin/php /var/www/magento2/bin/magento setup:cron:run >> /var/www/magento2/var/log/setup.cron.log

Results in error log reporting the following when line one in cron runs, the other two run without issues.

Subject: Cron <[email protected]> /usr/bin/php /var/www/vhosts/magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/vhosts/magento2/var/log/magento.cron.log
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
X-Cron-Env: <LANG=en_US.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/ec2-user>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=ec2-user>
X-Cron-Env: <USER=ec2-user>
[Exception]                                                                                                                                                                                                                       
Warning: file_put_contents(/var/www/vhosts/magento2/var/cache//mage-tags/mage---1b8_CONFIG): failed to open stream: Permission denied in /var/www/vhosts/magento2/vendor/colinmollenhour/cache-backend-file/File.php on line 663  

cron:run [--group="..."] [--bootstrap="..."]

[ihor-sviziev]

@wired-circuit looks like you skipped step with setting permissions.
http://devdocs.magento.com/guides/v2.0/install-gde/prereq/file-system-perms.html

[wired-circuit]

Don't think so, if I had then the pre-installation check would have failed and the installation would not have commenced.

I have rerun these commands as per the documentation but with the problem persisting.;

cd /var/www/vhosts/magento2/
find var vendor pub/static pub/media app/etc -type f -exec chmod u+w {} \;
find var vendor pub/static pub/media app/etc -type d -exec chmod u+w {} \;
chmod u+x bin/magento

If there are additional commands to be run can you assist so that they are added to the documentation. I am certainly no Linux SysAdmin, so presumption in documentation or assistance will not fix the issue or assist anyone else who followed the same instruction.

[ihor-sviziev]

@wired-circuit could you execute following commands and put result there?

ls -lha /var/www/vhosts/magento2/

I think owner for files isn't correct for you.

[wired-circuit]

@ihor-sviziev Here you go. Thank you for all your assistance.

[ec2-user ~]$ ls -lha /var/www/vhosts/magento2/
total 4.7M
drwxrwsr-x 15 root www 4.0K Aug 28 17:13 .
drwxrwsr-x  3 root www 4.0K Aug 27 16:16 ..
-rw-rw-r--  1 root www 3.8M Aug 28 08:19 access_log
drwxrwsr-x  4 root www 4.0K Jun 23 09:41 app
drwxrwsr-x  2 root www 4.0K Jun 23 09:41 bin
-rw-rw-r--  1 root www 425K Jun 23 09:41 CHANGELOG.md
-rw-rw-r--  1 root www 1.9K Aug 28 09:11 composer.json
-rw-rw-r--  1 root www 346K Aug 28 09:12 composer.lock
-rw-rw-r--  1 root www 3.4K Jun 23 09:41 CONTRIBUTING.md
-rw-rw-r--  1 root www  631 Jun 23 09:41 COPYING.txt
drwxrwsr-x  5 root www 4.0K Jun 23 09:41 dev
-rw-rw-r--  1 root www  26K Aug 28 07:18 error_log
-rw-rw-r--  1 root www 1.1K Jun 23 09:39 .gitignore
-rw-rw-r--  1 root www 2.8K Jun 23 09:41 Gruntfile.js.sample
-rw-rw-r--  1 root www  11K Aug 28 17:13 .htaccess
-rw-rw-r--  1 root www 9.2K Jun 23 09:41 .htaccess.sample
-rw-rw-r--  1 root www 1.4K Aug 26 07:17 htaccess_tester.php
-rw-rw-r--  1 root www 1.4K Aug 25 17:38 index.php
-rw-rw-r--  1 root www  315 Jun 23 09:41 ISSUE_TEMPLATE.md
drwxrwsr-x  4 root www 4.0K Jun 23 09:41 lib
-rw-rw-r--  1 root www  11K Jun 23 09:41 LICENSE_AFL.txt
-rw-rw-r--  1 root www  11K Jun 23 09:41 LICENSE.txt
drwxrwsr-x  4 root www 4.0K Aug 26 07:59 media
-rw-rw-r--  1 root www 5.2K Jun 23 09:41 nginx.conf.sample
-rw-rw-r--  1 root www 1.4K Jun 23 09:41 package.json.sample
-rw-rw-r--  1 root www 1.7K Jun 23 09:41 .php_cs
-rw-rw-r--  1 root www  804 Jun 23 09:41 php.ini.sample
drwxrwsr-x  2 root www 4.0K Jun 23 09:41 phpserver
drwxrwsr-x  6 root www 4.0K Jun 23 09:41 pub
-rw-rw-r--  1 root www  289 Aug 27 14:05 robots.txt
drwxrwsr-x  7 root www 4.0K Jun 23 09:41 setup
-rw-rw-r--  1 root www  884 Jun 23 09:41 .travis.yml
drwxrwsr-x  7 root www 4.0K Jun 23 09:39 update
drwxrwsr-x 14 root www 4.0K Aug 28 18:44 var
drwxrwsr-x 33 root www 4.0K Aug 28 09:12 vendor

Could we work on producing a full and complete list of commands, to assist anyone else with the same issue(s)?

Previous to this I have run these commands based on documentation from Amazon and Magento;

[(http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-LAMP.html#SettingFilePermissions)]
[http://devdocs.magento.com/guides/v2.1/install-gde/prereq/file-system-perms.html]

[ec2-user ~]$ sudo groupadd www
[ec2-user ~]$ sudo usermod -a -G www ec2-user
[ec2-user ~]$ exit
[ec2-user ~]$ groups
[ec2-user ~]$ wheel www
[ec2-user ~]$ sudo chown -R root:www /var/www
[ec2-user ~]$ sudo chmod 2775 /var/www
[ec2-user ~]$ find /var/www -type d -exec sudo chmod 2775 {} \;
[ec2-user ~]$ find /var/www -type f -exec sudo chmod 0664 {} \;
[ec2-user ~]$ sudo su
[root ~]$ cd /var/www/vhosts/magento2/
[root ~]$ find var vendor pub/static pub/media app/etc -type f -exec chmod u+w {} \;
[root ~]$ find var vendor pub/static pub/media app/etc -type d -exec chmod u+w {} \;
[root ~]$ chmod u+x bin/magento
[root ~]$ exit
[ec2-user ~]$ touch /var/www/vhosts/magento2/var/log/magento.cron.log
[ec2-user ~]$ touch /var/www/vhosts/magento2/var/log/update.cron.log
[ec2-user ~]$ touch /var/www/vhosts/magento2/var/log/setup.cron.log
[ec2-user ~]$ sudo su
[root ~]$ export EDITOR=nano
[root ~]$ crontab -u ec2-user -e
[root ~]$ crontab -u ec2-user -l
* * * * * /usr/bin/php /var/www/vhosts/magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/vhosts/magento2/var/log/magento.cron.log
* * * * * /usr/bin/php /var/www/vhosts/magento2/update/cron.php >> /var/www/vhosts/magento2/var/log/update.cron.log
* * * * * /usr/bin/php /var/www/vhosts/magento2/bin/magento setup:cron:run >> /var/www/vhosts/magento2/var/log/setup.cron.log

[ihor-sviziev]

@wired-circuit it's not good idea to use user root as owner. Would be great to use apache or www-data user as owner.
Also cron jobs should be ran from that user

[veloraven]

@wired-circuit thank you for your feedback.
Please, format this issue according to the Issue reporting guidelines: with steps to reproduce, actual result and expected result.
Please, also provide the used Magento version.

[wired-circuit]

I have updated the title and content to include the version number 2.1.0 as requested. The steps are followed are listed (in full) and follow the documentation provided by the host Amazon and those provided by Magento, both URL's are listed. My approach has not been to just resolve the issue, but to provide updated documentation for others which is formatted better and without presumption that you are a Linux SysAdmin. The internet is full of posts with the same or very similar problems, so the documentation clearly falls short.

@ihor-sviziev Based on what you have written and reading between the lines I should do the following;

[ec2-user ~]$ sudo chown -R **ec2-user**:www /var/www
[ec2-user ~]$ sudo chmod 2775 /var/www
[ec2-user ~]$ find /var/www -type d -exec sudo chmod 2775 {} \;
[ec2-user ~]$ find /var/www -type f -exec sudo chmod 0664 {} \;

Is that correct? Since the ec2-user is already a member of the www group.

[ihor-sviziev]

@wired-circuit Right now it's correct

[wired-circuit]

Perfect! I did that, cleaned down the log and waited.. There are no errors now and I think that may well have also solved the performance issue I was experiencing. Thank you @ihor-sviziev for your input. Maybe the Magento document author could update with some more notes?

[wired-circuit]

Nope, its back again;

There has been an error processing your request
Exception printing is disabled by default for security reasons.
Error log record number: 1261900375713

a:4:{i:0;s:224:"Warning: file_put_contents(/var/www/vhosts/magento2/var/cache//mage-tags/mage---1b8_CONFIG): failed to open stream: Permission denied in /var/www/vhosts/magento2/vendor/colinmollenhour/cache-backend-file/File.php $
#1 /var/www/vhosts/magento2/vendor/colinmollenhour/cache-backend-file/File.php(663): file_put_contents('/var/www/vhosts...', '1b8_FRONTEND__R...', 10)
#2 /var/www/vhosts/magento2/vendor/colinmollenhour/cache-backend-file/File.php(179): Cm_Cache_Backend_File->_updateIdsTags(Array, Array, 'merge')
#3 /var/www/vhosts/magento2/vendor/magento/zendframework1/library/Zend/Cache/Core.php(390): Cm_Cache_Backend_File->save('a:0:{}', '1b8_FRONTEND__R...', Array, NULL)
#4 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Core.php(74): Zend_Cache_Core->save('a:0:{}', 'FRONTEND::RESOU...', Array, NULL, 8)
#5 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Adapter/Zend.php(47): Magento\Framework\Cache\Core->save('a:0:{}', 'FRONTEND::RESOU...', Array, NULL)
#6 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/Bare.php(75): Magento\Framework\Cache\Frontend\Adapter\Zend->save('a:0:{}', 'frontend::resou...', Array, NULL)
#7 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/TagScope.php(49): Magento\Framework\Cache\Frontend\Decorator\Bare->save('a:0:{}', 'frontend::resou...', Array, NULL)
#8 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/Bare.php(75): Magento\Framework\Cache\Frontend\Decorator\TagScope->save('a:0:{}', 'frontend::resou...', Array, NULL)
#9 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/Bare.php(75): Magento\Framework\Cache\Frontend\Decorator\Bare->save('a:0:{}', 'frontend::resou...', Array, NULL)
#10 /var/www/vhosts/magento2/vendor/magento/framework/App/Cache/Type/AccessProxy.php(85): Magento\Framework\Cache\Frontend\Decorator\Bare->save('a:0:{}', 'frontend::resou...', Array, NULL)
#11 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/Bare.php(75): Magento\Framework\App\Cache\Type\AccessProxy->save('a:0:{}', 'frontend::resou...', Array, NULL)
#12 /var/www/vhosts/magento2/vendor/magento/framework/Cache/Frontend/Decorator/TagScope.php(49): Magento\Framework\Cache\Frontend\Decorator\Bare->save('a:0:{}', 'frontend::resou...', Array, NULL)
#13 /var/www/vhosts/magento2/var/generation/Magento/Framework/App/Cache/Type/Config/Proxy.php(103): Magento\Framework\Cache\Frontend\Decorator\TagScope->save('a:0:{}', 'frontend::resou...', Array, NULL)
#14 /var/www/vhosts/magento2/vendor/magento/framework/Config/Data/Scoped.php(105): Magento\Framework\App\Cache\Type\Config\Proxy->save('a:0:{}', 'frontend::resou...')
#15 /var/www/vhosts/magento2/vendor/magento/framework/Config/Data/Scoped.php(81): Magento\Framework\Config\Data\Scoped->_loadScopedData()
#16 /var/www/vhosts/magento2/vendor/magento/framework/App/ResourceConnection/Config.php(60): Magento\Framework\Config\Data\Scoped->get()
#17 /var/www/vhosts/magento2/var/generation/Magento/Framework/App/ResourceConnection/Config/Proxy.php(95): Magento\Framework\App\ResourceConnection\Config->getConnectionName('default')
#18 /var/www/vhosts/magento2/vendor/magento/framework/App/ResourceConnection.php(91): Magento\Framework\App\ResourceConnection\Config\Proxy->getConnectionName('default')
#19 /var/www/vhosts/magento2/vendor/magento/framework/Model/ResourceModel/Db/AbstractDb.php(322): Magento\Framework\App\ResourceConnection->getConnection('default')
#20 /var/www/vhosts/magento2/vendor/magento/framework/Module/ModuleResource.php(81): Magento\Framework\Model\ResourceModel\Db\AbstractDb->getConnection()
#21 /var/www/vhosts/magento2/vendor/magento/framework/Module/DbVersionInfo.php(55): Magento\Framework\Module\ModuleResource->getDbVersion('Magento_Store')
#22 /var/www/vhosts/magento2/vendor/magento/framework/Module/DbVersionInfo.php(79): Magento\Framework\Module\DbVersionInfo->isSchemaUpToDate('Magento_Store')
#23 /var/www/vhosts/magento2/vendor/magento/framework/Module/Plugin/DbStatusValidator.php(55): Magento\Framework\Module\DbVersionInfo->getDbVersionErrors()
#24 /var/www/vhosts/magento2/vendor/magento/framework/Interception/Chain/Chain.php(67): Magento\Framework\Module\Plugin\DbStatusValidator->aroundDispatch(Object(Magento\Framework\App\FrontController\Interceptor), Object(Closure),$
#25 /var/www/vhosts/magento2/vendor/magento/framework/Interception/Chain/Chain.php(63): Magento\Framework\Interception\Chain\Chain->invokeNext('Magento\\Framewo...', 'dispatch', Object(Magento\Framework\App\FrontController\Interc$
#26 /var/www/vhosts/magento2/vendor/magento/module-page-cache/Model/App/FrontController/BuiltinPlugin.php(73): Magento\Framework\Interception\Chain\Chain->Magento\Framework\Interception\Chain\{closure}(Object(Magento\Framework\Ap$
#27 /var/www/vhosts/magento2/vendor/magento/framework/Interception/Chain/Chain.php(67): Magento\PageCache\Model\App\FrontController\BuiltinPlugin->aroundDispatch(Object(Magento\Framework\App\FrontController\Interceptor), Object(C$
#28 /var/www/vhosts/magento2/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\Interception\Chain\Chain->invokeNext('Magento\\Framewo...', 'dispatch', Object(Magento\Framework\App\FrontController\Inter$
#29 /var/www/vhosts/magento2/vendor/magento/module-page-cache/Model/App/FrontController/VarnishPlugin.php(55): Magento\Framework\App\FrontController\Interceptor->Magento\Framework\Interception\{closure}(Object(Magento\Framework\A$
#30 /var/www/vhosts/magento2/vendor/magento/framework/Interception/Interceptor.php(142): Magento\PageCache\Model\App\FrontController\VarnishPlugin->aroundDispatch(Object(Magento\Framework\App\FrontController\Interceptor), Object($
#31 /var/www/vhosts/magento2/var/generation/Magento/Framework/App/FrontController/Interceptor.php(26): Magento\Framework\App\FrontController\Interceptor->___callPlugins('dispatch', Array, Array)
#32 /var/www/vhosts/magento2/vendor/magento/framework/App/Http.php(135): Magento\Framework\App\FrontController\Interceptor->dispatch(Object(Magento\Framework\App\Request\Http))
#33 /var/www/vhosts/magento2/vendor/magento/framework/App/Bootstrap.php(258): Magento\Framework\App\Http->launch()
#34 /var/www/vhosts/magento2/index.php(39): Magento\Framework\App\Bootstrap->run(Object(Magento\Framework\App\Http))
#35 {main}";s:3:"url";s:44:"/url.html";s:11:"script_name";s:10:"/index.php";}

[ihor-sviziev]

@wired-circuit From which user are you executing your php-fpm or apache? He should be owner

[wired-circuit]

I just changed again to ec2-user:apache;

[ec2-user ~]$ sudo chown -R ec2-user:apache /var/www
[ec2-user ~]$ sudo chmod 2775 /var/www
[ec2-user ~]$ find /var/www -type d -exec sudo chmod 2775 {} \;
[ec2-user ~]$ find /var/www -type f -exec sudo chmod 0664 {} \;
[ec2-user ~]$ php bin/magento cache:clean
[ec2-user ~]$ php bin/magento cache:flush
[ec2-user ~]$ php bin/magento cache:enable

Disabled the ec2-user cron job and enabled as the apache user account;

[ec2-user ~]$ sudo su
[root ~]$ export EDITOR=nano
[root ~]$ crontab -u apache -e
[root ~]$ crontab -u apache -l
* * * * * /usr/bin/php /var/www/vhosts/magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/vhosts/magento2/var/log/magento.cron.log
* * * * * /usr/bin/php /var/www/vhosts/magento2/update/cron.php >> /var/www/vhosts/magento2/var/log/update.cron.log
* * * * * /usr/bin/php /var/www/vhosts/magento2/bin/magento setup:cron:run >> /var/www/vhosts/magento2/var/log/setup.cron.log

Which brings the site back online. Commands like "php bin/magento cache:flush" work, but only for a period of time, then after ten or more minutes the problem returns. Running php bin/magento cache:flush for example returns error messages of this sort;

[Exception]                                                                                                                                                                                                                       
  Warning: file_put_contents(/var/www/vhosts/magento2/var/cache//mage-tags/mage---1b8_CONFIG): failed to open stream: Permission denied in /var/www/vhosts/magento2/vendor/colinmollenhour/cache-backend-file/File.php on line 663 

Which I can only presume is related to the same issue?

This is the 'odd' bit, it I wait another ten minutes or so then run the same command with absolutely not changes, it runs without issue;

[wired-circuit]

I am abandoning this project like so many others before me. This product is unfit for market and has been badly constructed. Poor documentation only makes matters worse. It is clunky, slow and the path to production poorly developed with little consideration for those that may not be season Linux Systems Administrators. For the record I have worked around the globe as a Senior Windows Sys Admin for more than 20 years.

[juhanix]

@wired-circuit
why don't you let magento run as www-data:www-data? It just makes things easier.

But to be honest I had the same issue after running the setup:static-content:deploy command.
Trying to open the magento2-instance in chrome: Permission Denied: Some Cache File has the ownership root:root with restricted rights -> execution error, because webserver-user can't open/read it.

The problem was that I executed the command as root-user. All php bin/magento commands should be run as file-owner/webserver-user, e.g. as www-data.

just simply execute magento commands like:
sudo -u www-data php bin/magento setup:static-content:deploy de_DE
sudo -u www-data php bin/magento cache:flush
and so on...

that should do the trick. hopefully it's not too late for you!

[wired-circuit]

I did all that, followed the documentation to the letter. In the end I have trashed the EC2 instance and commissioned a preconfigured Bitnami Instance with Magento 2.0.6, which works perfectly in every way I would expect.

[giappv]

I used apache as webserver and this cron setting fixed my issue:

* * * * * www-data /usr/bin/php /path to magento2/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /path to magento2/var/log/magento.cron.log
* * * * * www-data /usr/bin/php /path to magento2/update/cron.php >> /path to magento2/var/log/update.cron.log
* * * * * www-data /usr/bin/php /path to magento2/bin/magento setup:cron:run >> /path to magento2/var/log/setup.cron.log

I run a magento 2 instance on aws ubuntu 14.04 trusty

[tschirmer]

This is continuing for me as well.
For my setup:

• File user is centos
• Web server user is apache
• var/cache is set to 2770 (770 with a GUID sticky bit)

What happens is:

• Cron is run before the frontend and any cache files created have the file user (e.g. ec2-user / centos) as the owner and group.
• The web server (running php-fpm) tries to create a cache file under the cache dir.
  eg: var/cache/mage--0

Let's review it now:
var/cache/ <--- 2770
var/cache/mage--0 <--- 0770

This means we get this behaviour:
var/cache/

BEFORE CRON is run:
[centos@ip-10-0-48-6 cache]$ touch afile
[centos@ip-10-0-48-6 cache]$ ls -al
-rw-rw-r--. 1 centos apache 0 Jan 16 04:49 afile

AFTER CRON is run:
[centos@ip-10-0-48-6 cache]$ ls -al
-rw-rw-r--. 1 centos apache 0 Jan 16 04:49 afile
drwxrwxrwx. 2 centos apache 149 Jan 16 04:23 mage--0
drwxrwxrwx. 2 centos apache 147 Jan 16 02:39 mage--1
drwxrwxrwx. 2 centos apache 92 Jan 16 02:37 mage--2
drwxrwxrwx. 2 centos apache 119 Jan 16 02:40 mage--3
drwxrwxrwx. 2 centos apache 202 Jan 16 02:50 mage--4
drwxrwxrwx. 2 centos apache 118 Jan 16 02:37 mage--5
drwxrwxrwx. 2 centos apache 262 Jan 16 03:46 mage--6
drwxrwxrwx. 2 centos apache 39 Jan 16 02:37 mage--7
drwxrwxrwx. 2 centos apache 208 Jan 16 02:40 mage--8
drwxrwxrwx. 2 centos apache 197 Jan 16 02:37 mage--9
drwxrwxrwx. 2 centos apache 145 Jan 16 02:37 mage--a
drwxrwxrwx. 2 centos apache 72 Jan 16 02:37 mage--c
drwxrwxrwx. 2 centos apache 123 Jan 16 02:44 mage--d
drwxrwxrwx. 2 centos apache 114 Jan 16 02:43 mage--e
drwxrwxrwx. 2 centos apache 191 Jan 16 04:41 mage--f
drwxrwxrwx. 2 centos apache 4096 Jan 16 02:50 mage-tags

Sort of looks good HOWEVER:
[centos@ip-10-0-48-6 cache]$ cd mage--0

[centos@ip-10-0-48-6 mage--0]$ ls -al
-rw-rw----. 1 centos centos 281 Jan 16 04:50 mage---dac_APP_35C0BF370AE35F8C482DC31B8A81FC2F
-rw-rw----. 1 apache apache 116 Jan 16 02:46 mage---dac_FRONTEND__RESOURCESCACHE
-rw-rw----. 1 centos centos 27864 Jan 16 02:37 mage---dac_GLOBAL__EVENT_CONFIG_CACHE

we have the wrong user/groups on these files:
-rw-rw----. 1 centos centos 27864 Jan 16 02:37 mage---dac_GLOBAL__EVENT_CONFIG_CACHE

the cron will be able to write but the webserver won't.

We need to make a pull request so that all folders created BY magento in the cache has that sticky bit for 2 user systems.

AS a workaround, I'm running the cron as the webserver user (as @giappv commented).

Still super frustrating though.