Magento2: Permission issues with static generated files in pub directory

[miguelbalparda]

We have been experiencing issues with permissions using the latest version of Magento 2.

It seems that it all comes from 'vendor/magento/framework/Filesystem/DriverInterface.php' which explicitly sets the umask for folders to something that isn't going to work on a lot of setups, including our own.

Is there any reason for not setting the default for directories to 0755 instead of 0770?

Our issues were mostly 403 errors inside the pub folder for assets and templates and we suspect a lot of hosting companies might face this issue too.

[mazhalai]

Hello @miguelbalparda, did you create a group as recommended? http://devdocs.magento.com/guides/v2.0/install-gde/install/file-system-perms.html

[miguelbalparda]

@mazhalai yes, we tried that. Please keep in mind this is not me as an individual but a hosting company. It also looks like we are not the only one facing this issue https://twitter.com/mbalparda/status/667029022099046400
Is there any reason in particular to use 0755 instead of 0770?

[piotrekkaminski]

@miguelbalparda can you provide more details on the setup and why you think it fails? Are the newly generated files not getting proper permissions?

[vancoz]

@miguelbalparda we changed it from 755 to 770 because we would like to close access to anybody except file owner and group. Php files can contain configuration settings, keys, etc... so it is better to keep it safe.

Please follow the instruction to set correct ownership setting http://devdocs.magento.com/guides/v2.0/install-gde/prereq/apache-user.html#install-update-depend-user-group

Thank you

[buskamuza]

Just to clarify: we recommend 0770.
The reason:

• we want the owner to have write permissions. The owner user should be used for CLI commands/operations on the application. It may include necessity to modify files
• we want the group to have write permissions to some folders. A user in the group would be the web-server user, so Web Magento application could modify necessary files (e.g., cache)

[miguelbalparda]

Adding a bunch of users to the apache group is less than ideal, as it may cause security concerns on shared platforms and in some cases it is even impossible to implement. We will investigate more about adding apache to the users group since it might be the solution.

[robfico]

The issue we see is in shared hosting where groups and ownerships cannot easily be set up in the desired fashion. With php-fpm, PHP runs as the owner. Apache runs as another less privileges user (i.e. nobody, apache, etc...). Apache can not "out of the box" pull css, js, and images (essentially static resources) from pub/static/xxx. php-fpm can read any file the owner has read permission on, so no issues with the php files themselves not being world readable.

If we add the user to the nobody group, or nobody to their group, it does work, but it alters the standard way for shared hosting, and may open up security issues. As Miguel mentioned, we added the apache group to the user's group, and that seems to work. However, this could lead to other issues if a perl/cgi based script were to be used.

[miguelbalparda]

+1 for the setup @robfico described.

[piotrekkaminski]

so @miguelbalparda @robfico would it be the right solution for you to have only the generated files (js/css/images) world readable (given they are already being sent outside, not a security issue)? and the code (including generated) could stay as is?

[miguelbalparda]

@piotrekkaminski yes, that seems to be a solid solution. So far we haven't seen a single issue with files/folders outside the pub folder.

[robfico]

Yes, I think that would work well. Since those files are publicly viewable anyway on the web, they should be world readable and directories world executable permissions wise to match. This way Apache is unrestricted in its static content access, while php-fpm can be used to manage access to mroe restricted files (xml, php, etc...). Seems like a win-win, and will avoid issues with other standard hosting in the shared world.

[lngla]

After installing Magento 2 with data and running the content deploy, all permissions on those files were created at 770 & 660. None of the content was viewable for me, including the css and product images. It took me awhile, but once I figured out to change permissions to 755 and 644, my css and images were loaded and visible. I am continuing to have this issue- every time I add a new product image permissions are pre-loaded to 770 and newly cached images both appear broken.

I would love to see the solution talked about here.

[miguelbalparda]

We started experiencing errors in the vendor dir too.

[20-Nov-2015 14:22:42 America/Detroit] PHP Fatal error: require(): Failed opening required '/chroothtml/vendor/magento/module-media-storage/registration.php' in html/vendor/composer/autoload_real.php on line 59

[CalamityJames]

We've been having this issue with our dedi server running shared hosting too, I understand the reasons for changing to 0770 and 0660 but in a WHM/cPanel environment where everything is pretty much locked down by suPHP etc, it will refuse to load because it can't access any of the .htaccess files

Here's an excerpt from the error log:

[Wed Dec 09 11:25:33.153817 2015] [core:crit] [pid 3011] (13)Permission denied: [client ***.***.***.***:5769] AH00529: /home/*******/public_html/pub/static/frontend/*********/***********/en_GB/Magento_Checkout/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/home/*******/public_html/pub/static/frontend/********/**********/en_GB/Magento_Checkout/' is executable, referer: http://************/

Having to run

find . -type f -exec chmod 644 {} \;
find . -type d -exec chmod 755 {} \;

on pub/static every time is getting tiresome!

Our environment:

OS:      CentOS 6.7
WHM:     11.52.1
PHP:     5.5.30
Handler: suphp
Apache:  2.4.16
suEXEC:  On

[CalamityJames]

The permissions are set as constants in lib/internal/Magento/Framework/Filesystem/DriverInterface.php.

Just sayin', if anyone wanted to maybe manually edit them on their install just to not have to reset the permissions until such time as there's a neat way around this. It does seem to fix the problem on my install.

_NB: I know editing core files is very bad practice, but as far as I'm aware there's no way to redefine a const in PHP. For development sites, this is probably OK. Production, I wouldn't if I were you_

[miguelbalparda]

@piotrekkaminski @mazhalai any news on this issue?

[Vinai]

There is some documentation on this issue in the devdocs.
I've found the section on setting the setgid bit has been useful for some people encountering permission issues.

[miguelbalparda]

Some more info about this issue via @ryebell. It seems symlinks are being used for some files inside pub folder (JS in our case) and a chmod/chown -R is not enough to change its permissions.

[user@server html]$ stat pub/static/frontend/Magento/luma/en_US/images/logo.svg
  File: `pub/static/frontend/Magento/luma/en_US/images/logo.svg' -> `/chroot/home/magento1/magento2-demo.nexcess.net/html/vendor/magento/theme-frontend-luma/web/images/logo.svg'
  Size: 107         Blocks: 8          IO Block: 4096   symbolic link

[user@server html]$ stat pub/static/frontend/Magento/luma/en_US/mage/requirejs/mixins.js
  File: `pub/static/frontend/Magento/luma/en_US/mage/requirejs/mixins.js' -> `/chroot/home/magento1/magento2-demo.nexcess.net/html/lib/web/mage/requirejs/mixins.js'
  Size: 85          Blocks: 8          IO Block: 4096   symbolic link

Wasn't the use of symlink highly discouraged in Magento 1?

[markshust]

Same issue as https://github.com/magento/magento2/issues/2449 -- pub directory is getting perm denied errors. I'm running Debian Jessie.

I'm trying a fresh install on a variation of my docker image which is a really clean operating system snapshot. I did manage to get this working by:

sed -i 's/0770/0775/g' vendor/magento/framework/Filesystem/DriverInterface.php
sed -i 's/0660/0664/g' vendor/magento/framework/Filesystem/DriverInterface.php
find pub -type f -exec chmod 664 {} \;
find pub -type d -exec chmod 775 {} \;

Note that I didn't run the suggested find . -type d -exec chmod 770 {} \; && find . -type f -exec chmod 660 {} \; && chmod u+x bin/magento command after the magento install, and just used default perms as they come (default is 755/644 on nix). Without setting direct perms of 775/664 on pub dir, nothing worked right.

I did setup a new magento user that belongs to the www-data group. The setgid recommendation had no effect for me.

This does seem to be an issue, as the recommended install settings don't seem to be working on a clean os image.

[pantaoran]

hmm @markoshust that file for me is located at lib/internal/Magento/Framework/Filesystem/DriverInterface.php
but I'm using 2.0.0 and may not have the latest changes.

[markshust]

@pantaoran I'm installing with composer create new project. This may be why.

This may be related issue that could possible by merged in https://github.com/magento/magento2/issues/2525

Please see my setup script here https://github.com/mageinferno/docker-magento2-setup/blob/master/m2setup.sh#L42

These lines were required to get things working on Ubuntu Jessie.

Update: I did need to setgid on var/generation after all with the recommended user/group setup/config, otherwise web user could not write to that folder.

[HaydenFromIT]

+1 to this issue.

@markoshust Thanks for the script to fix the var/generation directory. Changing the core DriverInterface.php file was required.

However, this issue still persists with the pub/static directory. Would anyone be able to point me in the right direction for a fix for this? Clearing/Flushing the cache from Admin dashboard causes some files to be regenerated with incorrect permissions.

[markshust]

Please see my post at https://github.com/mageinferno/magento2-docker-compose/issues/5#issuecomment-174817382

This seems to have resolved the owner/perm issue. That said.... I'm jumping through mini-sized hula hoops lit on fire to get all of this working. Changing perms to 775/664, ditching the secondary web server user and reverting back to www-data user, writing a wrapper for ./bin/magento to exec things for the www-data user, etc etc etc.

This is a bit ridiculous. My Docker setup executes a perfectly clean, simulated install environment that isn't prone to nuances and edge-cases. This is out-of-the-box, default stuff. The fact that none of this is working out of the box and contains overzealous perms & ownerships is extremely concerning for onboarding new developers to the M2 platform. There has also been a lack of communication from the core team here on tickets such as this one for real, actual resolutions to simplify the install process.

I'm hearing a lot of backlash from anyone who is new to M2 (even some Magento Inc. developers who are known) that can't even get a standard M2 store installed. I'm wondering if anyone is willing to start closing out these tickets here. This ticket alone is over 2 months old, and people are continuing to have the same issues still...

[CalamityJames]

This is out-of-the-box, default stuff. The fact that none of this is working out of the box and contains overzealous perms & ownerships is extremely concerning for onboarding new developers to the M2 platform.

+1 to this, I've never known such a simple install process (M1.x) to become so convoluted in one fell swoop. Hopefully someone from the team can shed some light as to why these issues persist.

As I said before, 770 and 660 do not play well with a WHM/cPanel server that runs on suPHP (most do out of the box) so there needs to be some way round this that doesn't involve editing core files.

[Vinai]

I don't think they are overzealous permissions, in fact they should be tighter (that is, the web server user should not have write permissions anywhere except where files are generated.
This is a little script which seems to work fine. I guess it won't help with WHM/cPanel installs though.

[robfico]

The write permissions are not the problem though. Removing world readable permissions from files and world executable permissions from directories on publicly available files stops the webserver from even being able to serve them in a standard php-fpm / suPHP set up where the owner of the files is not the same owner as the web server that serves static content (images, css/js files, etc...). 644 for files and 755 for directories would alleviate most of these issues.

[RawPhunky]

+1 to the issue.
Was not able to set up M2 on shared hosting.

[markshust]

@Vinai are you using php-fpm?

@robfico perhaps this is the issue -- where we are using php-fpm and others are not. I'm experiencing the same situation, where web server owner _must_ be owner of files, and perms do need 775/664 or 755/644, otherwise nothing works.

@Vinai what is the purpose of locking things down further? We've never done this for M1 (at least not that I know of), and I've never heard of security risks with making everything world executable. Are there inherent security risks that are specific to M2, and/or why do these exist in M2 and not in M1?

Regardless, I'm all for locking down everything, everywhere. But this setup just doesn't work. It's important to note the stack I'm working on:

• debian jessie
• nginx 1.9.9
• php-fpm 7.0.2

Running php-fpm is a requirement in properly setup containerized environments.

[miguelbalparda]

I don't think there is a permission schema that will work for every setup out there.
What is the downside of removing the code where permissions are set and leave the permissions to the owner of the stack? Another option could be a place to set the permissions without the need of modifying core files, pre or post install but right now there is no clean way to adapt the software to a particular setup without touching the core.

[clwells]

I've read through the Magento docs and the requirements don't seem to state if mod_php is supported (or not). Ideally it isn't and if that's the case the following solution should both work and be secure.

Notes:

• I'm using "httpuser" here for the Apache/Nginx unix user/group for the sake of making it clear. It may be nginx, nobody, apache, http, www etc.
• I'm using "enduser" here as the unix user for the sake of making it clear.

Use case: Apache OR Nginx + PHP-FPM

• Websever user: _httpuser_
• PHP user: _enduser_

Note that these steps should be done in sequence and part of a "better" Magento base install.

1) All directories and files should have ownership: enduser:enduser

2) All directories should be: 0711

^^ This allows the websever to get access to what it needs directory-wise without allowing directory listing. It also disallows Nginx/Apache from having any control over the files in those directories write-wise. This is a good thing in the case whereby a bug in Nginx/Apache could otherwise be exploited to write into PHP (or other) files. There is complete PHP/webserver isolation from a write sense.

3) All "normal files" should be: 0644

^^ By normal files I mean CSS, JS, images and other webserver servable assets. The webserver needs to read these and that's OK. That said the webserver must have the full-path to them or it can't access them otherwise (because of our 0711 directory permissions).

4) All PHP scripts should be: 0600

^^ The webserver never needs to even read these. Only PHP-FPM does so we've further isolated PHP scripts from even being read by the webserver.

5) All sensitive files read only by PHP should be: 0600

^^ This means files in app/etc (at least) and any other file that can be read-only by PHP.

What does this mean for https://github.com/magento/magento2/blob/1d97cd9ddee038c9b93c43b74c8e318702a70a33/lib/internal/Magento/Framework/Filesystem/DriverInterface.php#L20 ? If the above steps were taken you can make the following change:

/**
 * Permissions to give read/write/execute access to owner and owning group, but not to all users
 */
const WRITEABLE_DIRECTORY_MODE = 0770;

/**
 * Permissions to give read/write access to owner and owning group, but not to all users
 */
const WRITEABLE_FILE_MODE = 0660;

becomes:

/**
 * Permissions to give read/write/execute access to owner and owning group, but not to all users
 */
const WRITEABLE_DIRECTORY_MODE = 0711;

/**
 * Permissions to give read/write access to owner and owning group, but not to all users
 */
const WRITEABLE_FILE_MODE = 0644;

• New directories are created 711 - the least perms you can reasonably give but leave the webserver able to access.
• New files are created 644 - again, the least perms you can reasonably give but leave the webserver able to read.
• User separation is assumed based on FPM running as _enduser_ and the webserver running as _httpuser_.

All of this said - why not just make this a configuration setting instead of hardcoding the values? It will allow for a variety of different environments and configurations and ultimately give the end-user the control they need.

Chris

[Vinai]

@markoshust I've always removed write permissions from the webserver user. It helps indeed. Just recently I got to see a hacked M1 site where the malicious code was injected into lib/Varien/Object.php so it was executed on every request (it wasn't a client of mine). That attack wouldn't have been possible if the webserver user wouldn't have been able to write to the PHP file.
The webserver user should not be able to write to any PHP files in my opinion. That isn't possible in Magento 2 because of code generation, but it doesn't prevent me from locking down all non-generated files.
When I say webserver user, that also refers to the user id php-fpm would be running under if that is used.
Regarding your question about php-fpm: on my dev instance I use apache + php-fpm, but my Magento 1 clients use a wide range of setups (ranging from mod_php over nginx + php-fpm to apache + php-fpm in a variety of server configurations).

I haven't yet been involved in a live Magento 2 project.

[markshust]

@clwells hit everything on the head here -- 100% agree with his findings and suggestions.

@Vinai thanks for the explanation, starting to understand this now. I really like @clwells's findings, going to implement these best practices into my Docker suite.

[fooman]

@clwells thanks for chiming in here. One thing which I think further complicates things is that M2 also writes php files under /var/generation. So having

New files are created 644 - again, the least perms you can reasonably give but leave the webserver able to read.

would write those as 644 but would then contradict

4) All PHP scripts should be: 0600

right?

[clwells]

4) All PHP scripts should be: 0600
right?

It would contradict it yes, but it would not complicate anything. While ideally all PHP scripts remain 0600 things would work just fine were PHP scripts 0644. The side effect here would be that the webservers (and other services) could read the PHP. For the sake of having M2 do something workable for all I'd take the 0644 PHP scripts in /var/generation if it meant we got "fixed" permissions otherwise.

Chris

[korzynski]

Is there any workaround at this time that will get Magento 2.0 working in a shared hosting environment?

[ganey]

+1 on not getting it to work on a shared hosting environment. Tried cPanel and plesk, and no luck, constant 403 errors due to file permissions

[grasdaggel]

[ganey]

[alankent]

[Mag2]

[dsikkema-magento]

[astorm]

[Marko-M]

[johnhughes1984]

[swieczorek]

[brideo]

[siment]

[miguelbalparda]

[dracon80]

[benmarks]

[dracon80]

[glenara]

[romeof1980]

[benmarks]

[dracon80]

[WesMage]

[WesMage]

[astorm]

[clwells]

[markshust]

[benmarks]

[romeof1980]

[ClaudiuCreanga]

[fooman]

[miguelbalparda]

[dsikkema]

[astorm]

[benmarks]

[CalamityJames]

[miguelbalparda]

[miguelbalparda]

[wbyrnetx]

[tdgroot]

[wbyrnetx]

[mehh]

[comprensys]

[magedmik]

[sammarcus]

[joeNmass]

[markshust]

[crantron]

[piotrekkaminski]

[markshust]

[piotrekkaminski]

[markshust]

[pen65978]

[miguelbalparda]

[ryantfowler]

[elenleonova]

[miguelbalparda]

[swapnilagarwwal]

[dsikkema-magento]

[devamitbera]

[dsikkema-magento]

[nmallepally]

[romeof1980]

[nmallepally]

[romeof1980]

[giappv]