Origin: Permission denied in pod

[bkmagnetron]

Trying to deploy Wordpress using https://github.com/openshift/origin/tree/master/examples/wordpress

Fedora 23
Docker version 1.9.1, build ee06d03/1.9.1
openshift v1.3.0-alpha.1
kubernetes v1.3.0-alpha.1-331-g0522e63
etcd 2.3.0

# sestatus -v

SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      30

Process contexts:
Current context:                unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Init context:                   system_u:system_r:init_t:s0
/usr/sbin/sshd                  system_u:system_r:sshd_t:s0-s0:c0.c1023

File contexts:
Controlling terminal:           unconfined_u:object_r:user_devpts_t:s0
/etc/passwd                     system_u:object_r:passwd_file_t:s0
/etc/shadow                     system_u:object_r:shadow_t:s0
/bin/bash                       system_u:object_r:shell_exec_t:s0
/bin/login                      system_u:object_r:login_exec_t:s0
/bin/sh                         system_u:object_r:bin_t:s0 -> system_u:object_r:shell_exec_t:s0
/sbin/agetty                    system_u:object_r:getty_exec_t:s0
/sbin/init                      system_u:object_r:bin_t:s0 -> system_u:object_r:init_exec_t:s0
/usr/sbin/sshd                  system_u:object_r:sshd_exec_t:s0

# oc get scc restricted

NAME         PRIV      CAPS      SELINUX     RUNASUSER   FSGROUP     SUPGROUP   PRIORITY   READONLYROOTFS   VOLUMES
restricted   false     []        MustRunAs   RunAsAny    MustRunAs   RunAsAny   <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim secret]

_MySQL Pod_

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    name: mysql
spec:
  containers:
    - resources:
        limits :
          cpu: 0.5
      image: openshift/mysql-55-centos7
      name: mysql
      env:
        - name: MYSQL_ROOT_PASSWORD
          value: yourpassword
        - name: MYSQL_USER
          value: wp_user
        - name: MYSQL_PASSWORD
          value: wp_pass
        - name: MYSQL_DATABASE
          value: wp_db
      ports:
        - containerPort: 3306
          name: mysql
      volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql/data
  volumes:
    - name: mysql-persistent-storage
      persistentVolumeClaim:
        claimName: claim-mysql

_Persitent volume_

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0002
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    server: 10.28.27.36
    path: /home/data/pv0002

# oc status -v

In project wordpress on server https://10.28.27.36:8443

svc/mysql - 172.30.189.150:3306
  pod/mysql runs openshift/mysql-55-centos7

svc/wpfrontend - 172.30.3.189:5055 -> wordpress
  pod/wordpress runs wordpress

Errors:
  * container "wordpress" in pod/wordpress is crash-looping

    The container is starting and exiting repeatedly. This usually means the container is unable
    to start, misconfigured, or limited by security restrictions. Check the container logs with

      oc logs wordpress -c wordpress

Warnings:
  * container "mysql" in pod/mysql has restarted within the last 10 minutes

View details with 'oc describe <resource>/<name>' or list everything with 'oc get all'.

# oc logs mysql

---> 08:38:05     Processing MySQL configuration files ...
---> 08:38:05     Initializing database ...
---> 08:38:05     Running mysql_install_db ...
mkdir: cannot create directory '/var/lib/mysql/data/mysql': Permission denied
chmod: cannot access '/var/lib/mysql/data/mysql': No such file or directory
mkdir: cannot create directory '/var/lib/mysql/data/test': Permission denied
chmod: cannot access '/var/lib/mysql/data/test': No such file or directory
160602  8:38:07 [Note] /opt/rh/mysql55/root/usr/libexec/mysqld (mysqld 5.5.45) starting as process 44 ...
160602  8:38:07 [Warning] Can't create test file /var/lib/mysql/data/mysql.lower-test
160602  8:38:07 [Warning] Can't create test file /var/lib/mysql/data/mysql.lower-test
ERROR: 1049  Unknown database 'mysql'
160602  8:38:07 [ERROR] Aborting
...

[deads2k]

@bkmagnetron What are the permissions on the /home/data/pv0002 directory?

@markturansky @pweil- fsgroup or just the permissions problem of old: https://docs.openshift.org/latest/install_config/persistent_storage/persistent_storage_nfs.html#nfs-volume-security ?

[markturansky]

@childsb

[bkmagnetron]

@deads2k same thing found on
https://github.com/openshift/origin/tree/master/examples/wordpress/nfs

# chmod -R 777 /home/data/

# cat /etc/exports
/home/data/pv0001 *(rw,sync)
/home/data/pv0002 *(rw,sync)

# setsebool -P virt_use_nfs 1

# oc get scc

NAME               PRIV      CAPS      SELINUX     RUNASUSER          FSGROUP     SUPGROUP    PRIORITY   READONLYROOTFS   VOLUMES
anyuid             false     []        MustRunAs   RunAsAny           RunAsAny    RunAsAny    10         false            [configMap downwardAPI emptyDir persistentVolumeClaim secret]
hostaccess         false     []        MustRunAs   MustRunAsRange     MustRunAs   RunAsAny    <none>     false            [configMap downwardAPI emptyDir hostPath persistentVolumeClaim secret]
hostmount-anyuid   false     []        MustRunAs   RunAsAny           RunAsAny    RunAsAny    <none>     false            [configMap downwardAPI emptyDir hostPath nfs persistentVolumeClaim secret]
hostnetwork        false     []        MustRunAs   MustRunAsRange     MustRunAs   MustRunAs   <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim secret]
nonroot            false     []        MustRunAs   MustRunAsNonRoot   RunAsAny    RunAsAny    <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim secret]
privileged         true      []        RunAsAny    RunAsAny           RunAsAny    RunAsAny    <none>     false            [*]
restricted         false     []        MustRunAs   RunAsAny           MustRunAs   RunAsAny    <none>     false            [configMap downwardAPI emptyDir persistentVolumeClaim secret]

Sorry accidentally clicked Close.

[bkmagnetron]

@deads2k @markturansky @pweil @childsb
I can able to mount and write to nfs from other system. I'm getting this error only in the pods. Can anyone please help me solve this problem.

[bkmagnetron]

From Centos 7 I did the following steps and everything worked perfectly. Important steps are 3, 4 and 5. And there is no need to edit the restricted scc to RunAsAny.

Step 1:

# chkconfig docker on
# systemctl enable rpcbind
# systemctl enable nfs-server

and reboot

Step 2:
start cluster

Step 3:

# mkdir -p /home/data/pv0001
# mkdir -p /home/data/pv0002
# chmod -R 777 /home/data/
# chown -R nfsnobody:nfsnobody /home/data/

Step 4:

# cat /etc/exports
/home/data/pv0001 *(rw,sync,no_root_squash)
/home/data/pv0002 *(rw,sync,no_root_squash)

# exportfs -a
# setsebool -P virt_use_nfs 1

Step 5:
and from wordpress project as system:admin

# oadm policy add-scc-to-user anyuid -z default

which means allow the containers in this project to run as root.

Step 6:
deploy wordpress

Also closes #9737

[ankushchadha]

Is there a way to avoid 777 permission?

[beezerbt]

I am having the same issue but for MQ.
I tested the image on Ubuntu and it works fine.

java.io.FileNotFoundException: /opt/activemq/data/activemq.log (Permission denied)
Caused by: java.io.IOException: Failed to create directory '/opt/activemq/data/kahadb'

I am on Openshift v3.7.
I am only using the console to deploy from the image.
The image is verified as mentioned above on Ubuntu, but, with one crucial difference between the two (Ubuntu and OpenShift) which is that I kick off the ubuntu with command line parameters, but I do not do so for Openshift, since I am using the console and not oc CLI with a template.

Any help would be much appreciated, as this image should IMO, be deployable from the console without any Yaml scripting.

[beezerbt]

I found the problem.
On the FREE West Coast (Oregon) Openshift v3.7 the above /data directory is not writeable.
I experimented with one of the Openshift v3.7 used in the Playgrounds (this is the tutorial environments that openshift offers to enable hands-on learning) and set the .../data environment variable (APACHEMQ_DATA) to "/tmp". It solved the issue.
Doing the SAME thing for the FREE West Coast openshift, which I use for learning the logs report that /tmp is NOT Writeable. This is bonkers. It could be that I need a dedicated volume for this...but ... I mean ... /tmp not being writeable?

[kingsudhu]

i have issue on mount path on gcp. inside pod they will not give permission