Origin: Mounting file from ConfigMap using volume subPath causes file permission denied error
I'm trying to mount a single file from a ConfigMap into a container directory that already contains other files. Followed https://stackoverflow.com/a/43404857 it seems like it should be possible to do this using volume subPath, however, a permission denied error is shown when I try to open the file. Maybe I'm doing it wrong?
Version
oc v3.6.0+c4dd4cf
kubernetes v1.6.1+5115d708d7
features: Basic-Auth
Server XXX
openshift v3.6.0+c4dd4cf
kubernetes v1.6.1+5115d708d7
Steps To Reproduce
- Create a configMap:
apiVersion: v1 data: hello.txt: This is a test file kind: ConfigMap metadata: name: myconfig - Deploy a test pod, e.g. redis, and edit the deploymentConfig yaml to have:
volumes: <ul> <li>configMap:<br /> name: myconfig<br /> name: myconfig<br /> ....<br /> containers:</li> <li>volumeMounts:<br /> <ul><br /> <li>mountPath: /var/lib/hello.txt<br /><br /> name: myconfig<br /><br /> subPath: hello.txt<br /><br /> oc rshinto the pod andcd /var/lib/and notice how hello.txt can't be read:
ls -lah ls: cannot access hello.txt: Permission denied
Current Result
ls: cannot access hello.txt: Permission denied
Expected Result
file contents should be available from inside the container.
alikhajeh1
All 22 comments
Same issue here. Working on figuring out if it's selinux or standard permissions.
shraderdm
on 20 Oct 2017
Same issue here. This is likely an SELinux issue or something similar:
sh-4.2# ls /etc/samba
lmhosts smb.conf smb.conf.example
sh-4.2# cat /etc/samba/smb.conf
cat: /etc/samba/smb.conf: Permission denied
sh-4.2# cd /etc/samba
sh-4.2# ls -al
ls: cannot access smb.conf: Permission denied
total 20
drwxr-xr-x. 2 root root 61 Oct 22 16:51 .
drwxr-xr-x. 49 root root 4096 Oct 22 17:04 ..
-rw-r--r--. 1 root root 20 Sep 21 10:30 lmhosts
-?????????? ? ? ? ? ? smb.conf
-rw-r--r--. 1 root root 11327 Sep 21 10:30 smb.conf.example
sh-4.2# ls -lZ /etc/samba
ls: cannot access /etc/samba/smb.conf: Permission denied
-rw-r--r--. root root system_u:object_r:svirt_sandbox_file_t:s0:c4,c7 lmhosts
?--------- ? ? smb.conf
-rw-r--r--. root root system_u:object_r:svirt_sandbox_file_t:s0:c4,c7 smb.conf.example
thoraxe
on 22 Oct 2017
This is an SELinux issue, You need the content of whatever is mounted at /var/lib to be relabeled to be accessible by SELinux. In docker terms you would mount with :Z.
rhatdan
on 23 Oct 2017
@rhatdan is there a workaround for now? Aside from mounting all files in the directory together as a configmap (without any subPath entry), I haven't found a way to make this work.
Also, why does this work without subPath? Is there something special about how the subpath is handled?
shraderdm
on 23 Oct 2017
Does kube provide the ability to set an SELinux context with a subpath mount? Perhaps this is an upstream kube bug?
thoraxe
on 23 Oct 2017
I was unable to reproduce on latest master. Has anyone reproduced on master (3.7)?
$ cat configmap.yaml
apiVersion: v1
data:
hello.txt: This is a test file
kind: ConfigMap
metadata:
name: myconfig
$ oc create -f configmap.yaml
configmap "myconfig" created
$ cat fedora.yaml
apiVersion: v1
kind: Pod
metadata:
name: fedora
spec:
containers:
- name: fedora
image: fedora
resources:
requests:
cpu: 1
memory: 512Mi
command:
- sleep
- "3600"
volumeMounts:
- mountPath: /var/lib/hello.txt
name: myconfig
subPath: hello.txt
terminationGracePeriodSeconds: 0
restartPolicy: Never
volumes:
- name: myconfig
configMap:
name: myconfig
$ oc create -f fedora.yaml
pod "fedora" created
$ oc rsh fedora
sh-4.4# cd /var/lib
sh-4.4# ls -alZ hello.txt
-rw-r--r--. 1 root root system_u:object_r:svirt_sandbox_file_t:s0:c5,c10 19 Oct 23 16:18 hello.txt
sh-4.4# cat hello.txt
This is a test file
sjenning
on 23 Oct 2017
@sjenning you aren't seeing the issue bc you are running the container as root. Can you please try again while using the restricted SCC or specifying a user in your dockerfile? You'll see the issue pop up after that.
shraderdm
on 23 Oct 2017
I can reproduce in 3.6 though
openshift v3.6.0+008f2d5-64
kubernetes v1.6.1+5115d708d7
$ oc rsh fedora
sh-4.4# cd /var/lib
sh-4.4# ls -alZ hello.txt
ls: cannot access 'hello.txt': Permission denied
Well... and if I try to reproduce on an all-in-one in the same way as 3.6, I can reproduce on 3.7 as well
openshift v3.7.0-alpha.1+2ca711c-1271
kubernetes v1.7.6+a08f5eeb62
$ oc rsh fedora
sh-4.4# cd /var/lib
sh-4.4# ls -alZ hello.txt
ls: cannot access 'hello.txt': Permission denied
Ok, there is a bug here. When using the subPath, the relabel only happens on the file symlnk on the host not to underlying file.
On the host looking at the configmap volume:
# ls -alZ
total 12
drwxrwxrwx. 3 root root unconfined_u:object_r:user_home_t:s0 4096 Oct 23 13:05 .
drwxr-xr-x. 3 root root unconfined_u:object_r:user_home_t:s0 4096 Oct 23 13:05 ..
drwxr-xr-x. 2 root root unconfined_u:object_r:user_home_t:s0 4096 Oct 23 13:05 ..109810_23_10_13_05_21.314932975
lrwxrwxrwx. 1 root root unconfined_u:object_r:user_home_t:s0 33 Oct 23 13:05 ..data -> ..109810_23_10_13_05_21.314932975
lrwxrwxrwx. 1 root root system_u:object_r:container_file_t:s0:c0,c1 16 Oct 23 13:05 hello.txt -> ..data/hello.txt
# ls -alZ ..data/
total 12
drwxr-xr-x. 2 root root unconfined_u:object_r:user_home_t:s0 4096 Oct 23 13:05 .
drwxrwxrwx. 3 root root unconfined_u:object_r:user_home_t:s0 4096 Oct 23 13:05 ..
-rw-r--r--. 1 root root unconfined_u:object_r:user_home_t:s0 19 Oct 23 13:05 hello.txt
The hello.txt symlink is relabel correctly but the underlying file in ..data is not.
sjenning
on 23 Oct 2017
On second thought, I think this is more of documentation bug in that subPath is simply not supported on Secrets or ConfigMaps.
Related:
https://github.com/kubernetes/kubernetes/issues/50345
sjenning
on 23 Oct 2017
@sjenning Why would you say it's not supported? The issue you refer to deals with updates to content mounted via subPath, not the lack of support for subPath in general.
This should be supported, but it looks like, as you suggested, there is an selinux/symlink/etc. bug with respect to the context of the injected content.
thoraxe
on 24 Oct 2017
Ultimately I think this is an upstream Kube issue, so I'll open one there and reference this one.
thoraxe
on 24 Oct 2017
@thoraxe ok, you are right in that it is selinux + secrets/configmaps that is not supported. This is because we defer to the runtime to do the labeling and we only pass the path (hostPath + subPath) to the runtime. The runtime is unaware that what is it labeling is a symlink to a file in a sibling directory, namely ..data.
sjenning
on 24 Oct 2017
@sjenning ok, now we're on the same page.
Either way, I think this is an upstream issue, referenced above. Not sure how we "fix" it, though, other than volume mounting the whole folder.
thoraxe
on 25 Oct 2017
@sabre1041 can you please share the subPath: ..data/ workaround for this that you sent me via email? I don't want to take credit for your fix.
shraderdm
on 30 Oct 2017
@sabre1041 can you share the workaround that @shraderdm is referring to?
This issue is still open and still open upstream as well.
thoraxe
on 23 Jan 2018
This behavior seems to depend on the docker version on centos7.
We're having the same issue with docker-1.12.6-55 but using subPath with a configMap works fine with docker-1.12.6-61 and docker-1.12.6-68 (without the ..data/ workaround): the more recent docker versions mount the target of the symlink (cf. https://github.com/kubernetes/kubernetes/issues/50345). The file is visible as expected in the container, but is never updated if the contents of the configMap are changed (a container restart is required to get the updated data).
alexcern
on 31 Jan 2018
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
openshift-bot
on 1 May 2018
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle rotten
/remove-lifecycle stale
openshift-bot
on 31 May 2018
Rotten issues close after 30d of inactivity.
Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.
/close
openshift-bot
on 30 Jun 2018
Sorry for re-opening this one but has the answer I was looking for is not present here, I thought I should share it.
To fix the access denied on a file mounted with volumeMount and subPath,
it seems to work if we provide default_mode property on the volume and give an access containing read permission.
volume {
name = "<some-name>"
config_map {
name = "<some-file>"
default_mode = "0555"
}
}
volume_mount {
name = "<some_name>"
mount_path = "<file_path>"
read_only = true
sub_path = "<some-file>"
}
JnMik
on 26 Jun 2019
Related issues
thescouser89
路
3Comments
smugcloud
路
5Comments
enj
路
4Comments
surajssd
路
4Comments
akram
路
3Comments