Compose: Support --user option in "docker-compose up"

[mrzechonek]

I need to pass --user option to run orchestrated containers under my own UID. This is mostly because of mounted host volumes, and I'd like the dockerized app to generate files owned by me, not the root.

With docker-compose up, this is not possible, at least not directly. Right now I'm using a crazy workaround:

NAME=`compose run -d --user="$UID" someservicename`
docker rename $NAME ${NAME/_run/}

which is suboptiomal, to be gentle.

[dnephin]

I'm not sure I understand, docker-compose run --user is an option, and the docker-compose.yml supports the user key (http://docs.docker.com/compose/yml/#working95dir-entrypoint-user-hostname-domainname-mem95limit-privileged-restart-stdin95open-tty-cpu95shares).

I guess we'd need to support environment variable resolution in the user field so that you can set it to $UID is that right? #1377

[mrzechonek]

Hi. Yeah, $UID expansion would do the trick in this case. Still, --user option for docker-compose up command (not only for docker-compose run) would be useful to start the whole project as particular user.

The point is to allow specifying the user without touching yml.

[aanand]

start the whole project as particular user

It never occurred to me that people might want to do this. Could you elaborate on the use case?

[aanand]

Never mind, I scrolled up. Am I right in thinking you're on Linux? When using boot2docker it creates files with sensible permissions.

I wonder if there's a way to configure the Docker daemon to do this for every volume it mounts.

[mrzechonek]

Yes, on Linux.

I'm using compose to run tests on CI by mounting host volume (yeah, I know). Without this option, some files get created with "wrong" ownership, i. e. root:root

[mrzechonek]

I know I could use a data only container and inspect/copy files from it, but a host volume is much more convenient and requires less scripting.

[josephpage]

up --user could be dangerous if some containers/services already specify a user.

@mrzechonek I don't think that modifying a container because of its environment is a good practice.

[mrzechonek]

@josephpage yeah, it might be. $UID expansion would do the trick though.

Pretty much the only thing I need is proper permissions on host volumes. I don't know of any other way to achieve that, besides running container process as $UID...

I would actually prefer this feature to be present in the Damon itself, like @aanand said. Or maybe an option to mount a volume in such a mode (a storage driver maybe?).

[wildsurfer]

@mrzechonek, have you found a workaround?

[mrzechonek]

Not really, no. Right now we're doing docker-compose run --user and then rename/relabel the container to make compose think it was started by up command, so that stop and ps would work.

[dnephin]

1377 is in master and will be in the 1.5.0 release, so you'll be able to do user: $UID in docker-compose.yml. If you're comfortable using master you can try it out now, otherwise an RC release should be along in the next few weeks.

I'm going to close this issue since the feature itself is implemented as part of #1377

[dnephin]

and then rename/relabel the container to make compose think it was started by up command

This is actually not the first time I've heard of the need for this (although in this case I guess it's just temporary). I have a proposed fix for it in #2042

[mrzechonek]

Thank you, #1377 fixes this nicely.

[michaelmior]

@mrzechonek Could you clarify how this solved your problem? I have the exact same use case: "I'd like the dockerized app to generate files owned by me, not the root"

I tried adding user: $UID to the web container and when I use docker-compose run web touch foo I get the following:

WARNING: The UID variable is not set. Defaulting to a blank string.

The file foo is created, but it is still owned by root.

[mrzechonek]

I've used user: $USER, but $UID works as well. I don't know why your setup complains about missing variables :(

[maxfrigge]

Having the exact same problem as @michaelmior.

When using $USER instead I get System error: Unable to find user Max. Which makes sense because it's a host user.

What could cause $UID not being available during docker-compose execution?

[rdavaillaud]

Hi, same problem here.
I'm on Fedora 23, and the UID variable is not propagated when I call env command on the host.

workaround:
do an export UID on the host first, then call your docker-compose

[atrauzzi]

Would be nice if docker-compose just made $UID available without it having to be exported. Ends up being boilerplate.

[dnephin]

If UID is not exported there is no way for compose to get it, so I think what you're asking is impossible.

[atrauzzi]

So compose simply as a running application has no way of inferring the user it's running as?

[dnephin]

Sure, it could read the $USER environment variable, but you can do that too from the Compose file! If the variable isn't exported, it's not going to be available to child processes. The really easy solution seems to be just export it.

[atrauzzi]

I'm more thinking that if UID isn't declared, then the executable could look at the uid of who ran it and make it available as such.

Again, thinking about boilerplate and easy to overlook steps. Most people want a setup where the only step is docker-compose up. Why add one tiny variable requirement that 99% of people are going to all set in exactly the same way.

[gkop]

Is there a nice resource that spells out any other similar gotchas between Mac and Linux hosts?

[gkop]

Also, forcing the user isn't really what we want on Linux, we want the behavior that Mac has, where even when you run as root in the container, new files on host volumes have useful permissions :(

[gkop]

I was actually just confused about why my Mac behavior was better than my Linux behavior, and it's not strictly anything to do with this issue. I started an issue on Dinghy to search for solutions for a nice experience on Linux.

[dmitrym0]

@mrzechonek can you share how you use user: directive to ensure the file permissions are correct on host and container?

[mrzechonek]

I simply add user: $UID into the .yml file. That's on Linux, Gentoo and Ubuntu 12.04.

[dmitrym0]

@mrzechonek thanks. Does your container then do some run time magic? As far as I understand, user: mirrors the Dockerfile instruction. But that simply means that the commands inside the container run as the user: $UID. I can't grasp how that helps with volume permissions =/.

[atrauzzi]

@mrzechonek: This is different. We want a feature to control the user the container acts as, outside on the host system, regardless of who it is set up as inside the container itself.

Think: _"root in container writes as myuser user on host"_.

[mrzechonek]

In our use case, it's pretty much always the same user building the container and running it, so there is no issue.

If, however, you'd like to map container users to host users "dynamically", I think the only way to go about that is to support LXC user namespaces in docker daemon itself, feature that's not implemented (yet? as far as I know?).

I don't think that's something that docker-compose could do.

[mrzechonek]

Seems like I missed a few releases: https://integratedcode.us/2015/10/13/user-namespaces-have-arrived-in-docker/

Sorry, I don't use Docker anymore, at least not actively in the current project.

[dmitrym0]

Thanks @mrzechonek. Ultimately, I'm trying to do what @gkop is trying to do -- have the files generated by the container on the host-mounted volumes owned by the container starter. That's how it works on OS X with the latest Docker 1.12 beta (how?) and that's how I'd like to see it on linux as well.

[atrauzzi]

@mrzechonek - Some applications require that they be run as a particular user, or require that the user they're running as map to a real one in the system. In those cases, it's easier to just leave it as root and do the mapping over the container's head.

(this is in lieu of lots of ugly hacks to map the current running system's user scheme onto the container just to get everything lining up)

[mrzechonek]

@dmitrym0 I think that under OSX, you run boot2docker inside a native OSX virtualization (https://github.com/mist64/xhyve/) and then containers are created inside that VM. This means that it's really the VM doing all the user mapping, not docker daemon. Container's root is still host's root.

[luispabon]

$UID is not set by default on Ubuntu 16.04 hosts. It would be trivial for docker-compose to acquire the user id running it and injecting it nonetheless. A lot less boilerplate code and environment set up for app dev compose users, which makes sense since compose is all about docker UX.

[IMM0rtalis]

+1 for docker-compose up --user or executing user ....

[StalkAlex]

Is there any solutions or news on this problem?

[atrauzzi]

Nope, and I opened this feature against the docker engine quite some while ago: https://github.com/docker/docker/issues/22415

I think this is a much higher impact issue than it's being recognized for currently. Being able to alter what user the container touches the filesystem as without having to make the container itself aware of the permissions systems would open up quite a few doors.

If this is something you're interested in, I suggest sharing and upvoting the ticket I linked.

[jovanialferez]

this works for me: user: "1000:1000"

[nfm]

@jovanialferez FYI if you hardcode it like that, you'll run into trouble if other people are involved in the project that don't have their UID and GID set to 1000. I think OSX starts numbering regular users at 500, and any Linux install with multiple users would end up with a UID greater than 1000.

[luispabon]

@jovanialferez that will only work on Linux.

[jovanialferez]

noted. thanks @nfm @luispabon

[bluehallu]

I recently moved from Mac to Linux and just got hit by this, don't really understand how this is not solved yet

[atrauzzi]

Mentioning this issue again so that newcomers see it: https://github.com/moby/moby/issues/22415

We really need the ability to externally map docker processes to a specific user. Emphasis on externally. Not choose the UID/GID of the process in the container. But map the container process to a local UID/GID from the outside.

[luispabon]

Yes, indeed. Assigning the uid/gid of the current user into the container only works in Linux. Seems like a docker problem though.

[imanabu]

We are running into a bit more esoteric issue on Windows. We use an OrientDB local instance, which writes the files to the host's file system, and on Windows it does not seem to do it. Oh... may be it is because the host volume is not block volime?

[OnkelTem]

Almost one year ago when I was trying to resolve this I found this topic. Since no any solution was provided at that time, I created a bunch of wrapping bash scripts to run your docker-compose.

Now after a year it's all the same. I wonder at how much times it takes to resolve such a simple issue for a software like docker-compose which is not even a real provider but instead a wrapper.

$UID is not exported on Linux by default, ok? This prevents us from creating a universal docker-compose - which is a strange thing by itself as this piece of software is assumed to be a front-end solution, right?

If you guys don't like to waste time on trifles - and I understand that - then why don't you just let us to execute a random host command prior to running a service, eh? We could just do something like this:

services:
  web:
     ...
     host_command: export UID

Isn't that obvious?

[38leinaD]

+1

[darkguy2008]

This issue does not have enough likes and +1s and to me it's an overlooked yet needed and requested by a lot of users including myself.

I'm here to give my +1 as well because I'm hit by this. I currently don't want my containers to run as root -yet- I want them to share a volume in the host which is writeable by my user. This can't be done if I don't set the user and expansion would be the most natural way to do this instead of user: 1000:1000 because, as stated before, it will cause conflicts with users with another environment. $UID, even if it's not exported, proves to be a better solution as it's environment-based than docker-expose-based.

Just my 2 cents. So, anybody found a way to do this anyways?

[atrauzzi]

@darkguy2008 - Be sure to hype this one up: https://github.com/moby/moby/issues/22415

I suspect a feature will be needed within docker itself before compose could build on top of it.

[nan140114]

I'm not sure I understand, docker-compose run --user is an option, and the docker-compose.yml supports the user key (http://docs.docker.com/compose/yml/#working95dir-entrypoint-user-hostname-domainname-mem95limit-privileged-restart-stdin95open-tty-cpu95shares).

I guess we'd need to support environment variable resolution in the user field so that you can set it to $UID is that right? #1377

The link is broken.

[pandemicGander]

Found that adding a mandatory variable helped as a workaround:

version: "3"
services:
  testapp:
    image: ubuntu:20.04
    entrypoint: /bin/bash -c "cd $PWD && touch tmp"
    user: ${CURRENT_UID:?"Please run as follows 'CURRENT_UID=$(id -u):$(id -g) docker-compose up'"}
    volumes:
      - $PWD:$PWD

Would show:

ERROR: Missing mandatory value for "user" option in service "testapp": "Please run as follows 'CURRENT_UID=$(id -u):$(id -g) docker-compose up'"

Specifying that the file has to be executed like this:

CURRENT_UID=$(id -u):$(id -g) docker-compose up

[Luc45]

Found that adding a mandatory variable helped as a workaround:

version: "3"
services:
  testapp:
    image: ubuntu:20.04
    entrypoint: /bin/bash -c "cd $PWD && touch tmp"
    user: ${CURRENT_UID:?"Please run as follows 'CURRENT_UID=$(id -u):$(id -g) docker-compose up'"}
    volumes:
      - $PWD:$PWD

Would show:

ERROR: Missing mandatory value for "user" option in service "testapp": "Please run as follows 'CURRENT_UID=$(id -u):$(id -g) docker-compose up'"

Specifying that the file has to be executed like this:

CURRENT_UID=$(id -u):$(id -g) docker-compose up

The problem is that my services require root access to start. Eg:

app-php-fpm  | [14-Jun-2020 00:15:12] NOTICE: [pool www] 'user' directive is ignored when FPM is not running as root
app-php-fpm  | [14-Jun-2020 00:15:12] NOTICE: [pool www] 'group' directive is ignored when FPM is not running as root
app-redis    | 1:M 14 Jun 2020 00:15:12.710 * Ready to accept connections
app-php-fpm  | [14-Jun-2020 00:15:12] ERROR: Unable to create the PID file (/run/php-fpm.pid).: Permission denied (13)
app-php-fpm  | [14-Jun-2020 00:15:12] ERROR: FPM initialization failed
app-webserver exited with code 1
app-mysql exited with code 1
app-php-fpm exited with code 78