Moby: [feature] Allow mounting sub-directories of named volumes

Created on 13 Apr 2017  ยท  69Comments  ยท  Source: moby/moby

Similar to how bind mounts let you mount subfolders i.e -v /host/path:/container/path, is it possible for the same functionality to be available for named volumes? i.e -v namedvolume/path:/container/path. Right now in order to mount a named volume, you must mount the entire volume.

I apologize in advance if this has been discussed somewhere already, but I couldn't find anything saying this isn't doable.

arevolumes kinfeature

Most helpful comment

This is very important to be addressed. Would be great if this is taken up. Currently if you are mounting nginx as a volume, then we have to create 2 named volumes for nginx to mount /etc/nginx/ and /var/log/nginx/ separately in 2 different volumes which can be solved if this is addressed.

All 69 comments

This would be super useful for me as well ๐Ÿ‘ Sorry for my shallow knowledge but, what would be the difficulties in this? Making sure there's no "chroot escape -a-like"?

Even I am looking for this similar feature, but couldnt find any useful links explaining how to achieve this. Kubernetes has this support via sub-path notation.

Having ability to specify sub-directory from a named volume would be really helpful.
Some containers require mounting multiple volumes, while these multiple volumes logically fit well to a sub-directories in a single named volume.

E.g. https://hub.docker.com/r/tvial/docker-mailserver/ container requires 3 volumes by default:

my-mail-server:
     volumes:
     - maildata:/var/mail
     - mailstate:/var/mail-state
     - ./config/:/tmp/docker-mailserver/

Ideally, I would like to create one named volume, e.g. using RexRay volume driver named rex_mail and then mount sub-directories of this volume in the service. One of the options could be mounting at the service level, like this:

my-mail-server:
     volumes:
     - rex_mail/maildata:/var/mail
     - rex_mail/mailstate:/var/mail-state
     - rex_mail/config:/tmp/docker-mailserver/

Or maybe as an alternative way (e.g. if it will be easier from the point of view of backwards compatibility), to handle this at the top level for volumes, creating mappings between named volume sub-directories and new volumes to use at the service level. E.g. something like this:

volumes:
     rex_maildata:
          external:
              name: rex_mail
             src: /maildata
     rex_mailstate:
         external:
             name: rex_mail
             src: /mailstate
     rex_mailconfig:
         external:
             name: rex_mail
             src: /config

And then at the service level to use these sub-directories as:

my-mail-server:
     volumes:
     - rex_maildata:/var/mail
     - rex_mailstate:/var/mail-state
     - rex_mailconfig:/tmp/docker-mailserver/

There are multiple reasons why the ability to mount name volume only once and map various sub-directories in it to various directories in the container is important:

  1. Typically volumes at hosting providers (E.g. AWS, Digital Ocean) have a minimum size of about 1Gb, which is an overkill for many directories that need to be mapped.
  2. Storing all data needed for a container in one volume makes it easier to use it, backup, restore, etc.
  3. Most hosting providers impose restrictions on the number of volumes you can mount on a host. E.g. at Digital Ocean you can mount up to 5 volumes to a droplet. Creating a separate named volume for each folder a container needs can very easily make it not possible to run several containers on the same host. This is especially important when running in Docker Swarm mode, since it's hard to tell which container will start on which host (and how many volumes should be mounted on the host).

This could be added to the new mount syntax. Not too keen on supporting this with the old -v foo:/bar syntax... not sure it would even work with it.

Basically add a new field here called Subpath that could be taken into account.

There are likely some implications in supporting this that will limit future possibilities that which need to be explored...

A ๐Ÿ‘ from me also - posted a duplicate ticket here: https://github.com/docker/compose/issues/5367#issuecomment-344047495.

The ability to create a volume and build complex subfolder / file mappings (maybe supporting conditions) would be amazing.

@cpuguy83 taking your suggestion, I made a sample implementation of this feature. I checked that it works with a local copy of the cli. It works similar to how @spatialvlad described it.

Take a look at the code changes here and let me know if this design is acceptable.

If anyone would like to try it out with the cli, you can do so with this cli and running this sample command:
docker run -it --name test --mount source="testvol",target=/app,volume-subpath=/test alpine

@y2kpr I like the design. nice work.

@y2kpr Thanks! Looks good, but needs tests.
Also need to check for symlink traversal issues.

Thanks for the feedback! I will try to submit a PR with checks for symlink traversal and tests by this weekend.

Design looks good (from the description above); we should indeed have tests for various scenarios (single container using multiple paths from the same volume, multiple containers using the same, or different paths from a volume etc., possibly "corner cases" where paths overlap inside the container)

@y2kpr have you been able to continue on the PR?

@mathiasdm I just have some testing left to do. I was on vacation for a while and I get back tomorrow. I am planning to finish it by this weekend

Over the weekend I tried to write some unit tests but had no success. Being very new to the code base and without many reference unit tests around the code I added, I am having trouble mocking things correctly (especially in the daemon).

I would like someone who is more familiar with the code base to take it from here or guide me on how to do the unit tests (@cpuguy83 know anyone?). Unfortunately, I do not have enough time to tinker with it and figure it out on my own.

Once the unit tests are done, the integration tests (if we want some) are straightforward. I just need to add some CLI commands after updating the docker CLI.

@y2kpr I guess it would depend on what exactly you are going to unit test and what your implementation looks like.
We should definitely have an integration test, but there should be no need to update the CLI in moby/moby. We only currently accept integration API tests (see integration/ dir), not CLI.

Any updates on this? This seems like a really useful feature =)

Also looking forward to this feature!

@y2kpr were you still working on this?

@thaJeztah Unfortunately, I will not be able to work on it at least until summer (mid may)

This feature would be incredibly useful for the kinds of transparent/reproducible computational science workflows that we're supporting at Gigantum. None of us are Go experts (and certainly not Docker codebase experts), but we would be interested in knowing how we could best support getting this feature landed.

I'd be happy to take a crack at writing some API integration tests, or it might be more productive to put a bounty on finishing this up. It seems like this is the closest thing to a starting point, and I only see surface testing there - not any inspection to see if the data contained in volumes is correct.

I welcome any feedback, in particular from @y2kpr as this is your patch! In the meantime, I'll see about cloning your clone, and applying the changes recommended by @cpuguy83.

popping in to add my support for this feature.

at the moment, it seems like the mickey mouse verbose solution is to define each of the subdirectories as unique volumes. would be AWESOME to just define the parent and use namedvolume/subdir for the various service volumes.

love you long time.

This feature would make this much easier.

I totally support this feature, can't wait!

@y2kpr and @davclark, I see statements above from both of you about getting back to this PR. Any ETA? I'm stirring the pot, because this feature would help with what my team is trying to accomplish. If there are no immediate plans to bring this feature to completion, I'm interested in throwing my time at the problem starting next week, if my assistance would be welcome.

Sorry to have just gone silent... Currently, I'm evaluating dotmesh as an alternative to using stock docker volumes. So this is less relevant to us. Given the lack of input and my lack of knowledge about Docker, I was reluctant to put too much time into this. Please keep this thread updated with any activity. I'm happy to review or help develop tests, just want to make sure it's a reasonable use of time!

...do we need to start a gofundme to bribe incentivize some hero lurking in the shadows who successfully implements this? because i'll fuckin' do it...

Hi just here to add a useless +1... and I just want to point out that this would be a great way to use NFS volumes. Right now I'm building out a new core server for our infastructure and plan on storing all data folders for apps in production on our NFS server. It would be awesome if I could just designate some sort of "Server Data" volume, and then use a much more simplified syntax to mount each container's data...

nfs-vol/service1:/data

Instead of having to essentially define a new NFS mount for every single data directory

+1

+1. This would be a really useful feature for so many reasons.

Question: what would be the expected behavior for this feature if the volume's name and corresponding subdirectory matches an equivalent relative path wrt to the PWD or Docker-compose file? Would the volume take president or the local directory?

E.g:

version: '3'

services:
  example:
    image: busybox
    volumes:
      - foo/bar:/foo/bar
    working_dir: /foo/bar
    command: ls
$ tree ./
.
โ”œโ”€โ”€ docker-compose.yml
โ””โ”€โ”€ foo
    โ””โ”€โ”€ bar
        โ””โ”€โ”€ spam
$ docker volume create foo
$ docker run -it --rm -v foo:/foo busybox mkdir /foo/bar
$ docker-compose up

I can see two routes that might work. First if you define a path the system could look for existing volumes of the same name and if the volume exists then it could be used, if the volume does not exist then a local directory could be assumed. This method may be easier to implement but could lead to name collisions or confusion by users.

The second option would be some declarative way to differentiate between a directory and a volume. Maybe a new yaml tag. I assume this is more difficult to implement.

We could require local relative paths in yaml format and docker run args to use the ./ prefix to be explicit about being a path vs volume name. E.g:

version: '3'

services:
  example:
    image: busybox
    volumes:
-      - foo/bar:/foo/bar
+      - ./foo/bar:/foo/bar
    working_dir: /foo/bar
    command: ls

Forgive the potentially obvious question, as I'm new to Docker in the grand sceme of things and don't set up many build pipelines... but I wasn't aware that you could specify a relative bind path in a compose file at all. Are you sure that's possible? If so, it actually seems like a pretty bad practice, as docker-compose files seem engineered to be run from anywhere and produce the same service config. But again, I'm not super experienced in this area. As far as I was aware, the differentiation between a bind mount and a volume is as simple as starting with a / or not.

So with that in mind... I'd personally say that the volume sub-path should have precedence, if this is even an issue.

Edit: @ruffsl 's solution with the ./ also seems totally logical.

I wasn't aware that you could specify a relative bind path in a compose file at all. Are you sure that's possible? If so, it actually seems like a pretty bad practice, as docker-compose files seem engineered to be run from anywhere and produce the same service config.

Personally, I use relative bind paths in a compose file frequently, particularly when making a complex orchestration setups using source code and debug environments more reproducible for fellow devs, and I try to keep my compose files in close proximity to the dockerfiles and build/volume contexts they depend on. This helps me avoid, say, colliding everyone else's workstation /tmp directories or hardcoding absolute paths in the compose files themself. I can switch git branches in the repo the compose file is located and know the next time I down/up compose, the containers from compose will be using the appropriate mounted source or config data.

So with that in mind... I'd personally say that the volume sub-path should have precedence, if this is even an issue.

As a soft migration, we could allow volume sub-path to take precedence if the volume name (or subdir in volume?) exists, and allow for ./ to override this precedence, and encourage the best practice to be explicit with ./ when intentionally using relative bind paths.

Question: what would be the expected behavior for this feature if the volume's name and corresponding subdirectory matches an equivalent relative path wrt to the PWD or Docker-compose file? Would the volume take president or the local directory?

We should not add this feature to the "shorthand" syntax (-v <volume|path>:<container-path>), which is already overloaded, and only add it to the long/advanced (--mount) syntax.

For docker run / docker service create on the command line, and add an extra option to that syntax (e.g. source_path, subdir, src_dir, path, or dir - naming is hard ๐Ÿ˜… ) it could then look something like this;

docker run -d \
  --mount type=volume,source=myvolume,source_path=/foo/bar,target=/path/in/container \
  myimage:latest

And in the docker-compose file;

version: "3.9"
services:
  web:
    image: nginx:alpine
    volumes:
      - type: volume
        source: myvolume
        source_path: /foo/bar
        target: /path/in/container

One thing that should be taken into account in the design is; what to do if the specified path does not exist in the volume?

Automatically creating the path is troublesome, and led to a lot of problems in the past with the -v syntax when trying to refer to a _file_; for example:

--mount type=volume,source=myvolume,source_path=/etc/httpd.conf,target=/usr/local/apache2/conf/httpd.conf

If auto-creation is used, volume myvolume is empty (which can be on first use), docker will;

  1. create source_path inside the volume, but won't know if httpd.conf should be a file or a directory, so it will default to a _dir_ (creating directory /etc/httpd.conf/
  2. copy the content of /usr/local/apache2/conf/httpd.conf into the volume
  3. create the target-path (/usr/local/apache2/conf/httpd.conf) inside the container (if it doesn't exist)
  4. mount the volume's subdirectory at the target-path (which fails if the target-path is a file, but the source is a directory)

Requiring the source-path to exist would be an option, but then the question is; how to "initialise" the volume for the first time?

Requiring the source-path to exist would be an option, but then the question is; how to "initialise" the volume for the first time?

For me, it is more than acceptable to just have the limitation that subfolders on the specified volume should exist beforehand. Starting a container would simply fail when the path is not accessible...

Requiring the source-path to exist would be an option, but then the question is; how to "initialise" the volume for the first time?

For me, it is more than acceptable to just have the limitation that subfolders on the specified volume should exist beforehand. Starting a container would simply fail when the path is not accessible...

Or, iterate over the directories declared and create them recursively, just like it works outside named volumes ยฏ\_(ใƒ„)_/ยฏ

It's really not that hard.

+1

Hi!
I know, is closed, but I try to share here...
I test this with Docker Desktop (Docker version 18.09.2, build 6247962) and it works:

...
volumes:
  vol_base:
    driver: local
    name: vol_base
  vol_data:
    driver: local
    name: vol_base
    external: true
  vol_logs:
    driver: local
    name: vol_base
    external: true
...

I hope this helps!

Or, iterate over the directories declared and create them recursively, just like it works outside named volumes ยฏ_(ใƒ„)_/ยฏ

It's really not that hard.

I don't think that would be the right approach; see my comment before that;

create source_path inside the volume, but won't know if httpd.conf should be a file or a directory, so it will default to a _dir_ (creating directory /etc/httpd.conf/

I know, is closed, but I try to share here...

@dvillacesopra the issue is not closed ๐Ÿ˜…. I don't think your example is matching what's being requested here; your example shows that you can use the same volume (vol_base), and reference it under different names inside a compose file (vol_base, vol_data, vol_logs), however, each container/service will still mount the _whole_ volume (vol_base/). The request here is to allow mounting only a subpath of the volume (e.g. service1 uses vol_base/some/subdir, service2 uses vol_base/some/other/subdir, service3 vol_base/some/config.file)

Or, iterate over the directories declared and create them recursively, just like it works outside named volumes ยฏ_(ใƒ„)_/ยฏ
It's really not that hard.

I don't think that would be the right approach; see my comment before that;

create source_path inside the volume, but won't know if httpd.conf should be a file or a directory, so it will default to a _dir_ (creating directory /etc/httpd.conf/

I think metaa's point is just that this is already how a bind mount behaves. AFAIK, creating a bind mount to etc/httpd.conf without a preexisting file in place will, in fact, create a folder called httpd.conf/.

I understand YOUR point too though (I think), which is that the need for files to pre-exist inside a volume creates usage issues. It's more difficult to pre-populate a volume with needed files, than it is a host filesystem destined for a bind mount.

I think that keeping this feature simple and embracing the limitations that would come along with following the existing behavior is the best way. Basically, this feature seems like it'd be best used with volumes_from or something like that (if I'm understanding our logic here), because there would almost always be a need for another container to populate the volume first.

This, of course, doesn't apply to NFS volumes or other network volumes, which I personally would be using this feature for.

I think metaa's point is just that this is already how a bind mount behaves. AFAIK, creating a bind mount to etc/httpd.conf without a preexisting file in place will, in fact, create a folder called httpd.conf/.

That was indeed the case for the shorthand -v /some/path:/container/path syntax (and the cause of a _lot_ of issues. The --mount syntax will refuse to start the container if the src does not exist.

@dvillacesopra the issue is not closed ๐Ÿ˜….

I saw the closed status tag and I was wrong interpreting it, sorry

I don't think your example is matching what's being requested here; your example shows that you can use the same volume (vol_base), and reference it under different names inside a compose file (vol_base, vol_data, vol_logs), however, each container/service will still mount the _whole_ volume (vol_base/). The request here is to allow mounting only a subpath of the volume (e.g. service1 uses vol_base/some/subdir, service2 uses vol_base/some/other/subdir, service3 vol_base/some/config.file)

Ok, but... if you mount the volume (yes, with all the subdirectories) then, you can access to the desired one, unless it is strictly necessary not to access the rest of the directories and I recognize that the use case of the file is not resolved, but I think these particular details could be resolved from a configuration perspective.
I repeat, all this approach assumes it is not strictly necessary disallow the acces to the other subpaths.

Yeah, the use cases being described require the subfolder to be mounted specifically, without the additional folders next to it.

If I want to use this feature to mount an httpd.conf file into an nginx container but the file resides in a volume at the path myvolume/httpd.conf, I shouldn't have to mount the entirety of myvolume to /etc to accomplish this task.

Sorry to sound like a broken record but I just want to keep mentioning that an NFS volume (which would already have existing paths and files) is a really logical use-case for this feature. In fact accomplishing the same thing right now is very bulky considering the need to set up a ton of NFS volumes to essentially the same location.

I think metaa's point is just that this is already how a bind mount behaves. AFAIK, creating a bind mount to etc/httpd.conf without a preexisting file in place will, in fact, create a folder called httpd.conf/.

That was indeed the case for the shorthand -v /some/path:/container/path syntax (and the cause of a _lot_ of issues. The --mount syntax will refuse to start the container if the src does not exist.

Ah, I see. Still, I could see relying on existing behavior at least providing consistent results. So yeah, I would have to create and populate the volume with necessary subfolders and files before trying to mount any kind of subfolder or file within it. To me this seems reasonable.

Or the behavior from -v could be used. I understand the problems that can arise from assuming folders should be generated... but at this point, anybody using this subfolder feature is already dealing with a very specific use-case anyway. I don't think it needs to be user friendly enough to account for every single type of automatic file creation.

I need this too. I have a docker-compose.yml with 16 references to 16 sub-directories of ${BASE_PATH}.

  - ${BASE_PATH}/configs/service1:/config:rw

I'd like to move them to CIFS/SMB, but it would take 16x6 lines of yaml and probably 16 CIFS/SMB client connections.

This seems like an obvious requirement for docker swarm adoption.

Are there any news regarding this issue?

Whoa, this is weird (that it hasn't been addressed). This definitely needed in swarm mode.

up

This is very important to be addressed. Would be great if this is taken up. Currently if you are mounting nginx as a volume, then we have to create 2 named volumes for nginx to mount /etc/nginx/ and /var/log/nginx/ separately in 2 different volumes which can be solved if this is addressed.

up

Three years later, this issue still opening at 2020 ๐Ÿ˜ž

Would definitely be useful to have this, even if it's in the long handsystem and not in the shorthand!

Pihole docker for example requires two volumes (three if you want permanent logs):

 - pihole:/etc/pihole
  - dnsmasq:/etc/dnsmasq.d

This would help put this in a single volume ("pihole etc" for example)

Did a solution for this ever get implemented? I'm trying to create volumes from NFS shares with sub-folders and this seems like it would be the simplest way to do so.

We really need this feature to be able to develop faster on Mac. Mounting named volumes is much faster than creating bind mounts on macOS. It would be convenient to be able to reference needed directories inside named volumes in order not to have a lot of unnecessary data shared between the containers.

Almost 3 years and this still is in Open status? Seriously? The original post has 198 thumbs up on top of that! Get on this...

@myermian while a lot of us feel your frustration, I am not sure your approach is the most helpful. Most of us are incredibly thankful for the efforts constantly being done in this community-driven project. I'd bring it down a notch if I were you. I mean, unless you are like shadow-financing the whole thing, in which case, press on ahead ๐Ÿ˜ƒ .

@bkraul Could he/she have used more polite word? Sure! Is he/she being disrespectful? I feel (hope?) it's more on this side of incredulity rather than disrespect. I mean, he/she not terribly out of place for calling out on this issue. The entire Moby project has over 3500 open issues as of today. Within that set, this issue is

  • the 2nd oldest issue (2/3500)
  • the 8th most commented issue (8/3500)
  • the 7th most thumbs-up'd issue (7/3500)

To be honest, I landed here when investigating for a customer and no matter how you look, those are poor project management metrics.

Most of us are incredibly thankful for the efforts constantly being done in this community-driven project

That subtly suggests Moby could be a hobby project by solo developer building it in his/her personal time on weekends. The reality is Moby facilitates some significant commercial activities of Docker Inc. You can see the # of commits by current and former Docker employees and even docker compose send users back to this repo.

Now none of this is wrong. At all! But there is a relationship between the community parts and the commercial parts so the balance is (IMHO) somewhere in between "incredible thankfulness" of a hobby project and the "merciless accountability" of a commercial venture.

Anyway, @thaJeztah - any triage plans on this issue?

Nobody has opened a PR for this feature.
If people want it they need to open a PR.
They also need to show that it's secure (maybe want to look at kubelet's CVE related to volume subpath mounting).

If people want it they need to open a PR.

Why is it insufficient to request a feature? It's fair that if the community invests it's time into the ecosystem making the tools and business more valuable that folks from Docker Inc in turn invest time to develop the feature. That's the balance I was referring to. "Have your cake and eat it too"

I've tried for a couple hours at finishing up the branch started by y2kpr, but found myself struggling to write a decent testing suite (or figure out how to test it manually at all) in an unfamiliar repo. I'd be willing to devote some more time if I could get some guidance / pointers to examples from a more familiar dev.

I also have a suggestion: would the owners of this repo be ammenable to using some sort of bounty program such as issuehunt.io? I would definitely be willing to put up a bid for this feature, and participating in these programs is mutually beneficial to both OSS projects and the OSS dev community.

And there goes my hope for this to already have been implemented... :( Hope to see this getting more traction. It's really gonna help volume management in some cases (mine included).

Food for thoughts, another use-case.

Using the Traefik reverse-proxy (let's describe it that way), it manages all TLS certificates for my subdomains using Let's Encrypt, but stores them in a single JSON file ("proprietary" format).

Proxied services behind Traefik may need to access those certificates/private keys, but not _all_ of them, not in JSON. Hence conversion from a single JSON to a collection of standard PEM certs is required and handled by this service, which essentially outputs _multiple_ directories-one per subdomain.

It is unfortunately not possible to use a single named volume to populate with the multiple directories and _then_ pick a _single_ (or 2, or _some_โ€ฆ) of those directories to mount on each specific service.

It would be awesome though:

services:
  traefik:
    image: traefik:v2.2
    # ...
    volumes:
      - tls-certs-as-json:/letsencrypt # where Traefik stores it's managed certificates -- output

  tls-certs-converter:
    image: ldez/traefik-certs-dumper:latest
    # ...
    volumes:
      - tls-certs-as-json:/data/certs-raw # mounting Traefik's managed certicates (JSON) -- input
      - tls-certs-as-pem:/data/certs-converted # where conversion's output is stored (PEM) -- output

  service1:
    # ...
    volumes:
      - tls-certs-as-pem/some-sub.domain.tld/:/data/certs # TLS cert this service needs -- input

  service2:
    # ...
    volumes:
      - tls-certs-as-pem/another-sub.domain.tld/:/usr/local/share/ssl # TLS cert this service needs -- input

Unfortunately I cannot come up with a PR (not proficient enough with Go etc.) but hopefully that's a meaningful contribution to this issue :pray:

In the meantime, I use this workaround to mount the whole volume on a separate path and then symlink it to the sub path.

In the dockerfile
RUN mkdir -p /data/subdir
RUN ln -s /data/subdir /var/www/subdir

Then mount the volume as normal. The /subdir must exist in the volume.
docker run -d -v myvol:/data mycontainer

Now anything read or written by the webserver will be stored in the volume subdir and can't access the other data.

Looking at the current docker compose syntax
There is a long form for specifying volumes and bind mounts

    volumes:
      - type: volume
        source: mydata
        target: /data
        volume:
          nocopy: true
      - type: bind
        source: ./static
        target: /opt/app/static

As a suggestion what might be easiest to implement would be something like a volume prefix in the bind mount in the long form of the syntax.
Something like

    volumes:
      - type: bind
        source: somevolume:/subdir1/subdir2
        target: /opt/app/static

The approach I've moved to now is calling docker-compose from a wrapper python script.
To setup the volume via the python docker api, get the volume path then substitute this into the docker-compose.yml using jinja2 for bind mounts

This only works if the script is running on the same host that container docker though, since docker doesn't nativity have a way to copy paste files into volumes

also someone does have a convience script here for that sort of thing when needed remotely

Was this page helpful?
0 / 5 - 0 ratings