Mongo: Should not declare VOLUME for /data/db

[aparamon]

Currently, Dockerfile declares VOLUME for /data/db, /data/configdb:
https://github.com/docker-library/mongo/blob/32e5645325ae5dff2ae6f934049cbd6b4635aec0/4.1/Dockerfile#L87

This is sub-optimal, because some workflows in inherited images become excessively complicated.
For example, seeding a database from a dump now requires
````Dockerfile
RUN mkdir /data/local-db

COPY mongo-dump /var/mongo-dump
RUN mongod --fork --dbpath /data/local-db --logpath /var/log/mongodb.log && \
mongorestore --db mydb /var/mongo-dump/mydb && \
mongod --shutdown --dbpath /data/local-db
RUN chown -R mongodb:mongodb /data/local-db

CMD ["mongod", "--dbpath", "/data/local-db"]
instead of just Dockerfile
RUN mongod --fork --logpath /var/log/mongodb.log && \
mongorestore --db mydb /var/mongo-dump/mydb && \
mongod --shutdown
``` because /data/db [doesn't persist](https://docs.docker.com/engine/reference/builder/#notes-about-specifying-volumes) betweendocker buildanddocker run` invocations.

It is proposed to remove VOLUME directive, and leave volumes configuration up to end user.

[yosifkit]

Storing the already initialized database files in the images layers is not a great idea. Because it would be in a copy-on-write filesystem, the moment that you start a new container and MongoDB changes any of its files, it now uses twice as much space.

See also https://docs.docker.com/storage/storagedriver/overlayfs-driver/#modifying-files-or-directories:

Writing to a file for the first time: The first time a container writes to an existing file, that file does not exist in the container (upperdir). The overlay/overlay2 driver performs a copy_up operation to copy the file from the image (lowerdir) to the container (upperdir). The container then writes the changes to the new copy of the file in the container layer.

However, OverlayFS works at the file level rather than the block level. This means that all OverlayFS copy_up operations copy the entire file, even if the file is very large and only a small part of it is being modified. This can have a noticeable impact on container write performance.

And https://docs.docker.com/storage/storagedriver/overlayfs-driver/#performance-best-practices:

Use volumes for write-heavy workloads: Volumes provide the best and most predictable performance for write-heavy workloads. This is because they bypass the storage driver and do not incur any of the potential overheads introduced by thin provisioning and copy-on-write.

Have you thought of using /docker-entrypoint-initdb.d/ to have it restore the database on start instead? (hub docs)

[aparamon]

Hi @yosifkit, thank you for your detailed explanation!
My original motivation for restoring DB at build time was to optimize the container start-up time. However now it seems that the actual speed-up would only take place in read-only scenario, as in case of write operations Docker would still have to copy the huge file.
What do you believe to be the optimal way of setting-up a pre-seeded Mongo container for read-only usage?

[chobolt]

I spent quite some time understanding why my bind volume was not being used by mongo docker image, and eventually discovered the hardcoded VOLUME in the dockerfile.

I'm fine with rebuilding the image without this declaration (which I did), but I find it weird to force the use of volumes when it should be up to the final user to decide how to store data.

[mildmojo]

In my case, I want to construct a seeded database image that will be the basis for runtime containers that use a persistent, named volume for /data/db in my development environment. I can't base my seeded image Dockerfile on the official mongo image because of the VOLUME declaration. (It took me a couple of days to figure out why my Dockerfile writes to /data/db were being discarded at build time--yikes!)

If I fork the official mongo Dockerfile and remove the VOLUME instruction, the resulting image works great. I can base my Dockerfile on this -no-volume base image, do RUN seed-my-database.sh at build time, then later invoke docker or use a compose file that mounts a volume at /data/db, and my seed data is copied to the volume when the container's created. Perfect.

Storing the already initialized database files in the images layers is not a great idea. Because it would be in a copy-on-write filesystem, the moment that you start a new container and MongoDB changes any of its files, it now uses twice as much space.

The storage space tradeoff is worth it for me, since it saves me so much time when I need to reset to a known state. My seeded DB is a few hundred megabytes or a gigabyte, which I can easily spare on my dev machine or CI instances.

Have you thought of using /docker-entrypoint-initdb.d/ to have it restore the database on start instead?

In my case, the DB seed process from a known DB dump takes 8-10 minutes to complete on my workstation. I may need to reset the DB to a known state dozens of times while debugging DB migrations or a new feature. That reset takes seconds if I have a seeded image, but would be untenable if I had to restore the DB every time.

I can work around it, but I'd love to see an official -no-volume variant image. I'd expect the extra maintenance effort to be very small, given that the rest of it would be identical.

[tianon]

If you add "--dbpath" or set ".storage.dbPath" in a specified "--config"
file, that value will be respected.

[mildmojo]

If you add "--dbpath" or set ".storage.dbPath" in a specified "--config" file, that value will be respected.

I ended up using this, but it means I have to make sure compose files or docker run commands that use this image duplicate my --dbpath /mongo-data/db argument if they're providing their own command values, which isn't great or obvious. I'd really prefer to store my data in the default location.

[tianon]

If you're building an image with the data pre-seeded, you can combat that by setting CMD:

CMD ["--dbpath", "/mongo-data/db"]

(and then it'll be the default for users who don't specify a command)

[mildmojo]

If you're building an image with the data pre-seeded, you can combat that by setting CMD:

Yeah, this is the approach aparamon outlined, and it's the strategy I'm currently using.

I agree with aparamon that a seeded image is a legitimate use case, and the official image doesn't work well for seeding at build time because it contains VOLUME /data/db. The workaround for seeded images adds Dockerfile chaff, creates an extra unused anonymous volume at runtime, and it yields images that carry an asterisk: your seed data disappears when your users' containers naïvely provide their own commands (e.g. a compose file with command: --auth or a docker run --rm mongo-seeded --auth).

As a tooling developer, it would help to have an official mongo image without VOLUME so that I could build seeded mongo images that are easy-to-use drop-in replacements for the official (empty) image.