Postgres: Using volumes for persistent data:'initdb: directory "/var/lib/postgresql/data" exists but is not empty'

[ambrt]

I want to use docker volume named "taiga-postgres-pers" to store postgres data between removing ( postgres container (named "taiga-postgres")

The code for "taiga-postgres" is:

docker run \
--name taiga-postgres \
 -v taiga-postgres-pers:/var/lib/postgresql/data \
-e POSTGRES_PASSWORD=mypassword   -d postgres

The container exits immediately with error in logs:

initdb: directory "/var/lib/postgresql/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/var/lib/postgresql/data" or run initdb
with an argument other than "/var/lib/postgresql/data".

So how can i attach existing volume to newly created postgres docker container?

[yosifkit]

The volume needs to be empty or a valid already initialized postgres database with the file PG_VERSION in there so the init can be skipped.

Did you use a docker volume plugin to create the volume? If there are any files or folders in there like lost+found it will probably fail to initialize. If there are files that you want to keep in the volume (or have no control over) you could adjust the PGDATA environment variable to point to a sub-directory in there like -e PGDATA=/var/lib/postgresql/data/db-files/.

[yosifkit]

Closing old issue; seems solved.

[e-nouri]

Please reopen this, as of now I am using postgres:9.6.6 in Kubernetess 1.8 on GCE, when I updated the cluster I lost all of my data because this does not work.

The wanted behaviour:
I mount the volume (persistant claim) to the docker and the database files are stored in the mounted claim, each time I mount that volume, I always find my database there.

The acctual behavior of the postgres:9.6.6 docker:

• Each time the POD boot up, it sets a new database on the mounted claim ! I also exported PGDATA in the deployment file, but it creates a new database each time!

I also ssh'ed into the POD and tested that the PG_VERSION is there and not empty.

The documentation on the docker/_/postgres is confusing and not clear at all.

[tianon]

@e-nouri that sounds like an issue with your k8s pod configuration, not the image -- I've personally verified with stock Docker that attaching the same volume initialized by this image to a new container does indeed work properly and doesn't lose any data

I'd recommend trying the Docker Community Forums, the Docker Community Slack, Stack Overflow, or a Kubernetes-specific support forum for help debugging your issue/configuration.

[e-nouri]

I actually found what the problem was, and it is indeed not related to the docker env variables.
For future references, if the volume is mounted in the POD/Deloypment, don't export the PGDATA env variable because it will tell initdb to make a new one and to remove the old data.

[micdoher]

Hi e-nouri, thanks for posting back the fix!. I have the same issue but not the experience to know where to stop the PGDATA env var being exported. How do I do this please as I cannot see it in the pod yml? Is this done in the postgres image (9.6.6) and can be over-ridden by the k8s yml?

[gsf]

@micdoher The PGDATA env would be in the configuration yaml. See for example the helm chart:
https://github.com/helm/charts/blob/614cc591a518cb099ce1055e2cb140b20bf5facd/stable/postgresql/templates/deployment.yaml#L69

[vladkras]

so I managed it by adding a subPath to my Deployment yaml

not 100% sure this is correct, I'm new to this, found on stackoverflow

[vladkras]

also check https://github.com/docker-library/postgres/issues/263#issuecomment-431637087

[megalepozy]

I'm having an issue which is very similar, in my deployment yaml I set the following (cut out not relevant parts for brevity):

spec:
template:
spec:
containers:
- name: db-postgres
image: postgres:9.6.11
volumeMounts:
- name: tr-db-volume-mount
mountPath: /var/lib/postgresql/data
readOnly: false
volumes:
- name: tr-db-volume-mount
hostPath:
path: /data/bh # <- this folder does not exist before creating the deployment
type: DirectoryOrCreate

The result of this is that I get an error from initdb that the folder exist already AND the new folder /data/bh get created and get populated only with the global & pg_xlog folders (both r empty), so its both creating the folder partially AND complaining that it exist already...

If anyone got any idea I'll be so happy to hear about it, breaking my head on it for 2 days now.... :)

btw, @e-nouri with or without PGDATA env at the deployment didnt change a thing for me.. also @vladkras using subPath didnt help

[rbq]

I managed to run Postgres (after loosing some data and a lot of experimentation) like so (left out some stuff as well):

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
      - name: postgres
        image: postgres:9.5
        env:
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          value: "xxxxxxxx"
        - name: POSTGRES_DB
          value: "xxxxxxxx"
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-data
          mountPath: "/var/lib/postgresql/data"
      volumes:
      - name: postgres-data
        persistentVolumeClaim:
          claimName: postgres-data

[megalepozy]

@rbq first of all thx a lot for ur quick reply, at the end I discovered (about 5 minutes after posting here and just now finished to check everything works) that I was foolish with my intended goal, what I tried to do is to save the data of the db at my host instead of letting minikube deal with it on its own... I guess I was lost in translation somewhere.... for 2 days.... damn :)

btw if anyone want to do that (y I dont know), I forgot to mention about that I also ran $ minikube mount /data:/data

[yami12376]

Running this command gives me the same error:
docker run --name some-postgres -v E:\tmp:/var/lib/postgresql/data -d postgres:9.6.15

initdb: directory "/var/lib/postgresql/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/var/lib/postgresql/data" or run initdb
with an argument other than "/var/lib/postgresql/data".

The folder tmp was not empty and postgres needs an empty folder at start.

Now i get

waiting for server to start....FATAL: data directory "/var/lib/postgresql/data" has wrong ownership

Windows here.