Docker-images: Problem checkSpace.sh: The container needs at least 15 GB
Hi,
I have this problem again.
df -B 1G . | tail -n 1 | awk '{ print $4 }' return 25 and docker info says that the Base Device Size is 34.36 GB
I have 38G available on the host.
I'm still increasing the Base Device Size but the script always returns :
checkSpace.sh: The container needs at least 15 GB , but only 6 available.
What I'm doing wrong?
Thanks,
Adrien
adrien-zinger
All 22 comments
Hi @adrien-zinger,
You are doing nothing wrong, the checkSpace.sh script is doing something wrong.
I wonder, can you confirm the script one more time. We just recently fixed a bug (#532) where we are using df -B 1G / instead of df -B 1G .. The . causes issues on some platforms, the / should fix that.
I see that the above line you still have the . there however. Can you confirm that you are running with the latest version of the script?
gvenzl
on 14 Sep 2017
The problem is not solved.
The command df -B 1G / | tail -n 1 | awk '{ print $4 }' return 18.
adrien-zinger
on 15 Sep 2017
Can you please provide the output of just df -B 1G /.
gvenzl
on 16 Sep 2017
This is the output
Filesystem 1G-blocks Used Available Use% Mounted on
/dev/xvda1 75 57 15 80% /
Hm, based on that output the script returns the correct output. df -h reports that there are only 15GB available in /. Can you confirm the number? I'm slightly confused by the previous messages. First you said it reported 25GB, then 18GB and now the output suggests 15GB.
Did you change the base device size?
Can you also provide the output of docker info and df -h inside the container. I'm wondering where all the space is going to.
gvenzl
on 19 Sep 2017
We are two on the server. The memory available can change little day by day. I use to check the memory before each test. I can provide more memory if necessary.
Today df -h on my host return:
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 75G 54G 18G 76% /
That match exactly with the output of df -B 1G /
Now here is the full output of docker info:
Containers: 18
Running: 5
Paused: 0
Stopped: 13
Images: 64
Server Version: 17.05.0-ce
Storage Driver: devicemapper
Pool Name: docker-202:1-264373-pool
Pool Blocksize: 65.54kB
Base Device Size: 34.36GB
Backing Filesystem: ext4
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 36.72GB
Data Space Total: 107.4GB
Data Space Available: 22.84GB
Metadata Space Used: 25.25MB
Metadata Space Total: 4.295GB
Metadata Space Available: 4.27GB
Thin Pool Minimum Free Space: 10.74GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.90 (2014-09-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 13.74GiB
Name: datainco
ID: PK7E:2NC4:YMNM:F4NR:WSWE:DJEN:5MXK:6HLA:7ARK:YX2T:WMTC:ZM2U
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: jimmytek
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Here is the full output of the command line ./buildDockerImage.sh -v 12.2.0.1 -e:
==========================
DOCKER info:
Containers: 18
Running: 5
Paused: 0
Stopped: 13
Images: 64
Server Version: 17.05.0-ce
Storage Driver: devicemapper
Pool Name: docker-202:1-264373-pool
Pool Blocksize: 65.54kB
Base Device Size: 34.36GB
Backing Filesystem: ext4
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 36.72GB
Data Space Total: 107.4GB
Data Space Available: 22.84GB
Metadata Space Used: 25.25MB
Metadata Space Total: 4.295GB
Metadata Space Available: 4.27GB
Thin Pool Minimum Free Space: 10.74GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.90 (2014-09-01)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 13.74GiB
Name: xxxxxx
ID: PK7E:2NC4:YMNM:F4NR:WSWE:DJEN:5MXK:6HLA:7ARK:YX2T:WMTC:ZM2U
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: xxxxxx
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
==========================
Building image 'oracle/database:12.2.0.1-ee' ...
Sending build context to Docker daemon 3.454GB
Step 1/17 : FROM oraclelinux:7-slim
---> 08a01cc7be97
Step 2/17 : MAINTAINER Gerald Venzl <[email protected]>
---> Running in af879cf014ee
---> af0a3b96982e
Removing intermediate container af879cf014ee
Step 3/17 : ENV ORACLE_BASE /opt/oracle ORACLE_HOME /opt/oracle/product/12.2.0.1/dbhome_1 INSTALL_FILE_1 "linuxx64_12201_database.zip" INSTALL_RSP "db_inst.rsp" CONFIG_RSP "dbca.rsp.tmpl" PWD_FILE "setPassword.sh" RUN_FILE "runOracle.sh" START_FILE "startDB.sh" CREATE_DB_FILE "createDB.sh" SETUP_LINUX_FILE "setupLinuxEnv.sh" CHECK_SPACE_FILE "checkSpace.sh" CHECK_DB_FILE "checkDBStatus.sh" USER_SCRIPTS_FILE "runUserScripts.sh" INSTALL_DB_BINARIES_FILE "installDBBinaries.sh"
---> Running in 51a60dad73bb
---> c4fa326fb017
Removing intermediate container 51a60dad73bb
Step 4/17 : ENV INSTALL_DIR $ORACLE_BASE/install PATH $ORACLE_HOME/bin:$ORACLE_HOME/OPatch/:/usr/sbin:$PATH LD_LIBRARY_PATH $ORACLE_HOME/lib:/usr/lib CLASSPATH $ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
---> Running in 91d9dfb4897b
---> aa6421531fec
Removing intermediate container 91d9dfb4897b
Step 5/17 : RUN mkdir -p $INSTALL_DIR && chmod 666 $INSTALL_DIR
---> Running in 2587e9c61851
---> bdd181f583f4
Removing intermediate container 2587e9c61851
Step 6/17 : COPY $INSTALL_FILE_1 $INSTALL_RSP $SETUP_LINUX_FILE $CHECK_SPACE_FILE $INSTALL_DB_BINARIES_FILE $INSTALL_DIR/
---> a8569c8d776d
Removing intermediate container 1b8e1b09f05a
Step 7/17 : COPY $RUN_FILE $START_FILE $CREATE_DB_FILE $CONFIG_RSP $PWD_FILE $CHECK_DB_FILE $USER_SCRIPTS_FILE $ORACLE_BASE/
---> 5a895eb668fb
Removing intermediate container 779c231e2320
Step 8/17 : RUN chmod ug+x $INSTALL_DIR/*.sh && sync && $INSTALL_DIR/$CHECK_SPACE_FILE && $INSTALL_DIR/$SETUP_LINUX_FILE
---> Running in 92c49aaf4c27
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
checkSpace.sh: ERROR - There is not enough space available in the docker container.
checkSpace.sh: The container needs at least 15 GB , but only 6 available.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Removing intermediate container 92c49aaf4c27
The command '/bin/sh -c chmod ug+x $INSTALL_DIR/*.sh && sync && $INSTALL_DIR/$CHECK_SPACE_FILE && $INSTALL_DIR/$SETUP_LINUX_FILE' returned a non-zero code: 1
There was an error building the image.
And finally when I run a container with the image builded, inside the container I have:
bash-4.2# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/docker-202:1-264373-ac5a602235f9e8d52f6efe6d257f5aa73990edbdf49022edf36f0b24c36e6e0b 9.8G 3.4G 5.9G 37% /
tmpfs 6.9G 0 6.9G 0% /dev
tmpfs 6.9G 0 6.9G 0% /sys/fs/cgroup
/dev/xvda1 75G 54G 18G 76% /etc/hosts
shm 64M 0 64M 0% /dev/shm
tmpfs 6.9G 0 6.9G 0% /sys/firmware
This result explain the error but I don't understand why there isn't more space.
adrien-zinger
on 19 Sep 2017
Hi @adrien-zinger, as it turns out the script works correctly, which is good. I think that this is just a confusion. In order to install the Oracle binaries (and the way how the layered file system works in Docker and not having multi-stage builds available yet), you need at least 15GB to unzip that zip file and install the Oracle binaries (which will take up about 5GB themselves).
Your container simply hasn't enough space available anymore when the script is run which happens after copying the files into the container.
The zip file itself is 3GB I believe, so you have to have at least 18GB still available on your disk, ideally 20GB not to fill up your disk entirely.
gvenzl
on 21 Sep 2017
Now with 27G on my host the problem persists.
Why do we need to run this script while building the image ? When I run my container with this command line docker run -ti --storage-opt size=20G 3a1a5ceb6249 the container has more space. I don't know how to precise to a docker image that the container will be bigger than 10G.
adrien-zinger
on 22 Sep 2017
Again, what's the size within your container when you have 27GBs available. Where is the space going?
The script is just there to prevent a pointless installation. If there isn't enough space available inside the container, the database installation will fail.
The question is why there is not enough space available inside your container, even though you have the base device size set to 35GB and more than enough space available on your host.
Somewhere the space is lost, the questions are where and why.
gvenzl
on 22 Sep 2017
When I run the script even if I get the message "There was an error building the image." it result an image (here 3a1a5ceb6249). If I start a container with this image like this docker run -ti 3a1a5ceb6249 and run the command df -h I get this:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/docker-202:1-264373-d92bac353214244ff71c4daefefc2cdb7bb2ff5abd83a363c372016a01e6b9c4 9.8G 3.4G 5.9G 37% /
tmpfs 6.9G 0 6.9G 0% /dev
tmpfs 6.9G 0 6.9G 0% /sys/fs/cgroup
/dev/xvda1 75G 49G 23G 69% /etc/hosts
shm 64M 0 64M 0% /dev/shm
tmpfs 6.9G 0 6.9G 0% /sys/firmware
That's exacly the same output than before.
adrien-zinger
on 25 Sep 2017
I had the same problem, but I could fix it:
After editing storage-opt you have to delete your base image oraclelinux because all previously pulled images still have the old configuration.
Perhaps this should be added to the README docs.
samized
on 2 Oct 2017
Thank you, that solved the problem.
But now there is a new issue. There is an error with the script installDBBinaries.sh that return non-zero because of a "Read-only file system" on "/opt/oracle/..."
Is that a version problem again ?
adrien-zinger
on 4 Oct 2017
Is it possible that you have low disc space on your docker host?
samized
on 4 Oct 2017
Yes, I think this was my problem. The full image takes 22G on my host. I tried this morning with 30G space free and it worked. Thank you
adrien-zinger
on 9 Oct 2017
Hi @adrien-zinger, I'm happy that your problem was resolved. Could you please just one more time list the issues and their remedies here. I think your issue will be the perfect first entry for a new FAQ file coming up.
gvenzl
on 11 Oct 2017
I have add --storage-opt dm.basesize=30G to /etc/sysconfig/docker-storage then i restart docker service. But when i run docker info there was no Base Device Size: 34.36GB. What can i do?
CentOS 6.9 Final
Docker version 1.7.1, build 786b29d/1.7.1
Containers: 0
Images: 0
Storage Driver: devicemapper
Pool Name: docker-253:0-392971-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: extfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 305.7 MB
Data Space Total: 268.4 GB
Data Space Available: 75.56 GB
Metadata Space Used: 1.356 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.146 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.117-RHEL6 (2016-12-13)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 2.6.32-696.20.1.el6.x86_64
Operating System: <unknown>
CPUs: 8
Total Memory: 15.58 GiB
Name: cloudzone-centos6
ID: TIWY:HXU4:PI5C:QXCR:O36V:CTLU:ES7E:B6V3:IXGJ:UIWO:RUHZ:NSNV
quanght55
on 7 Feb 2018
I was able to change the base size to 20G by creating the file /etc/docker/daemon.json and adding the following JSON:
{
"storage-opts": ["dm.basesize=20G"]
}
Storage Driver: devicemapper
Pool Name: docker-8:17-2883586-pool
Pool Blocksize: 65.54kB
Base Device Size: 21.47GB
Unfortunately, that didn't fix the problem. I still get the error:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
checkSpace.sh: ERROR - There is not enough space available in the docker container.
checkSpace.sh: The container needs at least 15 GB, but only 6 GB are available.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I wonder if this is a problem:
df -h
tmpfs 5.5G 0 5.5G 0% /run/user/1000
FYI: I'm working on an OCI Compute instance running Oracle Linux Server release 7.4.
cruepprich
on 8 Mar 2018
For what it's worth, I struggled with not being able to build an image on Centos with it not having enough disk space, looks as though the default storage driver for Centos is devicemapper. Once I switched it to overlay2 I was able to successfully build.
gschrader
on 16 Apr 2018
This fixed it for me, allowing 20GB in the container:
stop docker service
running docker daemon from command line:
dockerd --storage-opt dm.basesize=20G
lioree
on 6 Jul 2018
I had to do the following to get this working for me on CentOS 7:
- Create the file /etc/docker/daemon.json with the following content:
{
"storage-driver": "devicemapper",
"storage-opts": [
"dm.basesize=45G"
]
}
I used 45 GB because the error I was getting said it needed 35 GB.
- Restart the docker daemon
systemctl restart docker
- Delete the oraclelinux docker image
- Kickoff the script to build the Oracle images.
alikashmar
on 9 Jul 2018
Remove the old image (Oracle-linux)
REstart the docker
Since the old image will holds the info of previous dm.base size. this can't be updated. so please pull the new latest image and work.
This worked for me
RakeshNagarajan
on 16 Jul 2018
I had to do the following to get this working for me on CentOS 7:
- Create the file /etc/docker/daemon.json with the following content:
{ "storage-driver": "devicemapper", "storage-opts": [ "dm.basesize=45G" ] }I used 45 GB because the error I was getting said it needed 35 GB.
- Restart the docker daemon
systemctl restart docker
- Delete the oraclelinux docker image
- Kickoff the script to build the Oracle images.
Aloha brother @alikashmar. Thanks for your comment and I would suggest (Docker in 2020) to update your daemon.json settings :D to the one suggested below:
{
"storage-driver": "overlay",
"storage-opt": [
"dm.basesize=45G"
],
"debug": true
}
Otherwise, docker will fail to start/restart ;D. Thanks again and greets
UPDATE
Fix for checkSpace script with modified docker service (data-root).
https://github.com/oracle/docker-images/issues/781#issuecomment-685683844
alanamircruz
on 25 Aug 2020
Related issues
maheoasic
路
5Comments
montge
路
5Comments
zneil123
路
4Comments
arnab-das
路
4Comments
skoky
路
4Comments
Most helpful comment
I had to do the following to get this working for me on CentOS 7:
I used 45 GB because the error I was getting said it needed 35 GB.