Mysql: Cannot persist the data by doing docker commit
Hi,
Please help! This is urgent to me.
I have tried to start the mysql image (5.6.22) in a couple of ways:
docker run -it -h mysql --name mysqlt1 -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
docker run -it -h mysql --name mysqlt1 -v $PWD/var/lib/mysql:/var/lib/mysql -v $PWD/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test mysql:5.6.22
Then, I commit the image, stop the old one, and start the new image by executing the same docker run command with the tag replaced with the new image's.
Result:
Every time when the new image starts, it will execute /tmp/mysql-first-time.sql and so the data is removed.
What I'd like to achieve is to persist the data inside the container. In the mounted volume is fine as well.
Startup Logs:
$ docker run -it -h mysql --name mysqlt2 -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:t2
Running mysql_install_db ...
Installing MySQL system tables...2015-01-28 05:29:26 12 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-01-28 05:29:26 12 [Note] InnoDB: The InnoDB memory heap is disabled
2015-01-28 05:29:26 12 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-01-28 05:29:26 12 [Note] InnoDB: Memory barrier is not used
2015-01-28 05:29:26 12 [Note] InnoDB: Compressed tables use zlib 1.2.7
2015-01-28 05:29:26 12 [Note] InnoDB: Using Linux native AIO
2015-01-28 05:29:26 12 [Note] InnoDB: Not using CPU crc32 instructions
2015-01-28 05:29:26 12 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-01-28 05:29:26 12 [Note] InnoDB: Completed initialization of buffer pool
2015-01-28 05:29:26 12 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created!
2015-01-28 05:29:26 12 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB
2015-01-28 05:29:26 12 [Note] InnoDB: Database physically writes the file full: wait...
2015-01-28 05:29:26 12 [Note] InnoDB: Setting log file ./ib_logfile101 size to 48 MB
2015-01-28 05:29:26 12 [Note] InnoDB: Setting log file ./ib_logfile1 size to 48 MB
2015-01-28 05:29:27 12 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
2015-01-28 05:29:27 12 [Warning] InnoDB: New log files created, LSN=45781
2015-01-28 05:29:27 12 [Note] InnoDB: Doublewrite buffer not found: creating new
2015-01-28 05:29:27 12 [Note] InnoDB: Doublewrite buffer created
2015-01-28 05:29:27 12 [Note] InnoDB: 128 rollback segment(s) are active.
2015-01-28 05:29:27 12 [Warning] InnoDB: Creating foreign key constraint system tables.
2015-01-28 05:29:27 12 [Note] InnoDB: Foreign key constraint system tables created
2015-01-28 05:29:27 12 [Note] InnoDB: Creating tablespace and datafile system tables.
2015-01-28 05:29:27 12 [Note] InnoDB: Tablespace and datafile system tables created.
2015-01-28 05:29:27 12 [Note] InnoDB: Waiting for purge to start
2015-01-28 05:29:27 12 [Note] InnoDB: 5.6.22 started; log sequence number 0
2015-01-28 05:29:27 12 [Note] Binlog end
2015-01-28 05:29:27 12 [Note] InnoDB: FTS optimize thread exiting.
2015-01-28 05:29:27 12 [Note] InnoDB: Starting shutdown...
2015-01-28 05:29:28 12 [Note] InnoDB: Shutdown completed; log sequence number 1625977
OK
Filling help tables...2015-01-28 05:29:29 35 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-01-28 05:29:29 35 [Note] InnoDB: The InnoDB memory heap is disabled
2015-01-28 05:29:29 35 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-01-28 05:29:29 35 [Note] InnoDB: Memory barrier is not used
2015-01-28 05:29:29 35 [Note] InnoDB: Compressed tables use zlib 1.2.7
2015-01-28 05:29:29 35 [Note] InnoDB: Using Linux native AIO
2015-01-28 05:29:29 35 [Note] InnoDB: Not using CPU crc32 instructions
2015-01-28 05:29:29 35 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-01-28 05:29:29 35 [Note] InnoDB: Completed initialization of buffer pool
2015-01-28 05:29:29 35 [Note] InnoDB: Highest supported file format is Barracuda.
2015-01-28 05:29:29 35 [Note] InnoDB: 128 rollback segment(s) are active.
2015-01-28 05:29:29 35 [Note] InnoDB: Waiting for purge to start
2015-01-28 05:29:29 35 [Note] InnoDB: 5.6.22 started; log sequence number 1625977
2015-01-28 05:29:29 35 [Note] Binlog end
2015-01-28 05:29:29 35 [Note] InnoDB: FTS optimize thread exiting.
2015-01-28 05:29:29 35 [Note] InnoDB: Starting shutdown...
2015-01-28 05:29:31 35 [Note] InnoDB: Shutdown completed; log sequence number 1625987
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h mysql password 'new-password'
Alternatively you can run:
/usr/bin/mysql_secure_installation
which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.
See the manual for more instructions.
You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &
You can test the MySQL daemon with mysql-test-run.pl
cd mysql-test ; perl mysql-test-run.pl
Please report any problems at http://bugs.mysql.com/
The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at http://shop.mysql.com
WARNING: Found existing config file /usr/my.cnf on the system.
Because this file might be in use, it was not replaced,
but was used in bootstrap (unless you used --defaults-file)
and when you later start the server.
The new default config file was created as /usr/my-new.cnf,
please compare it with your file and take the changes you need.
WARNING: Default config file /etc/mysql/my.cnf exists on the system
This file will be read by default by the MySQL server
If you do not want to use this, either remove it, or use the
--defaults-file argument to mysqld_safe when starting the server
Finished mysql_install_db
2015-01-28 05:29:32 1 [Note] Plugin 'FEDERATED' is disabled.
2015-01-28 05:29:32 1 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-01-28 05:29:32 1 [Note] InnoDB: The InnoDB memory heap is disabled
2015-01-28 05:29:32 1 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-01-28 05:29:32 1 [Note] InnoDB: Memory barrier is not used
2015-01-28 05:29:32 1 [Note] InnoDB: Compressed tables use zlib 1.2.7
2015-01-28 05:29:32 1 [Note] InnoDB: Using Linux native AIO
2015-01-28 05:29:32 1 [Note] InnoDB: Not using CPU crc32 instructions
2015-01-28 05:29:32 1 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-01-28 05:29:32 1 [Note] InnoDB: Completed initialization of buffer pool
2015-01-28 05:29:32 1 [Note] InnoDB: Highest supported file format is Barracuda.
2015-01-28 05:29:32 1 [Note] InnoDB: 128 rollback segment(s) are active.
2015-01-28 05:29:32 1 [Note] InnoDB: Waiting for purge to start
2015-01-28 05:29:32 1 [Note] InnoDB: 5.6.22 started; log sequence number 1625987
2015-01-28 05:29:32 1 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: a3ae727e-a6ae-11e4-8719-0242ac110032.
2015-01-28 05:29:32 1 [Note] Server hostname (bind-address): '*'; port: 3306
2015-01-28 05:29:32 1 [Note] IPv6 is available.
2015-01-28 05:29:32 1 [Note] - '::' resolves to '::';
2015-01-28 05:29:32 1 [Note] Server socket created on IP: '::'.
2015-01-28 05:29:32 1 [Note] Event Scheduler: Loaded 0 events
2015-01-28 05:29:32 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' started.
2015-01-28 05:29:32 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' ended.
2015-01-28 05:29:32 1 [Note] mysqld: ready for connections.
Version: '5.6.22' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
2015-01-28 05:36:47 1 [Note] mysqld: Normal shutdown
Env:
Os: OSX 10.10.1
Docker: Docker version 1.3.1, build 4e9bbfa
Boot2Docker-cli version: v1.3.1
Git commit: 57ccdb8
Thanks so much!
skwokmag
All 34 comments
In your example, you're not using a bind-mount, because of this, docker will create a new (empty) volume and the mysql container will initialise a new database.
The Dockerfile of the mysql image contains a volume definition for the mysql-data (https://github.com/docker-library/mysql/blob/master/5.7/Dockerfile#L35), so the data _itself_ will never be included in the container itself, but also won't be deleted if you delete the container (unless doing docker rm -v container)
Possible solutions;
- use a bind-mounted volume to store the data and use the same location for the new container
- Use a data-only container (built from the mysql image) and use
--volumes-fromto use that data in your mysql container.
Let me know if you need more help with this.
Also wondering, what was your reason to _commit_ the container? Were you trying to commit the database so that you could "clone" the container and have a second container with the same data in it?
thaJeztah
on 28 Jan 2015
Thanks for your quick reply!
Your answer makes sense to me but the problem is, I am mounting '/var/lib/mysql' and '/etc/mysql' already in my second command. Did I do something wrong?
Saving the data inside the container is for the ready-to-use out-of-the-box experience. It is fine if it is not as I don't mind executing a sql script afterwards.
skwokmag
on 28 Jan 2015
Yes, I noticed that you used bind-mounted volumes in your first part. You weren't using them in the "long" example.
I also just noticed you're starting the container with -it (interactive), I take it this is just for debugging here? Because once you CTRL+C, MySQL (and the container) will stop.
I _think_ something like this should work;
docker run -d -h mysql --name mysqlt1 -v $PWD/var/lib/mysql:/var/lib/mysql -v $PWD/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test mysql:5.6.22
#once mysql is ready, stop and remove the container
docker stop mysqlt1
docker rm mysqlt1
# and start a new container using the same bind-mounts
docker run -d -h mysql --name mysqlt2 -v $PWD/var/lib/mysql:/var/lib/mysql -v $PWD/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test mysql:5.6.22
Untested, just "from the top of my head".
To export/import your database, here's some (unfinished) notes I made a while ago that may be useful;
https://gist.github.com/thaJeztah/8d0e901bd21329d80cf2
thaJeztah
on 28 Jan 2015
Thanks again for your quick reply. I just tried the following. Would you know why I got the error regarding running mysqld as root?
$ docker run -it -h mysql --name mysqltmount_var_etc -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -v $PWD/mysql/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
2015-01-28 06:47:26 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-01-28 06:47:26 1 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2015-01-28 06:47:26 1 [ERROR] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
2015-01-28 06:47:26 1 [ERROR] Aborting
2015-01-28 06:47:26 1 [Note] Binlog end
2015-01-28 06:47:26 1 [Note] mysqld: Shutdown complete
(yes, -it is only for debugging)
skwokmag
on 28 Jan 2015
Would you know why I got the error regarding running mysqld as root?
Interesting, I got the same error with mysql:5.6.22, however, running mysql:5.6(which looks to be mysql:5.6.21 mysql:5.6.21 worked fine;
update; mysql:5.6 _is_ mysql:5.6.22, I hadn't pulled the latest version of that image. I'll update my example below.
docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -v $PWD/mysql/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.21
Checking logs of the first start;
docker logs mysqltmount_all_dir
...
2015-01-28 20:56:40 1 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 2888b34e-a730-11e4-8a66-0242ac110037.
2
....
2015-01-28 20:56:40 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' started.
2015-01-28 20:56:40 1 [Note] Execution of init_file '/tmp/mysql-first-time.sql' ended.
2015-01-28 20:56:40 1 [Note] mysqld: ready for connections.
Version: '5.6.21' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
Remove the first container and start a new one
docker stop mysqltmount_all_dir
docker rm mysqltmount_all_dir
docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -v $PWD/mysql/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.21
And check the logs of the second/new container
docker logs mysqltmount_all_dir
...
2015-01-28 20:57:37 1 [Note] Event Scheduler: Loaded 0 events
2015-01-28 20:57:37 1 [Note] mysqld: ready for connections.
Version: '5.6.21' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
So, yes, there seems to be an issue with mysql:5.6.22
thaJeztah
on 28 Jan 2015
Created a separate issue for the "root" problem here: https://github.com/docker-library/mysql/issues/45
thaJeztah
on 28 Jan 2015
Hi, thanks so much for your support. I think there may be something subtle happening here. I copied and pasted your command and it still failed for me:
$ docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -v $PWD/mysql/etc/mysql:/etc/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6
7dc5db3f8a0a34ef489765ac465e298d550b5120f81c35f68d018cca0f06602a
$ docker logs mysqltmount_all_dir
2015-01-28 07:37:04 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-01-28 07:37:04 1 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2015-01-28 07:37:04 1 [ERROR] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
2015-01-28 07:37:04 1 [ERROR] Aborting
2015-01-28 07:37:04 1 [Note] Binlog end
2015-01-28 07:37:04 1 [Note] mysqld: Shutdown complete
I am quite sure this is related to bind mounting over the config dir: https://github.com/docker-library/mysql/commit/12ffb2f321313dce32d24bff53a317f8f6b2e9c5#commitcomment-9444466
yosifkit
on 28 Jan 2015
@skwokmag My mysql:5.6 image was outdated (updated my comment above), doing mysql:5.6.21 _did_ work.
However, the "root" problem is because of a recent change in the MySQL image as @yosifkit mentioned. I forgot about that.
The problem occurs if you bind-mount the whole mysql configuration directory. Not doing that, will make it work also with the mysql:5.6.22 image;
docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
Also see the link that @yosifkit added (https://github.com/docker-library/mysql/commit/12ffb2f321313dce32d24bff53a317f8f6b2e9c5#commitcomment-9444466), that explains how to mount the configuration if you have to.
I hope you find your way around with these examples. I don't think there's an issue with the MySQL image and I suggest to close this issue (the GitHub issue tracker is not really meant as a support forum :smile:)
thaJeztah
on 28 Jan 2015
Hi @thaJeztah , I am not sure why but it still failed for me:
skwok-wpc:docker_demo skwok$ docker rmi mysql:5.6.22
Untagged: mysql:5.6.22
skwok-wpc:docker_demo skwok$
skwok-wpc:docker_demo skwok$
skwok-wpc:docker_demo skwok$
skwok-wpc:docker_demo skwok$ docker rm mysqltmount_all_dir
mysqltmount_all_dir
skwok-wpc:docker_demo skwok$ docker run -it -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
Unable to find image 'mysql:5.6.22' locally
Pulling repository mysql
335228ceb173: Download complete
511136ea3c5a: Download complete
d0a18d3b84de: Download complete
4d6ce913b130: Download complete
628e63ee44af: Download complete
ad01a54357ee: Download complete
119481e10155: Download complete
fb88d88b1c80: Download complete
f53e99de0738: Download complete
4320535b4010: Download complete
40403c37203c: Download complete
c9495b31f987: Download complete
6d5b269aa9e0: Download complete
66100f5f15c6: Download complete
169fc3e22468: Download complete
526f51d1feaa: Download complete
Status: Image is up to date for mysql:5.6.22
2015-01-28 07:57:43 1 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2015-01-28 07:57:43 1 [Note] Plugin 'FEDERATED' is disabled.
mysqld: Table 'mysql.plugin' doesn't exist
2015-01-28 07:57:43 1 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
2015-01-28 07:57:43 1 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-01-28 07:57:43 1 [Note] InnoDB: The InnoDB memory heap is disabled
2015-01-28 07:57:43 1 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-01-28 07:57:43 1 [Note] InnoDB: Memory barrier is not used
2015-01-28 07:57:43 1 [Note] InnoDB: Compressed tables use zlib 1.2.7
2015-01-28 07:57:43 1 [Note] InnoDB: Using Linux native AIO
2015-01-28 07:57:43 1 [Note] InnoDB: Not using CPU crc32 instructions
2015-01-28 07:57:43 1 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-01-28 07:57:43 1 [Note] InnoDB: Completed initialization of buffer pool
2015-01-28 07:57:43 7f3a1b77d720 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
2015-01-28 07:57:43 7f3a1b77d720 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
2015-01-28 07:57:43 1 [ERROR] InnoDB: Creating or opening ./ibdata1 failed!
2015-01-28 07:57:43 1 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
2015-01-28 07:57:43 1 [ERROR] Plugin 'InnoDB' init function returned error.
2015-01-28 07:57:43 1 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2015-01-28 07:57:43 1 [ERROR] Unknown/unsupported storage engine: InnoDB
2015-01-28 07:57:43 1 [ERROR] Aborting
2015-01-28 07:57:43 1 [Note] Binlog end
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'partition'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'BLACKHOLE'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'ARCHIVE'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_METRICS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMPMEM'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_CMP'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_LOCKS'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'INNODB_TRX'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'MEMORY'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'CSV'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'MRG_MYISAM'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'MyISAM'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'sha256_password'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'mysql_old_password'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'mysql_native_password'
2015-01-28 07:57:43 1 [Note] Shutting down plugin 'binlog'
2015-01-28 07:57:43 1 [Note] mysqld: Shutdown complete
skwok-wpc:docker_demo skwok$
I think your problem there is running the container _interactively_ and with a TTY (-it), try doing the same, but "detached"; docker run -d -h mysql ......
(at least, I got the same result when running with -it, but it ran fine detached (-d))
thaJeztah
on 28 Jan 2015
Hi @thaJeztah ,
This is what I got with using '-d' instead of '-it':
skwok-wpc:docker_demo skwok$ docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
e931bdff3b0105ae4cecdd8aa061c7825232e106e903f390db53ca560228867f
skwok-wpc:docker_demo skwok$ docker logs mysql
2015/01/28 14:01:12 Error response from daemon: No such container: mysql
skwok-wpc:docker_demo skwok$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
skwok-wpc:docker_demo skwok$
because you named your container mysqltmount_all_dir?
docker logs mysqltmount_all_dir should give you the logs.
If you're running on a host with limited memory, and the container stops during the startup, you may have to enable swap. See this ticket for more information on that; https://github.com/docker-library/mysql/issues/3
thaJeztah
on 28 Jan 2015
Hi @thaJeztah , yes you are right, I used a wrong container name there. I have corrected that and here is the new screen output:
skwok-wpc:docker_demo skwok$ docker run -d -h mysql --name mysqltmount_all_dir -v $PWD/mysql/var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=test -e MYSQL_USER=mmx -e MYSQL_PASSWORD=mmx -e MYSQL_DATABASE=mmx mysql:5.6.22
7035f582d6ebb55dd0bf0f824e86bf42aea99053b769a44e400bda7dd03ee80c
skwok-wpc:docker_demo skwok$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
skwok-wpc:docker_demo skwok$ docker logs mysqltmount_all_dir
2015-01-29 01:13:10 1 [Warning] Setting lower_case_table_names=2 because file system for /var/lib/mysql/ is case insensitive
2015-01-29 01:13:10 1 [Note] Plugin 'FEDERATED' is disabled.
mysqld: Table 'mysql.plugin' doesn't exist
2015-01-29 01:13:10 1 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
2015-01-29 01:13:10 1 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-01-29 01:13:10 1 [Note] InnoDB: The InnoDB memory heap is disabled
2015-01-29 01:13:10 1 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-01-29 01:13:10 1 [Note] InnoDB: Memory barrier is not used
2015-01-29 01:13:10 1 [Note] InnoDB: Compressed tables use zlib 1.2.7
2015-01-29 01:13:10 1 [Note] InnoDB: Using Linux native AIO
2015-01-29 01:13:10 1 [Note] InnoDB: Not using CPU crc32 instructions
2015-01-29 01:13:10 1 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2015-01-29 01:13:10 1 [Note] InnoDB: Completed initialization of buffer pool
2015-01-29 01:13:10 7ffd9b922720 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
2015-01-29 01:13:10 7ffd9b922720 InnoDB: Operating system error number 13 in a file operation.
InnoDB: The error means mysqld does not have the access rights to
InnoDB: the directory.
2015-01-29 01:13:10 1 [ERROR] InnoDB: Creating or opening ./ibdata1 failed!
2015-01-29 01:13:10 1 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data!
2015-01-29 01:13:10 1 [ERROR] Plugin 'InnoDB' init function returned error.
2015-01-29 01:13:10 1 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2015-01-29 01:13:10 1 [ERROR] Unknown/unsupported storage engine: InnoDB
2015-01-29 01:13:10 1 [ERROR] Aborting
2015-01-29 01:13:10 1 [Note] Binlog end
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'partition'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'BLACKHOLE'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'ARCHIVE'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_METRICS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMPMEM'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_CMP'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_LOCKS'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'INNODB_TRX'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'MEMORY'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'CSV'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'MRG_MYISAM'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'MyISAM'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'sha256_password'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'mysql_old_password'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'mysql_native_password'
2015-01-29 01:13:10 1 [Note] Shutting down plugin 'binlog'
2015-01-29 01:13:10 1 [Note] mysqld: Shutdown complete
skwok-wpc:docker_demo skwok$
Please note that I have already shared the dir in Virtual box by executing:
VBoxManage sharedfolder add boot2docker-vm -name mysqlshare -hostpath /Users/skwok/Work/sources/mmx1/docker_demo/mysql/var/lib/mysql
Would it be possible that it is related to:
http://bugs.mysql.com/bug.php?id=72658
skwokmag
on 29 Jan 2015
FYI, I ran bash with the container and saw the followings (please note the group:user of the mysql dir):
root@mysql1:/# ls -la /var/lib/
total 44
drwxr-xr-x 16 root root 4096 Jan 22 01:31 .
drwxr-xr-x 18 root root 4096 Jan 22 01:31 ..
drwxr-xr-x 7 root root 4096 Jan 22 01:31 apt
drwxr-xr-x 14 root root 4096 Jan 22 01:31 dpkg
drwxr-xr-x 2 root root 4096 Jul 14 2013 initscripts
drwxr-xr-x 2 root root 4096 Jul 14 2013 insserv
drwxrwsr-x 2 libuuid libuuid 4096 Dec 11 2012 libuuid
drwxr-xr-x 2 root root 4096 Dec 24 13:41 misc
drwxr-xr-x 1 1000 staff 204 Jan 29 01:39 mysql
drwxr-xr-x 2 root root 4096 Jan 16 21:15 pam
drwxr-xr-x 2 root root 4096 Nov 23 2012 update-rc.d
drwxr-xr-x 2 root root 4096 Jul 14 2013 urandom
skwokmag
on 29 Jan 2015
The problem you're having there is most likely caused by the fact you're using the VirtualBox shared folder. Docker will always preserve the uid/gid of the host when mounting a volume. You can try to chown those before using, but even then the performance will be really bad (VirtualBox shared folders are really slow)
Anyway, these are not issues that can be solved by the MySQL image; I suggest to use a _regular_ mounted host directory, not a VBox shared folder and browse the boot2docker issue tracker for tips on working with the VBox shares, because there are a lot of issues with those.
thaJeztah
on 29 Jan 2015
What's the best approach for solving this? I'm simply trying to mount the data directory of the container's to the host one
mikeys
on 5 Feb 2015
I'm simply trying to mount the data directory of the container's to the host one
Mounting the data-directory of a container to the "host" should work. But if you're using Boot2Docker and the directory on the "host" (host being the VirtualBox VM) is a VirtualBox shared folder, permissions can become tricky. At least, that's what I've heard in various places (I don't use Boot2Docker myself).
thaJeztah
on 5 Feb 2015
I'm using Boot2Docker on OSX and from what I understand, one of the main benefits of Boot2docker is to abstract the mounting of volumes from your host machine into the docker containers through some sort of mapping between the VBox folders and your host folders. I'm not sure about the internals of such abstraction.
mikeys
on 5 Feb 2015
@mikeys Boot2Docker creates a VirtualBox share of /Users on your Mac to /Users on the Linux host. This is what lets you share any folder under your Mac's /Users directory with a docker container, because the paths are the same.
Really what it does is automatically mount a VirtualBox share named Users to /Users (under Linux), and it so happens that it creates a share called Users from your Mac's /Users.
The problem is that these shares are owned by the docker user (uid 1000) on in the Linux host. This means that any container that needs to write to a shared folder on your Mac will need to run its process as UID 1000.
This is only a problem with Boot2Docker because it uses the virtual box sharing, which changes the uid of the share to the docker user and you can't change this easily from Linux.
In Vagrant, I usually solved this by changing my Vagrant file to use different permissions for the virtual box share:
config.vm.synced_folder ".", "/vagrant", :mount_options => ['dmode=777,fmode=666']
Maybe you could do something like that inside of your Boot2Docker image, I haven't looked into it. I think you would make these changes to /etc/rc.d/automount-shares in the Linux VM
One way I solved this was to just set my containers up so that I could configure them with the UID I wanted them to run their processes as, so I could just set it to 1000. The downside to this is that you basically need to create your own images for everything since most containers I've found don't let you do this.
I hope this helps.
caleb
on 6 Feb 2015
Thanks for helping out, @caleb! Didn't have the right info at hand to give a helpful answer myself.
thaJeztah
on 6 Feb 2015
I built my own Boot2Docker iso that mounts filesystems with more liberal permissions, it was very easy:
Clone https://github.com/boot2docker/boot2docker
Edit rootfs/rootfs/etc/rc.d/automount-shares and change the line that says:
mountOptions='defaults'
to
mountOptions='defaults,dmode=777,fmode=666'
Then build the image with your existing boot2docker installation and write the iso to the current directory:
docker build -t boot2docker . && docker run --rm boot2docker > boot2docker.iso
Then you can destroy your current boot2docker vm: boot2docker stop && boot2docker delete
Then copy the new iso to your ~/.boot2docker directory: cp boot2docker.iso ~/.boot2docker
Then re-init your boot2docker VM: boot2docker init && boot2docker up
There you go, when you ssh into your boot2docker vm now, all the directories will be 777 and files will be 666.
caleb
on 7 Feb 2015
@caleb interesting. Given that Boot2Docker + VBox will (probably) only be used for development setups, this might actually be a nice (although duct-tape-ish) workaround that saves many people headaches.
Have you discussed that solution in the Boot2Docker repository? I think you should; to see what the maintainers think of it?
thaJeztah
on 7 Feb 2015
@thaJeztah I played around with those same mountOptions a bit. The first drawback that I noticed of setting fmode=0666 was that executable files were not executable via the mount. You could use fmode=0777, but then every file was exectuable.
md5
on 7 Feb 2015
The first drawback that I noticed of setting fmode=0666 was that executable files were not executable via the mount.
Ouch! Yup that's not handy.
Bummer, because that would have reduced a lot of support questions wrt b2d. Otoh, is the situation _worse_ than the current case?
thaJeztah
on 7 Feb 2015
Bummer, because that would have reduced a lot of support questions wrt b2d. Otoh, is the situation _worse_ than the current case?
@thaJeztah if you're implying that the situation is in fact better with fmode=0777 than without it, I'd tend to agree. Then again, it seems like enough of a hack that it could have unpleasant repercussions.
md5
on 7 Feb 2015
I knew that would be a problem, but I just store uploaded files, databases, etc inside of shared volumes, so the executable bit is not important to me (maybe except for shell scripts, but you can always run them through the interpreter).
Maybe this shouldn't be a default, but I believe that for me it's better that the mysql container can write to a mounted directory.
In production you have to worry about permissions as well, but you can just chown/chmod the directories to fit your needs, whereas boot2docker's shared filesystems don't allow you do this.
And of course, the other solution is to just use data containers (that live inside the b2d VM) and samba containers to peek into them when you need to. This is something that I haven't looked into yet.
caleb
on 7 Feb 2015
@md5 yeah. I'm really not sure what's best here. In the end, VBox sharing in itself was added as a stop-gap solution until something better was implemented. I see numerous questions come by from people running into both _performance_ and _permission_ problems. It's hard to explain, especially to people not fully understanding the relation to their workstation (OS X) and the boot2docker VM, let alone volumes and containers.
So anything that would make it work better "out of the box" would be welcome. Not sure if this is the way to go either.
thaJeztah
on 7 Feb 2015
Thanks @caleb, your solution worded for me on box ubuntu/trusty64
Added to my Vagrantfile:
config.vm.synced_folder ENV["MYSQL_PATH"], "/mysql_datadir", mount_options: ['dmode=777,fmode=666']
arnlen
on 9 Apr 2015
None of these are great solution to me because it requires everyone to modify the base box every time the install a new version or like just recently with docker toolbox you would have to do the above again to configure file permissions.
I just copy /usr/local/var/mysql/* into the volume i am going to mount and then chmod 664 the whole folder.
michaeljs1990
on 3 Sep 2015
@skwokmag, just add --user 1000 (1000 - docker UID) like this:
docker run --name mysql -v $PWD/var/lib/mysql:/var/lib/mysql -v $PWD/etc/mysql:/etc/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=yes --user 1000 mysql:5.7.13
an you'll get this:
...
2016-07-28T10:40:31.770773Z 0 [Note] mysqld: ready for connections.
Version: '5.7.13' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
brlo
on 28 Jul 2016
Also note that the permissions issue should be solved by Docker for Mac
thaJeztah
on 28 Jul 2016
Hi,
I'm using docker for Mac and still same issue: Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
docker run --name mysqlt1 -h mysql --rm -it -v ~/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=pass -e MYSQL_USER=pass -e MYSQL_PASSWORD=pass -e MYSQL_DATABASE=pass_dev mysql:5.7
sebastiannm
on 3 Aug 2016
@brlo that worked for me! Thanks
thiagoh
on 26 Sep 2016
Related issues
BirkhoffLee
路
4Comments
EnziinSystem
路
3Comments
KomaBeyond
路
4Comments
jicki
路
3Comments
mcandre
路
4Comments
Most helpful comment
Hi,
I'm using docker for Mac and still same issue: Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
docker run --name mysqlt1 -h mysql --rm -it -v ~/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=pass -e MYSQL_USER=pass -e MYSQL_PASSWORD=pass -e MYSQL_DATABASE=pass_dev mysql:5.7