Apollo: Where can I find the Dockerfiles?
I would like to rebuild the Docker containers used for building and running Apollo.
It looks like pieces of the Dockerfile have been posted in the issues comments (#2468).
I also found an issue (#61) suggesting that the Dockerfile used to be here docker/dev.dockerfile.
Looking at Docker Hub it looks like the Docker container is being updated frequently.
Where can I find the latest Dockerfiles used to create the containers?
jamiehowarth
All 9 comments
I would suggest you build your own docker container on top of existing one, not from scratch.
The existing docker container are built on some binaries, which is hard to re-create without some internal resource. That is the reason why the dockerfile is not listed on github anymore.
startcode
on 3 Feb 2018
Due to the restrictions of the network I use I have to be able to review the Dockerfile and make modification to base container for it to work on my network.
jamiehowarth
on 4 Feb 2018
2468 has the majority of the stuff, without perception module dependencies.
startcode
on 4 Feb 2018
For the sake of transparency, I too would also like to see the Dockerfiles supposably used to build the docker images pushed to apolloauto/apollo. It seem odd to me that they are not provided publicly, as I could not find a reason why they were removed https://github.com/ApolloAuto/apollo/commit/a763f93c241eec442d7cbfcb6c907fb598411440, nor a description if they were relocated https://github.com/ApolloAuto/apollo/pull/807.
Inspecting the image binaries that users are downloading provides little to no insight as to what is done to construct every layer in the images.
$ docker history --no-trunc apolloauto/apollo:run-x86_64-20180130_1338
$ docker history --no-trunc apolloauto/apollo:run-x86_64-20180130_1338
IMAGE CREATED CREATED BY SIZE COMMENT
sha256:495f674edec9512f68724a28f0b9919c126b1fd8068e89c7d480a679a6f3ede8 7 days ago /bin/bash 11.2MB
<missing> 2 weeks ago /bin/bash 1.59MB
<missing> 4 weeks ago /bin/bash 47.6MB
<missing> 4 weeks ago /bin/bash 47.6MB
<missing> 5 weeks ago /bin/bash 1.01MB
<missing> 5 weeks ago /bin/bash 48.9MB
<missing> 5 weeks ago /bin/bash 1.32MB
<missing> 6 weeks ago /bin/bash 11.7kB
<missing> 6 weeks ago /bin/bash 3.97MB
<missing> 6 weeks ago /bin/bash 128kB
<missing> 6 weeks ago /bin/bash 1.22MB
<missing> 7 weeks ago /bin/bash 47.6MB
<missing> 8 weeks ago /bin/bash 1.21MB
<missing> 8 weeks ago /bin/bash 29.6MB
<missing> 2 months ago /bin/bash 84.1MB
<missing> 2 months ago /bin/bash 26.7MB
<missing> 2 months ago /bin/bash 9.52MB
<missing> 2 months ago /bin/bash 4.86MB
<missing> 2 months ago /bin/bash 87.1MB
<missing> 2 months ago /bin/bash 26.4MB
<missing> 2 months ago /bin/bash 7.38MB
<missing> 2 months ago /bin/bash 65.3kB
<missing> 3 months ago /bin/bash 122kB
<missing> 3 months ago /bin/bash 842kB
<missing> 3 months ago /bin/bash 43.4MB
<missing> 3 months ago /bin/bash 17.5MB
<missing> 4 months ago /bin/bash 7.26MB
<missing> 5 months ago /bin/bash 2.71GB
<missing> 5 months ago /bin/bash 143MB
<missing> 5 months ago /bin/sh -c rm -fr /tmp/* 0B
<missing> 5 months ago /bin/sh -c ln -s /usr/lib64/libGLEW.so.2.0 /usr/lib/libGLEW.so.2.0 25B
<missing> 5 months ago /bin/sh -c ln -s /usr/lib64/libGLEW.so /usr/lib/libGLEW.so 21B
<missing> 5 months ago /bin/sh -c make && make install 8.93MB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp/glew-2.0.0 0B
<missing> 5 months ago /bin/sh -c unzip glew-2.0.0.zip 4.35MB
<missing> 5 months ago /bin/sh -c wget https://github.com/nigels-com/glew/releases/download/glew-2.0.0/glew-2.0.0.zip 1.04MB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp 0B
<missing> 5 months ago /bin/sh -c apt-get update && apt-get install -y --force-yes libglfw3 libglfw3-dev 27.2MB
<missing> 5 months ago /bin/sh -c echo "deb-src http://ppa.launchpad.net/keithw/glfw3/ubuntu trusty main" | sudo tee -a /etc/apt/sources.list.d/fillwave_ext.list 126B
<missing> 5 months ago /bin/sh -c echo "deb http://ppa.launchpad.net/keithw/glfw3/ubuntu trusty main" | sudo tee -a /etc/apt/sources.list.d/fillwave_ext.list 61B
<missing> 5 months ago /bin/sh -c ln -s -f /bin/true /usr/bin/chfn 9B
<missing> 5 months ago /bin/sh -c pip install -r py27_requirements.txt 20.7MB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp 0B
<missing> 5 months ago /bin/sh -c n 8.0.0 106MB
<missing> 5 months ago /bin/sh -c make install 14.7kB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp/n-2.1.0 0B
<missing> 5 months ago /bin/sh -c tar xzf v2.1.0.tar.gz 24.3kB
<missing> 5 months ago /bin/sh -c wget https://github.com/tj/n/archive/v2.1.0.tar.gz 7.95kB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp 0B
<missing> 5 months ago /bin/sh -c #(nop) ENV ROSCONSOLE_FORMAT=${file}:${line} ${function}() [${severity}] [${time}]: ${message} 0B
<missing> 5 months ago /bin/sh -c chmod 755 /usr/bin/protoc 185kB
<missing> 5 months ago /bin/sh -c ./configure --prefix=/usr && make && make install 896MB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp/protobuf-3.3.0 0B
<missing> 5 months ago /bin/sh -c tar xzf protobuf-cpp-3.3.0.tar.gz 24MB
<missing> 5 months ago /bin/sh -c wget https://github.com/google/protobuf/releases/download/v3.3.0/protobuf-cpp-3.3.0.tar.gz 4.22MB
<missing> 5 months ago /bin/sh -c #(nop) WORKDIR /tmp 0B
<missing> 5 months ago /bin/sh -c #(nop) COPY file:88278e7e23ec00ef5d6f153c81454d28303876054b2b7122150068a3354d7585 in /tmp/ 371B
<missing> 5 months ago /bin/sh -c rm -fr /var/lib/apt/lists/* 0B
<missing> 5 months ago /bin/sh -c apt-get clean autoclean && apt-get autoremove -y 0B
<missing> 5 months ago /bin/sh -c apt-get update && apt-get install -y build-essential curl git libatlas-base-dev libboost-all-dev libconsole-bridge-dev libcurl4-openssl-dev libflann-dev libfreetype6-dev libgflags-dev libgoogle-glog-dev libhdf5-serial-dev libicu-dev libleveldb-dev liblmdb-dev liblog4cxx10 liblz4-dev libopencv-dev libopenni-dev libpoco-dev libproj-dev libpython2.7-dev libqhull-dev libsnappy-dev libtinyxml-dev libvtk5-dev libvtk5-qt4-dev libyaml-cpp-dev libyaml-dev mpi-default-dev python-matplotlib python-pip python-scipy python-software-properties realpath software-properties-common tmux unzip wget 1.24GB
<missing> 7 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B
<missing> 7 months ago /bin/sh -c mkdir -p /run/systemd && echo 'docker' > /run/systemd/container 7B
<missing> 7 months ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/\1/g' /etc/apt/sources.list 1.9kB
<missing> 7 months ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B
<missing> 7 months ago /bin/sh -c set -xe && echo '#!/bin/sh' > /usr/sbin/policy-rc.d && echo 'exit 101' >> /usr/sbin/policy-rc.d && chmod +x /usr/sbin/policy-rc.d && dpkg-divert --local --rename --add /sbin/initctl && cp -a /usr/sbin/policy-rc.d /sbin/initctl && sed -i 's/^exit.*/exit 0/' /sbin/initctl && echo 'force-unsafe-io' > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup && echo 'DPkg::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' > /etc/apt/apt.conf.d/docker-clean && echo 'APT::Update::Post-Invoke { "rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true"; };' >> /etc/apt/apt.conf.d/docker-clean && echo 'Dir::Cache::pkgcache ""; Dir::Cache::srcpkgcache "";' >> /etc/apt/apt.conf.d/docker-clean && echo 'Acquire::Languages "none";' > /etc/apt/apt.conf.d/docker-no-languages && echo 'Acquire::GzipIndexes "true"; Acquire::CompressionTypes::Order:: "gz";' > /etc/apt/apt.conf.d/docker-gzip-indexes && echo 'Apt::AutoRemove::SuggestsImportant "false";' > /etc/apt/apt.conf.d/docker-autoremove-suggests 195kB
<missing> 7 months ago /bin/sh -c #(nop) ADD file:8c7b62577f9147353650fd7fe54e3fc119f7f39441b3b03fbba3e97e6de4a52a in / 188MB
$ docker inspect apolloauto/apollo:run-x86_64-20180130_1338
$ docker inspect apolloauto/apollo:run-x86_64-20180130_1338
[
{
"Id": "sha256:495f674edec9512f68724a28f0b9919c126b1fd8068e89c7d480a679a6f3ede8",
"RepoTags": [
"apolloauto/apollo:run-x86_64-20180130_1338"
],
"RepoDigests": [
"apolloauto/apollo@sha256:7535aaa28ffdd7cd75089b1a013387e113fe77734f2e6f956a3398fb06f7dd16"
],
"Parent": "",
"Comment": "",
"Created": "2018-01-30T21:46:23.205717014Z",
"Container": "8901691d7f0596266f3c892795da2f411d428d2db4ee496b421e0155fe6f4d6f",
"ContainerConfig": {
"Hostname": "lei",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": true,
"OpenStdin": true,
"StdinOnce": false,
"Env": [
"DISPLAY=:0",
"RELEASE_DOCKER=1",
"DOCKER_USER=apollo",
"USER=apollo",
"DOCKER_USER_ID=1000",
"DOCKER_GRP=apollo",
"DOCKER_GRP_ID=1000",
"PYTHONPATH=/apollo/lib:/apollo/ros/lib/python2.7/dist-packages",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"ROSCONSOLE_FORMAT=${file}:${line} ${function}() [${severity}] [${time}]: ${message}"
],
"Cmd": [
"/bin/bash"
],
"Image": "ea2613c9f33c",
"Volumes": null,
"WorkingDir": "/mnt",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
"DockerVersion": "17.05.0-ce",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"DISPLAY=:0",
"RELEASE_DOCKER=1",
"DOCKER_USER=apollo",
"USER=apollo",
"DOCKER_USER_ID=1000",
"DOCKER_GRP=apollo",
"DOCKER_GRP_ID=1000",
"PYTHONPATH=/apollo/lib:/apollo/ros/lib/python2.7/dist-packages",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"ROSCONSOLE_FORMAT=${file}:${line} ${function}() [${severity}] [${time}]: ${message}"
],
"Cmd": [
"/bin/bash"
],
"Image": "",
"Volumes": null,
"WorkingDir": "/mnt",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
"Architecture": "amd64",
"Os": "linux",
"Size": 5932762955,
"VirtualSize": 5932762955,
"GraphDriver": {
"Data": null,
"Name": "aufs"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:9669d6b7338336d92ca8e8e5bb90a4a35a934d1c94017322bef35a838615b4ea",
"sha256:f5cfc06b640d49f4e5b0778531394ed4819d8824c46cbc21481f5531db35d970",
"sha256:768dcfe5d05fd2355facaacaf64a45e84ac4c6b895f335ee02f322a35e912d80",
"sha256:d0757a6730d0f8242d131bdb86df046fbd5eb6e3a04328aa10e8881166101a94",
"sha256:d649a240e4537a905c43c0f9fd7bfa3f3cbe693b963e838eb6e84065b57aae3d",
"sha256:848c10056dab1694bda93ca2fb164ae104bf8f1fb8c4cb523c96c9be579888e8",
"sha256:2951daa792e2ec1724da397b50e9e691f59375c754c81fe4174d3be362413ba7",
"sha256:5f7da9782f239a520cced4f3555c1de382e2b49d184bf7dfe6e11106f634101b",
"sha256:76e59530ffa605a72f3c1e9dc9f38cffb939b2b2f34a7ea2facadaa8833c69e0",
"sha256:0ead6857ab115aae92ddad043eefb37f4fdafd893004c0e4b1f62b827684c84e",
"sha256:574d608e9adeec662867f950fa35feb0c2b996319ee49e6c1855e488a6519d3c",
"sha256:33c699a280e61123cac53582306d4f7ac31f2f97d7ef0bd04fe4dae96f29f3a0",
"sha256:7d00699f928dade7ad3185ba1c319a6f71f1bb3370bebdf5617fda9a13d148fc",
"sha256:b6ad21a13b04f87d5a156cdaf20371a0cd4c03ba38e1c1938cbff03dc08a0a36",
"sha256:6ad5128af4c596fe386c296e63a4d705f9e5014bd1413b383be0ceb1119c6100",
"sha256:200148d828c95c31750c12566664db6d2a8fea672981cb1dfdad1382eedb3868",
"sha256:90bef3f788f543346bef98f528f7b66afb31e73d7362ded87b45a245527ca9d6",
"sha256:2020c7925aa58fba6306226420c6bfe8e86afadf2b54867e1e69b2b7078271cc",
"sha256:a0152c5e3baec05744cbc650e422625d24e4e79cc4b3dc22a8e0bb7528a2ed20",
"sha256:69c9602d9478dba38b454a1201ca7cd09c78fc3182c6878c68a4a5b46c76a886",
"sha256:ea3b24f9bd445dc61ec609b292b0ca88dd85d1ffe9ad218927e5b027bf1499fb",
"sha256:4d400a886451c50520fdc60afaf80bb53a22296b3f2a54d6af4ef80d830ff14e",
"sha256:a6c2e12bc17a2a39ed061294a1e08b643c21cac4123740e6f3cd377305ee10d6",
"sha256:8fc92dd9edbf5e11977c890e173ae937ce38e2ee9a62121d7498469709553b8b",
"sha256:72cc87f4d81efdaedee48f7ed8dbfb4f4563a56eeb511fff003b03dc65f6034a",
"sha256:b023897d14d8c1975e295a5121dbfd0025f33d1d9a301eee8b3c35af15452ea7",
"sha256:3887cd4bfa5a7c314abf5481fe66b1644f084bcd201f1bff141a9f605dd4a7ee",
"sha256:33b34c3d37ed8c04246ca2b398358df30585db1d098465094ecb6592e788a4f9",
"sha256:288df758f1dfe93cd3a32f5279b72de3fc4588ec0f8e4d8d654781d7473dcf8b",
"sha256:9f60e77fee4e2919f51d796e26800a9387fe0e66f1e237b85da25a7480a804eb",
"sha256:862f29cc925faa8ffefae390fe51e16ea8cc5030225ed548809addffc777a2e5",
"sha256:ae91cdc4f39730685e464dfa2a574056df3dc1bc13b36832553c1b6f8cd18840",
"sha256:d1079845da233fa0307e7f94bc3c6f10e1a78bcb5cf60abdd65e070624920fbb",
"sha256:b9947bc173878272cf995324b9cb263bfb24eb0196e80979a94be7e0a478afd5",
"sha256:e742316cf90861e1c07b4ac615f9822ecde242b8cdcee93af0bbb16dc4795bd3",
"sha256:a298d332ebca6d92e0699e78a4b2ded468ef14876cc1a85d1d5a7e5ce53add94",
"sha256:a940048cfcd8eb57e9437d59c16a19d96da4eac9689a31eae15872847054c5c0",
"sha256:00f0657a14cac85a0b786ef3a4d194ba6d1a9c943521c43ef52dc029b565b058",
"sha256:1185fdca738d90c22d106f3df5958dbc4cc1da0a9b952c4cba0682f92e1e7563",
"sha256:335cc2e7b8637436fc39ab1e016df21200308d8be4e43fbad1248af8a3529f56",
"sha256:5193824fdb161c316195b0daf39f7eec500e565226f1e7455ed9059429ac628c",
"sha256:e3eab189d8831345de20d0621fa16bf206ef2b48ddbc161a5b1d475ea5abf858",
"sha256:c1829631cb33ba373108d3a806af7ab7260b91ed935f21e115a201b115a37dc9",
"sha256:3b7dbb00e767c99997be2d0fe7251d8a6f53634ab84b0fedf75cfcfda70631b2",
"sha256:12406657e2b6ffb4d9912b293a17b37ef33531dae45e04632b943259cdeecc93",
"sha256:70188b3df1e5988da02df55eb4efd27108ff20d90f4c8444556b0dcde2cfbecf",
"sha256:c94562323b72e465d870f83af287a80a90c89d953f2fc70312274e72125a7737",
"sha256:6d0795cc941a85306c70e70e5e6a2850dd95b9e3207eae93e05d8ce85fe3ec94",
"sha256:a3d69bb5645af83540d6cc4a6942aec9df8b63aed19151262d0fffb78f5092d7",
"sha256:296f469b9e0587cda0517fb505aa1ec01b9d68e43398d5c2bb7fd92901847ee8",
"sha256:6bf32b78899b6322c0964a3a47383543cacce41ef2a1b4a7b73977e914a41694",
"sha256:feecb99bc6009ee88d88b406ffb57a197b807d8b55d49209ff8197421d2a4ad9",
"sha256:182fd38af922ba744e5755dabcc0712665fd3326eeba81aca52f93cd491b3c4c",
"sha256:8f159d020c770433eae4121ccbd20d1e450ac3b33f0b4ab2fa571495155678ad",
"sha256:416ecdd401a9751de8125f082bd17c8bd71dce55756d8af844a433fb017a99e8",
"sha256:277c4fc88dabe16337c4e3645ff70e0cbc422fd9ba8530d60739844eb1ddc488",
"sha256:74ba767f9a543f048a9d2d1dddf109ec4e5359570cfad63788027a7866c6dd2f"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]
From a security standpoint, I would like to know the origins of the binaries other people are deploying and testing on our roads. Could these Dockerfiles be hosted under the ApolloAuto github organization somewhere? From looking at the apollo/apollo_docker.sh, is the image build proccess now sequence of scrippted docker commits on a workspace container?
ruffsl
on 7 Feb 2018
I want to amplify and agree with the comment above from @ruffsl.
slivingston
on 1 Apr 2018
Thanks for the sincere suggestion!
As you can imagine, it's really complicated to bring everything together and run it on road. The Dockerfile as well as the building process is extremely hacky and error prone. For now, we are short of bandwidth to maintain it in public and also answer piles of questions about failures in different environments.
But I agree with you, it should be transparent. We'll try to bring it back!
xiaoxq
on 1 Apr 2018
You can release the Dockerfile but write that using Docker images is currently the only method supported by Baidu engineers.
Writing that the build process is "extremely hacky and error prone" begs the question of details about the process itself, especially in an open source project. Without being able to review the process ourselves, as users, it also raises concerns about whether the provided Docker image is fragile and, thus, not ready for practical applications.
In addition to the above, my final argument is that a founding motivation of open source is to allow users to also be contributors. Give us a chance to debug the failures ourselves when we try to apply the build process in new environments.
slivingston
on 2 Apr 2018
Great point @slivingston and thank you for raising the issue. After discussing the matter internally, we will invest resources in making the issue more open and more friendly for our developer community in the future releases. We appreciate your feedback and patience with us, and please keep your good suggestions coming! :)
Zhenni17
on 2 Apr 2018
Hi @jamiehowarth @ruffsl @slivingston , an quick update on the question you asked: we released the dockerfile during Apollo 2.5 and you can see it here: https://github.com/ApolloAuto/apollo/blob/master/docker/build/dev.x86_64.dockerfile
However, this dockerfile is not up to date due to we have a major release that's coming up. So if you run this dockerfile, it will NOT produce the image that you expected, because the dockerfile is written with the future release in mind. Please stay tuned for the future releases, and we will provide more info on how to use the dockerfile. If you'd like, feel free to make changes on this dockerfile to create a image that better suited for your needs.
Zhenni17
on 5 Jun 2018
Related issues
maziqiqi
·
3Comments
JSnobody
·
3Comments
ahuer2435
·
3Comments
chilihua
·
3Comments
Wsine
·
3Comments
Most helpful comment
You can release the Dockerfile but write that using Docker images is currently the only method supported by Baidu engineers.
Writing that the build process is "extremely hacky and error prone" begs the question of details about the process itself, especially in an open source project. Without being able to review the process ourselves, as users, it also raises concerns about whether the provided Docker image is fragile and, thus, not ready for practical applications.
In addition to the above, my final argument is that a founding motivation of open source is to allow users to also be contributors. Give us a chance to debug the failures ourselves when we try to apply the build process in new environments.