Pcl: [ci] Docker image building as part of CI

[shrijitsingh99]

Context

Currently, Docker images are built manually, any updates to be performed in the images needs to be manually done by the maintainer.

Expected behavior

Docker image generation and pushing to docker hub becomes part of the CI pipeline. It can be added as the first stage which only builds and pushes if there is a change in the Dockerfile else this stage can be skipped.

Current Behavior

Currently, this docker image generation is done manually. The Dockerfile only exists for Ubuntu 16.04, for the other images isn't available on the repo.

Describe the solution you'd like

The first stage of CI is Docker image generation from the Dockerfiles in the repo.
The CI check's to see if Dockerfile was updated (using git diff) if it was it builds and pushes the new image to docker hub else the stage is skipped.

Additional context
Have already implemented some of the above-mentioned stuff here here. (CI Results)

[SergioRAgostinho]

Even if a certain PR requires updating the docker image, you don't really want to push that update until the PR is merged to master. With that in mind, I believe we need a pipeline which behaves more like this:

• PRs: if there are changes in the image, don't pull it from dockerhub. Build it locally and keep it cached to be used in the followup build stages.
• Merges to master: if there changes in the image script, build and deploy to dockerhub, then run tests normally by pulling from docker hub.

[shrijitsingh99]

Even if a certain PR requires updating the docker image, you don't really want to push that update until the PR is merged to master. We that in mind, I believe we need a pipeline which behaves more like this:

• PRs: if there are changes in the image, don't pull it from dockerhub. Build it locally and keep it cached to be used in the followup build stages.
• Merges to master: if there changes in the image script, build and deploy to dockerhub, then run tests normally by pulling from docker hub.

Yeah, this functionality is certainly needed.

Merge to master one is pretty straightforward and can be easily done with a seperate trigger.

Not sure if using local Docker images is possible without increasing the complexity of CI by a lot. A simpler solution might be:
Push image with temporary tag.
Pull that temporary image and build.
Last stage, delete temporary image fro docker hub.
Any issues with this?

[kunaltyagi]

if using local Docker images is possible

Yes, definitely. You can have a separate build stage and then use that tag in next step. I don't know if this works cross-stage too.

The push can be conditional on it being the master branch.

[taketwo]

The Dockerfile only exists for Ubuntu 16.04, for the other images isn't available on the repo.

I assume you are referring to the env image. Its Dockerfile has a build argument to specify the desired Ubuntu version.

---

Guys, unless there are aspiring DevOps engineers among us who want to prove something to the world, I'd propose to keep things simple and redirect the energy into something more worthy (e.g. stages). I'm not bothered to occasionally rebuild an image or two and push it to Dockerhub. We update them very seldom. Also, I'm not the only admin of PCL account at Dockerhub, so if something :crown: happens to me, we are not deadlocked.

[shrijitsingh99]

if using local Docker images is possible

Yes, definitely. You can have a separate build stage and then use that tag in next step. I don't know if this works cross-stage too.

Won't this require pushing that tag to docker hub first. Since Azure automatically fetches from it. If we go ahead with using local images and not from hub, we won't be able to use Azure's ' container feature. We would have to manually create and run the container.

[shrijitsingh99]

Guys, unless there are aspiring DevOps engineers among us who want to prove something to the world

Will still keep this issue open though, if someone wants to take it up later.

[stale[bot]]

Marking this as stale due to 30 days of inactivity. It will be closed in 7 days if no further activity occurs.

[kunaltyagi]

Update: env docker images are auto-built. Others, not so much.