Cirrus-ci-docs: Share container between tasks

Created on 14 Jul 2019 · 13Comments · Source: cirruslabs/cirrus-ci-docs

I'd like to build Docker image in one step, then test it, and if it is successful, upload it from a separate task for security reasons.

Right now I use the following config, which is insecure against 3rd party pull requests.

docker_builder:
  env:
    TAG: yakshaveinc/snapcraft:core18-edge
    DOCKER_USERNAME: ENCRYPTED[89bc5b18538c2260d2840aa63a69b21ea4f247085793e6ddfc2270629b4f615b684dfca53a5f5debcabbe1b4f6d8b965]
    DOCKER_PASSWORD: ENCRYPTED[5d5ec522913fd2c5b6dda34b044cdb55f01aae995150a98eba8a040a83126eb2e4b3e76209316aa3fe0af7ef8e4388ef]
  clone_script: git clone --depth=100 https://github.com/snapcore/snapcraft .
  # rewrite xenial (16.04) to bionic (18.04), which is also core18 base
  rewrite_script: sed -i 's/xenial/bionic/g' docker/edge.Dockerfile && echo docker/edge.Dockerfile
  build_script: docker build -t $TAG -f docker/edge.Dockerfile docker
  snap_script: docker run $TAG snapcraft --version
  login_script: docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
push_script: docker push $TAG

I want to run docker_builder for testing PRs, but use docker login only when PR is merged, and it should be a separate task with environment variables inaccessible from PRs. Is it possible to share built docker container with subsequent separate deploy task?

question

Source

abitrolly

All 13 comments

Hey @abitrolly, as I mentioned in your testing PR, using this config is secure since only builds from users with write permissions to the repository will have secure variable decrypted. A third party user with only read permissions (since I assume it's a public repository) won't see DOCKER_USERNAME and DOCKER_PASSWORD when they create a PR from their fork.

You can simply check if DOCKER_USERNAME and DOCKER_PASSWORD are presented in order to login and push. For example, see how we build a container for https://cirrus-ci.com: https://github.com/cirruslabs/cirrus-ci-web/blob/master/.ci/push_docker.sh

fkorotkov on 14 Jul 2019

👍1

Let me check.

abitrolly on 14 Jul 2019

@fkorotkov Yes, the variable was not decrypted. I am safe at last. However, the question is still the same - is it possible to share container between tasks? Or other artifacts?

abitrolly on 14 Jul 2019

Definitely! You can use CIRRUS_CHANGE_IN_REPO environment variable as a tag for your container and then use it in tasks that depend on the Docker Builder task. Does it help with your use case?

fkorotkov on 15 Jul 2019

No. I don't see how it is going to work. If I get this right, docker_deploy below won't be able to run docker.

docker_builder:
  env:
    TAG: yakshaveinc/snapcraft:core18-edge
    DOCKER_USERNAME: ENCRYPTED[89bc5b18538c2260d2840aa63a69b21ea4f247085793e6ddfc2270629b4f615b684dfca53a5f5debcabbe1b4f6d8b965]
    DOCKER_PASSWORD: ENCRYPTED[5d5ec522913fd2c5b6dda34b044cdb55f01aae995150a98eba8a040a83126eb2e4b3e76209316aa3fe0af7ef8e4388ef]
  clone_script: git clone --depth=100 https://github.com/snapcore/snapcraft .
  # rewrite xenial (16.04) to bionic (18.04), which is also core18 base
  rewrite_script: sed -i 's/xenial/bionic/g' docker/edge.Dockerfile && echo docker/edge.Dockerfile
  build_script: docker build -t $TAG -f docker/edge.Dockerfile docker
  snap_script: docker run $TAG snapcraft --version

docker_deploy:
  depends_on:
    - docker_builder
  login_script: docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
  push_script: docker push $TAG

abitrolly on 15 Jul 2019

Right, this won't work. Do you understand you correctly that you want:

Build and image and not push it in the first task.
You want to test the container in the second task.
If all of the above works, finally push the image in the third task?

fkorotkov on 15 Jul 2019

👍1

@fkorotkov that's correct.

abitrolly on 15 Jul 2019

@abitrolly ideally you can use some registry for such testing so you would be able to use a container you build with container instance. Config will look something like this:

env:
  DOCKER_USERNAME: ENCRYPTED[qwerty1]
  DOCKER_PASSWORD: ENCRYPTED[qwerty2]

docker_builder:
  name: Build
  build_script: docker build --tag my-image:$CIRRUS_CHANGE_IN_REPO .
  login_script: docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
  push_script: docker push my-staging-image:$CIRRUS_CHANGE_IN_REPO

task:
  name: Verify
  depends_on: Build
  container:
    image: my-staging-image:$CIRRUS_CHANGE_IN_REPO
  # script to verify the image

docker_builder:
  name: Push
  depends_on: Verify
  pull_script: docker pull my-staging-image:$CIRRUS_CHANGE_IN_REPO
  tag_script: 
    - docker tag my-staging-image:$CIRRUS_CHANGE_IN_REPO my-image:$CIRRUS_CHANGE_IN_REPO
    - docker tag my-image:$CIRRUS_CHANGE_IN_REPO my-image:latest
  login_script: docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD
  push_script: 
    - docker push my-image:$CIRRUS_CHANGE_IN_REPO 
    - docker push my-image:latest

But the example above won't work for external contributors since they won't have access to Docker registry.

Can I ask you what are you going to test on the second test? Why can't you do all three steps in one task.

Another idea might be to take advantages of docker save and docker load to pass around an archive of the docker image. But that sounds like a workaround rather than a solution. 🤔

fkorotkov on 15 Jul 2019

No specific tests. I want to leave security choices and logic visible at the CI config level. This is how I'd ideally design pipelines as a part of my DevOps consulting.

External registry most likely will be slow, and the setup is cumbersome. I think I've got my answer that sharing files between tasks and containers is impossible without a third-party service. It is also nice to see that docker_builder steps can be repeated, which is missing from documentation.

abitrolly on 19 Jul 2019

Another idea might be to take advantages of docker save and docker load to pass around an archive of the docker image. But that sounds like a workaround rather than a solution.

What is the easiest fast way to pass it between tasks?

abitrolly on 19 Jul 2019