Cirrus-ci-docs: Run shell-less images via `command` and `args`.

[geekflyer]

I'm trying to run docker builds using kaniko on cirrus-ci.

Technically this should work using a task like this:

KanikoBuildAndPush_task:
  gke_container:
    image: gcr.io/kaniko-project/executor:latest 
  env:
    GOOGLE_CREDENTIALS: ENCRYPTED[!abcd1234]
    # we're going to write the GCR credentials into this path
    GOOGLE_APPLICATION_CREDENTIALS: /$CIRRUS_WORKING_DIR/gcr_credentials.json
    APP: myapp
  InjectGCRCredentials_script: echo $GOOGLE_CREDENTIALS > $GOOGLE_APPLICATION_CREDENTIALS  
  BuildAndPush_script:
    - /kaniko/executor
      --cache=true
      --context=dir:/$CIRRUS_WORKING_DIR
      --destination=gcr.io/my-project/$APP:$CIRRUS_CHANGE_IN_REPO

However, because cirrus-ci attempts to run all scripts via /bin/sh, all scripts fail with Error starting command: fork/exec /bin/sh: no such file or directory .
The issue here is that the kaniko image is build FROM scratch and a pretty naked image that does not even have any shell.
However if cirrus-ci would offer the ability to simply reuse the ENTRYPOINT (docker-term)/command(k8s-term) and pass in args (or optionally also override command) etc. cirrus-ci should work even with such shell-less images.

While running kaniko is personally not a super high priority for me (due to better performance I'm more inclined towards makisu), it might be a good option to run docker builds on the community cluster.
Furthermore I can imagine that there are other images / scenarios that would benefit from shell-less image support. Especially when using pipe builds it is not unlikely to run steps in a shell-less minimal image.

I was able to workaround the issue for now but by building a custom image based off alpine which borrows from kaniko fyi:

FROM gcr.io/kaniko-project/executor:latest

FROM alpine

COPY --from=0 /kaniko /kaniko
COPY --from=0 /workspace /workspace

ENV SSL_CERT_DIR=/kaniko/ssl/certs \
    DOCKER_CONFIG=/kaniko/.docker/ \
    DOCKER_CREDENTIAL_GCR_CONFIG=/kaniko/.config/gcloud/docker_credential_gcr_config.json \
    PATH="${PATH}:/kaniko"

ENTRYPOINT ["executor"]

The regular gcr.io/makisu-project/makisu makisu image btw suffers from the same problem (very minimal and based off FROM scratch). However unlike kaniko the maintainers of makisu also publish an (undocumented) gcr.io/makisu-project/makisu-alpine image which kinda works.

[fkorotkov]

You can try to use some yet-undocumented features 😅You can directly execute a one-liner by setting CIRRUS_SHELL to direct and you can also use a file instruction to create a file from en environment variable. For example, here is a working example of a config for makisu:

makisu_push_task:
  environment:
    DOCKER_CONFIG: ENCRYPTED[qwerty]
  env:
    CIRRUS_SHELL: direct
    CIRRUS_WORKING_DIR: /makisu-context
  container:
    image: gcr.io/makisu-project/makisu:v0.1.11
  docker_config_file:
    path: /root/.docker/config
    variable_name: DOCKER_CONFIG
  build_script:
    /makisu-internal/makisu build
      --http-cache-addr="http://$CIRRUS_HTTP_CACHE_HOST"
      --registry-config=/root/.docker/config
      --tag cirrusci/web-front-end:makisu
      --commit=explicit
      --modifyfs=true
      --push=index.docker.io
      /makisu-context

[fkorotkov]

FYI kaniko has a debug image with busybox shell as well.

[geekflyer]

Cool, time to add this to the docs then :)
Seems you've tried out makisu as well. Any thoughts so far?
It's pretty fast and the distributed caching is pretty useful but there are some awkward bugs. For example did you experience this one as well? https://github.com/uber/makisu/issues/247

This is another weird one: https://github.com/uber/makisu/issues/246

[fkorotkov]

@geekflyer I faced a caching issue for multi-staged builds: https://github.com/uber/makisu/issues/234 So don't have that much of an opinion but would love to have as an example in the docs for simple, cacheable and super fast Docker builds.

[geekflyer]

I just tried to use the CIRRUS_SHELL: direct feature but it doesn't quite work the way I'd like it to in my case. 🤔 .

NotifySlack_task:
  depends_on:
    - PromoteImageToProd
  gke_container:
    image: cloudposse/slack-notifier:6b458991c6446d713beab04dcd6b6659568a087c
  env:
    CIRRUS_SHELL: direct
    SLACK_WEBHOOK_URL: ENCRYPTED[abcd]
    SLACK_TEXT: "A new image was sucessfully built + promoted: https://github.com/solvvy/my-project/commit/${CIRRUS_CHANGE_IN_REPO}"

It seems like cirrus is still not reusing the entrypoint defined in the image.
In this particular example I don't want to pass in a script or anything. All the required args are passed as env vars to the entrypoint.

[fkorotkov]

Yeah. At the moment you'll need to duplicate ENTRYPOINT or CMD. Cirrus agent leaves inside the container and runs the command so it can stream logs and an exit code back.

[fkorotkov]

Worth a separate issue so if there are no instructions are provided then Cirrus should run clone and entrypoint.

[geekflyer]

actually leaving this open until the docs are updated ;-)