Kind: Kind in GitHub Actions

[tamalsaha]

We are trying to use Kind to e2e test on GitHub actions. We need local storage privisioner to work in GitHub actions with Kind. We have been trying to use @radu-matei 's https://twitter.com/matei_radu/status/1165024478746894336 .

Has anyone sucessfully dynamically provisioned local storage on GitHub actions?

cc: @shudipta

[shudipta]

We have tried the following action:

name: Go
on: [push]
jobs:

  build:
    name: Build
    runs-on: ubuntu-latest
    steps:

    - name: Set up Go 1.12
      uses: actions/setup-go@v1
      with:
        go-version: 1.12
      id: go

    - name: Check out code into the Go module directory
      uses: actions/checkout@v1

    - name: Build
      run: make ci

    - uses: engineerd/[email protected]

    - name: Testing
      run: |
        export KUBECONFIG="$(kind get kubeconfig-path)"
        kubectl cluster-info
        kubectl get pods -n kube-system
        export REGISTRY=<registry>
        make install
        make e2e-tests

But, the tests are failing because in the container no data can be written.

[radu-matei]

^^ Do you happen to have the repo this is trying to test?
What is happening in make e2e-tests?

Could you please show the error?
Also, have you checked #118?

[tamalsaha]

We have all the files in this demo project.
https://github.com/tamalsaha/github-action-demo

We are trying both the rancher local-path provisioner and @rimusz 's hostpath provisioner. For both of them, we changed the installer so that provisioner uses /github/home as the writable dir. ref:
https://help.github.com/en/articles/virtual-environments-for-github-actions#filesystems-on-github-hosted-machines

Rancher provisioner is working ok.

With Rimusz's host-path provisioner we are able to R/W files but for some reason the container is crash looping. I am not able to figure out what is the problem yet. You can see the error here: https://github.com/tamalsaha/github-action-demo/runs/233912859#step:6:116

We are yet to explore the case of writing using non-root user. Our real test case creates DB pods which uses non-root user. It seems that GitHub does not allow non-root users to write to /github/workspace dir. So, we are using the /github/home for the provisioner.

/github/workspace - Note: GitHub Actions must be run by the default Docker user (root). Ensure your Dockerfile does not set the USER instruction, otherwise you will not be able to access GITHUB_WORKSPACE.

[rimusz]

@tamalsaha just use the Rancher provisioner one :)

[radu-matei]

Thanks for the detailed description, @tamalsaha - I'll investigate the hostpath provisioner some more.

From what I understand, the Rancher provisioner is working fine on the GH workers - let me know if you want to contribute a doc fix on the action explaining how to use it (otherwise, I can update it).

Thanks a lot!

[tamalsaha]

We are going with Rancher provisioner. We are still struggling with getting the non-root user permission issue. Once (if?) we can get it working, we open a docs pr.

[tamalsaha]

I am now able to run a Docker container with root and non-root user inside a KIND cluster on GitHub actions. But something is not as it should be, IMHO. Let me explain.

According to GitHub docs, the ubuntu-18.04 VMs has 14G SSD disk. https://help.github.com/en/articles/virtual-environments-for-github-actions#supported-virtual-environments-and-hardware-resources

I can see that in my build log.
https://github.com/tamalsaha/github-action-demo/commit/ea2c841df8ef2a95fd39fb372a1f55ebc368eda9/checks#step:4:14

# lsblk
sdb       8:16   0   14G  0 disk 
└─sdb1    8:17   0   14G  0 part /mnt

# cat /proc/mounts
/dev/sdb1 /mnt ext4 rw,relatime 0 0

When I am using the Rancher provisioner, I am trying to use /mnt on the node as the directory for local path.
https://github.com/tamalsaha/github-action-demo/blob/ea2c841df8ef2a95fd39fb372a1f55ebc368eda9/hack/github/rancher-provisioner.yaml#L101

But inside the pod, I see that /data folder is a overlay disk instead of etx4.

• https://github.com/tamalsaha/github-action-demo/blob/ea2c841df8ef2a95fd39fb372a1f55ebc368eda9/hack/github/rancher-provisioner-nonroot.yaml#L35

• https://github.com/tamalsaha/github-action-demo/commit/ea2c841df8ef2a95fd39fb372a1f55ebc368eda9/checks#step:6:242

# cat /proc/mounts inside pod
overlay /data overlay 

If I understand it correctly, then in ideal setup we want the /mnt dir on the GitHub action host to be eventually used to provision a localpath volume in the pod. But that is not happening. My question is, is there a way to tell KIND to mount some host dir inside the Kind node so that it can eventually be used as a hostpath for a pod?

If I am missing something, please let me know. I am new to Kind and there are too many layers of indirection.

[aojea]

If I understand it correctly, then in ideal setup we want the /mnt dir on the GitHub action host to be eventually used to provision a localpath volume in the pod. But that is not happening. My question is, is there a way to tell KIND to mount some host dir inside the Kind node so that it can eventually be used as a hostpath for a pod?

you can use extraMounts with kind for that

nodes:
- role: control-plane
  extraMounts:
    - containerPath: /container/path
      hostPath: /host/path
      readOnly: false

[tamalsaha]

@aojea , where do I pass this yaml?

[aojea]

@aojea , where do I pass this yaml?

is the kind config file you use to configure your cluster kind create cluter --config kind-config.yaml

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker
  extraMounts:
    - containerPath: /container/path
      hostPath: /host/path
      readOnly: false

[radu-matei]

^^ You can have this in your repo, and reference it in your GH Action:

    - uses: engineerd/[email protected]
        with:
            config: "<path-to-config-relative-to-repo-root>"        

[tamalsaha]

It worked! https://github.com/tamalsaha/github-action-demo/commit/b1b93b230c913ca45b7246c0eb71e8adf37730d4/checks#step:6:242

Thank you @radu-matei , @aojea!

[aojea]

/close

glad it worked

[k8s-ci-robot]

@aojea: Closing this issue.

In response to this:

/close

glad it worked

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.