Pipeline: x509 certificate signed by unknown authority

Created on 6 Aug 2019 · 12Comments · Source: tektoncd/pipeline

Expected Behavior

I am trying to run a container from my private registry within Tekton. I would like Tekton to invoke the entrypoint defined in the container rather than specify a command. Per, https://github.com/tektoncd/pipeline/blob/master/docs/container-contract.md I have added an ImagePullSecret to the service account running my pipeline.

I would expect Tekton to be able to pull my image, read the entrypoint and execute that in the pipeline.

Actual Behavior

In reality I am getting a certificate error from the entrypoint binary that is inserted into the execution.

Failed to create build pod "[my-pod-name]": couldn't create redirected TaskSpec: failed to add entrypoint to steps of TaskRun [my-pod-name]: Failed to fetch remote image [myprivateregistry.com]/pierretasci/tests:latest: Failed to get container image info from registry [myprivateregistry.com]/pierretasci/tests:latest: Get https://[myprivateregistry.com]/v2/: x509: certificate signed by unknown authority

The weird thing is that if I specify the command in the entrypoint, then this works. It is able to pull the image and execute it. It is only within the task execution when I don't specify a command that this happens.

Steps to Reproduce the Problem

Use a private docker repo like Artifactory
Create a k8s image pull secret for this and add it to your service account
Define a pipeline with a task that does not specify a command and references a container from the private docker

kinbug

Source

pierretasci

👍1

Most helpful comment

I hit the same problem in my use case，any process for this issue?

hougangliu on 5 Nov 2019

👍3

All 12 comments

@pierretasci Thanks for the issue. This is one pitfall of our entrypoint dark magic… WIthout a command, pipeline tries to get the image entrypoint from the registry, and, in your case fails because of certificates.

We need to find a way to use k8s image pull secret in pipeline (or do that in a separate container that would have this secret) when we try to fetch the image config from the registry. We will also need a way for the user to give that information (although if it's attach to the serviceaccount, we can manage)

/kind bug

vdemeester on 7 Aug 2019

Yeah that is what I figured was going on so I am glad to see that I'm not crazy 😄

Pardon my naivete, but is an image pull secret enough here? If the issue is that the image running the entrypoint "magic" doesn't trust the self-signed cert from my registry, being able to login to the registry wouldn't be enough. Is that about right?

pierretasci on 7 Aug 2019

@pierretasci it should be enough, but it's the pipeline controller itself that also needs it (to get the image config)…

vdemeester on 7 Aug 2019

I hit the same problem in my use case，any process for this issue?

hougangliu on 5 Nov 2019

👍3

I was able to workaround this based on the suggestion here: https://github.com/google/ko/issues/142#issuecomment-598837434, but I don't think the process would scale easily. It could be nice to have some kind of --skip-tls-verify option, like what can be used in the kaniko image to push to a private registry with a self-signed certificate.

Some details about the workaround: I copied the registry's self-signed certificate into a configmap in the tekton-pipelines namespace, and mounted that into the tekton-pipelines-controller deployment. Then I just needed to add the SSL_CERT_FILE environment variable pointing to that file. Basically these were the additions to the controller deployment:

spec:
  template:
    spec:
      containers:
        env:
        - name: SSL_CERT_FILE
          value: /etc/registry-cert/tls.crt
...
        volumeMounts:
        - mountPath: /etc/registry-cert
          name: registry-cert
...
      volumes:
      - configMap:
          name: registry-cert
        name: registry-cert

JustinKuli on 15 Mar 2020

👍1

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

tekton-robot on 13 Aug 2020

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

tekton-robot on 13 Aug 2020

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

tekton-robot on 13 Aug 2020

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot on 13 Aug 2020

/reopen
/remove-lifecycle rotten