Pipeline: Pipelines not working with Openshift 4.5 internal registry

[abudavis]

Hi
Tried to create the following Buildah task in which it is not able to pull the image from Openshift 4.5 internal registry. I have now tried with "image-registry.openshift-image-registry.svc" and this "default-route-openshift-image-registry.app.testing.oc.company.com". Also tried to add the pull secret and its token to service account "pipeline" (which was created by default by the Openshift Pipeline operator)

$ oc get sa pipeline -o yaml
apiVersion: v1
imagePullSecrets:
- name: pipeline-dockercfg-v2w68
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-09-01T21:50:00Z"
  name: pipeline
  namespace: app
  resourceVersion: "31842524"
  selfLink: /api/v1/namespaces/app/serviceaccounts/pipeline
  uid: 1e283908-0523-460b-bf2b-bc257faa6bc7
secrets:
- name: deployer-token-8phfl
- name: deployer-dockercfg-zmt64
- name: pipeline-token-jf5dh
- name: pipeline-dockercfg-v2w68
- name: gitreposecret

Docker file: "app1.Dockerfile"
FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest

How the image looks on Openshift:

$ oc get is 
NAME                         IMAGE REPOSITORY                                                                             TAGS     UPDATED
app1                     default-route-openshift-image-registry.app.testing.oc.company.com/app/app1                     latest   14 hours ago

Error on Buildah task:

+ df -h /var/lib/containers
Filesystem Size Used Avail Use% Mounted on
/dev/sdd 30G 45M 30G 1% /var/lib/containers
+ buildah --storage-driver vfs bud -f /workspace/sourcerepos/pipeline/dockerfiles/app1.Dockerfile -t image-registry.openshift-image-registry.svc:5000/app/app1-timer:latest /workspace/sourcerepos/artefacts
STEP 1: FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest
error creating build container: Error initializing source docker://image-registry.openshift-image-registry.svc:5000/app/app1:latest: error pinging docker registry image-registry.openshift-image-registry.svc:5000: Get "https://image-registry.openshift-image-registry.svc:5000/v2/": x509: certificate signed by unknown authority

Buildah task:

apiVersion: tekton.dev/v1beta1
  kind: Task
  metadata:
    name: bake-image
  spec:
    params:
      - name: imageTag
        type: string
        description: The image tag
      - name: imageName
        type: string
        description: The image name
      - name: dockerfile
        type: string
        description: The dockerfile
      - name: contextPath
        type: string
        description: The context to run the commands
      - name: pvc
        type: string
        description: The PVC to use for layers
    workspaces:
      - name: sourcerepos
    steps:
      - name: build
        image: quay.io/buildah/stable:latest
        securityContext:
          privileged: true
        script: |
          df -h /var/lib/containers
          buildah --storage-driver vfs bud \
            -f /workspace/sourcerepos/pipeline/dockerfiles/$(params.dockerfile) \
            -t image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag) \
            $(params.contextPath)
          buildah --storage-driver vfs push \
            --tls-verify=false \
            image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag)
          df -h /var/lib/containers
        volumeMounts:
           - mountPath: /var/lib/containers
             name: varlibcontainers
        resources:
          requests:
            memory: "4Gi"
            cpu: "1"
            ephemeral-storage: "1Gi"
          limits:
            ephemeral-storage: "1Gi"
    volumes:
      - name: varlibcontainers
        persistentVolumeClaim:
          claimName: $(params.pvc)

[vdemeester]

@abudavis this is an openshift pipeline bugs (so might be worth creating a bugzilla for it on the redhat bugzilla).

/cc @chmouel @gabemontero

[abudavis]

Bugzilla : https://bugzilla.redhat.com/show_bug.cgi?id=1879680

[adambkaplan]

Looks like you have tls verification enabled for the buildah bud command, and the container running in the Tekton task does not trust the internal registry.

Try using for the buildah bud command:

buildah --storage-driver vfs bud --tls-verify false \
...

[abudavis]

Thank you, that worked :)
buildah --storage-driver vfs bud \
--tls-verify=false \

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/close

[tekton-robot]

@vdemeester: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.