Harbor: Clarification of OCI image support

[Logibox]

I was wondering if someone could clarify gohabor's position on OCI images? The builtin registry claims support for it, but the surrounding tools don't seem to work which leads to issues like #7897. Thanks!

[reasonerjt]

Some of Harbor's APIs are not fully OCI compatible.
It's known issue and we will refactor in future to fully support OCI registry.

[xaleeks]

@Logibox can you share your use case and curiosity around this ? We're investigating and looking for customer usage cases to help hash out the requirement

[Logibox]

The basic problem right now is that the harbor registry advertises it self as OCI compliant (via the /v2/ registry interface) to tools like podman and skopeo, despite not supporting the format.

The tools automatically choose the format of the container based on what the remote registry supports, but for harbor I have to explicitly set the format to docker v2.

The upshot is harbor is a bit of pain to work with if you're not using docker.

[nlowe]

I think I just ran into this with one of my developers trying to push to harbor from concourse. They're using vito/oci-build-task to build the image and concourse/registry-image-resource. The image builds and pushes fine, however:

• If this is the only tag, the repo doesn't show up in the project's repo listing
• If I fiddle with the URL I can browse to the tag directly in the API. From here, I can
  
  
  
  • Scan the tag with clair
  
  
  • Re-tag it, causing the new tag to show up in the UI
  
  
  • Delete the tag
  
  
• Users can still docker pull this image

Running a docker import against the tarball that the oci-build-task image spits out and pushing that causes the tag to show up correctly in harbor's listing. docker save'ing the imported image and using the registry-image-resource task to push the converted tarball causes the tag to show up correctly in harbor's listing, so I don't think the problem lies in either of these tools.

[xaleeks]

is that concourse pipeline still up? If so, can we use it to run some tests? @nlowe

[nlowe]

They've changed their pipeline to work around this, and it's an internal only instance that we can't expose publicly. You can however run vito/oci-build-task yourself to make an OCI Image, without setting up concourse:

$ mkdir -p image cache build
$ touch Dockerfile # Any small dockerfile
$ docker run -it --rm --entrypoint /bin/ash -v "$(pwd):/w" -w /w --cap-add=SYS_ADMIN vito/oci-build-task
/w # echo '{"config":{"debug":true},"response_path":"/w/build/output.json"}' | /usr/bin/task
DEBU[0000] building                                      buildctl-args="[build --progress plain --frontend dockerfile.v0 --local context=. --local dockerfile=. --opt filename=Dockerfile --output type=docker,dest=/w/image/image.tar --export-cache type=local,mode=min,dest=/w/cache --import-cache type=local,src=/w/cache]"
#3 [internal] load .dockerignore
#3 ...

#1 importing cache manifest from 21asf6tg5kjxbvc86lnvioye5
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 116B 0.0s done
#2 DONE 0.1s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.1s

#4 [internal] load metadata for docker.io/library/alpine:latest
#4 DONE 0.8s

#5 [1/2] FROM docker.io/library/alpine@sha256:72c42ed48c3a2db31b7dafe17d275...
#5 resolve docker.io/library/alpine@sha256:72c42ed48c3a2db31b7dafe17d275b634664a708d901ec9fd57b1529280f01fb done
#5 DONE 0.0s

#8 importing cache manifest from 47lctu7ddcrt1ivfaz3n49ggz
#8 DONE 0.0s

#6 [internal] load build context
#6 transferring context: 104B 0.0s done
#6 DONE 0.1s

#7 [2/2] COPY entrypoint.sh /usr/bin/foo
#7 pulling sha256:e1f858479bee4f508a08828a814b5e970fc4050f0f25b72c55f440d5a0e013b4 0.1s done
#7 pulling sha256:9d48c3bd43c520dc2784e868a780e976b207cbf493eaff8c6596eb871cbd9609
#7 pulling sha256:9d48c3bd43c520dc2784e868a780e976b207cbf493eaff8c6596eb871cbd9609 0.1s done
#7 unpacking
#7 unpacking 0.3s done
#7 CACHED

#9 exporting to oci image format
#9 exporting layers done
#9 exporting manifest sha256:690d38b9c5e529aa4a21a2a2078692fa8d973758afcce87f924ae0e09392b290 0.0s done
#9 exporting config sha256:640d611749a476dd053be51c348ce58ae6264d8e39189a85a69779ed5fc61fd7 0.0s done
#9 sending tarball
#9 sending tarball 6.2s done
#9 DONE 6.3s

#10 exporting cache
#10 preparing build cache for export done
#10 writing layer sha256:9d48c3bd43c520dc2784e868a780e976b207cbf493eaff8c6596eb871cbd9609 done
#10 writing layer sha256:e1f858479bee4f508a08828a814b5e970fc4050f0f25b72c55f440d5a0e013b4 done
#10 writing config sha256:de649f7e137c3b3a09a7e50da4a140d949d044f738d4cd5ec0d7e3e7c480e898 done
#10 writing manifest sha256:5aa0b95af000b50adf7303017c9fc9e0f5de71ca6e9164ce4bca77c5a961bafb done
#10 DONE 0.0s

And then try to push:

$ docker run -it --rm --entrypoint /bin/ash -v "$(pwd):/w" -w /w concourse/registry-image-resource
/w # echo '{"source":{"repository":"harbor.mydomain.net/nlowe/registry-image-resource", "tag":"test", "username": "user", "password": "password", "debug":true},"params":{"image":"image.exported.tar"}}' | /opt/resource/out ./image
INFO[0000] pushing sha256:8561912e317e7c189639c011e6bcf7112cc177a59154ddfb88b10be48c7fbde2 to harbor.mydomain.net/nlowe/registry-image-resource:test
2019/09/17 17:15:47 existing blob: sha256:1959c0dbf323eadb384e5bf91db2a37a9322d4d088c0b4ec9a7dc55617028b8e
2019/09/17 17:15:47 existing blob: sha256:cc903a12356892a310d083ee86833e710949c3188adf9c884f44978fa8793c4b
2019/09/17 17:15:48 harbor.mydomain.net/nlowe/registry-image-resource:test: digest: sha256:8561912e317e7c189639c011e6bcf7112cc177a59154ddfb88b10be48c7fbde2 size: 427
INFO[0002] pushed
{"version":{"digest":"sha256:8561912e317e7c189639c011e6bcf7112cc177a59154ddfb88b10be48c7fbde2"},"metadata":[{"name":"repository","value":"harbor.mydomain.net/nlowe/registry-image-resource"},{"name":"tags","value":"test"}]}

[xaleeks]

The basic problem right now is that the harbor registry advertises it self as OCI compliant (via the /v2/ registry interface) to tools like podman and skopeo, despite not supporting the format.

The tools automatically choose the format of the container based on what the remote registry supports, but for harbor I have to explicitly set the format to docker v2.

The upshot is harbor is a bit of pain to work with if you're not using docker.

@Logibox Hi, I'm curious where Harbor advertises itself as OCI compliant because as of this time it is not fully compliant. Can you describe this in greater detail so we can rectify this

[Logibox]

@xaleeks - Maybe 'advertises' isn't the correct word. I can't speak to how other container builders works but for tools in the containers org (podman, buildah, skopeo), the tools work by repeatedly attempting uploads to the registry in different formats until the registry accepts the image. It's up to the registry to explicitly reject manifest types it does not support.

See github.com/containers/image/copy/copy.go.

For goharbor the service behind /v2 will accept OCI images, but the rest of harbor doesn't know what to do with them. I don't know where goharbor/registry-photon comes from but I assume it's a modified version of the docker registry, correct?

[xaleeks]

Ok to be clear, we don't claim to be compliant with OCI Images (or advertise anywhere) yet but it is in the works. So just because user can upload these images initially without error before they are eventually blocked on UI doesn't mean harbor 'partially supports' it, we just don't have a floodgate or some kind of mechanism to check the types right now. We also have not vetted against Podman and buildah so I don't know what exact kind of images they are producing in terms of their schema types. I know Podman is a daemon-less approach that spits out OCI but is untested for us.

So official stance is this is WIP but don't be surprised if they fail to show up on web console as of the latest release. I'm interested in chatting about the tooling around OCI Images so feel free to drop me a message: [email protected]

[xaleeks]

@Logibox Harbor will support all OCI artifacts including manifest lists on the upcoming 2.0 release. That means if tooling like podman are generating OCI-compliant images, they should be able to be pushed onto Harbor.

[Logibox]

Thanks for the update @xaleeks - looking forward to trying it out!

[xaleeks]

@Logibox We are nearing the release which will have this capability, let me know if you are interested in helping us test OCI images generated with podman.

[xaleeks]

Check latest release for this. We are now the first open source registry to fully support OCI :)

[renmaosheng]

@Logibox have you got any chance try the new v2.0 version? your feedbacks are always welcome. thanks.

[reasonerjt]

I'm closing this as Harbor now supports OCI images, if you see any problems please open new issue so we can fix.