Harbor: Harbor API broken for Docker Manifest List

[Richard-Payne]

If you are reporting a problem, please make sure the following information are provided:

Expected behavior and actual behavior:
Expected: Return json object representing the image.
Actual: Receive 404

Steps to reproduce the problem:
On windows 1803 system:
docker build . -t myrepo/project/app:version-Windows1803
docker push myrepo/project/app:version-Windows1803

On windows 1809 system:
docker build . -t myrepo/project/app:version-Windows1809
docker push myrepo/project/app:version-Windows1809

On any other system, with Docker CLI experimental turned on:
docker manifest create myrepo/project/app:version myrepo/project/app:version-Windows1803 myrepo/project/app:version-Windows1809
docker manifest push myrepo/project/app:version

At this point, you can docker pull myrepo/project/app:version successful and the repository will return the correct version for the host OS.

However, running this: curl https://myrepo/api/repositories/project/app/tags/version
Returns: resource: project/app:version not found

Listing all tags by: curl https://myrepo/api/repositories/project/app/tags
Returns:

{
    "digest": "",
    "name": "version",
    "size": 0,
    "architecture": "",
    "os": "",
    "docker_version": "",
    "author": "",
    "created": "0001-01-01T00:00:00Z",
    "config": null,
    "signature": null,
    "labels": []
},
{
    "digest": "sha256:62e90528dbb490d127e5f33cbdc323cfb9d978b827c13436f5363f4fca979a71",
    "name": "version-WindowsServerCore1803"
    "size": 2197049286,
    "architecture": "amd64",
    "os": "windows"
    "docker_version": "18.09.0",
    "author": "my@address",
    "created": "2018-12-11T08:55:24.0645376Z",
    "signature": null,
    "labels": []
},
{
    "digest": "sha256:cfce02ff28bbc2bce5ec46cbc82984c74634035b02d60ca99db88af414646396",
    "name": "version-Windows1809",
    "size": 2197049286,
    "architecture": "amd64",
    "os": "windows"
    "docker_version": "18.09.0",
    "author": "my@address",
    "created": "2018-12-11T08:55:24.0645376Z",
    "signature": null,
    "labels": []
},

note that the entry for the manifest list in the tags list has default field values which is also wrong.

Versions:
Please specify the versions of following systems.

• harbor version: v1.5.1-8bbd84ea

Will add this stuff later when I can get access to it

• docker engine version:
• docker-compose version:

Additional context:

• Harbor config files: You can get them by packaging harbor.cfg and files in the same directory, including subdirectory.
• Log files: You can get them by package the /var/log/harbor/ .

[ywk253100]

The Harbor API doesn't support manifest list yet.
The manifest list can be pulled/pushed from/to the backend docker registry, so the docker client works well.
When a client that doesn't support manifest list tries to pull a manifest list, the registry returns the manifest of the image in the list whose arch is amd64 and OS is linux. If the image isn't found, an error will be returned, this is what your case.

[Richard-Payne]

ok, thanks @ywk253100
Is support planned or is it just on the uncommitted backlog?

[SteveOss]

Is there a plan for supporting manifests properly, for those of us with multi-architecture images this is an important feature. Would be nice to see the feature tagged against a potential release, rather than just lingering in the backlog forever.

[ywk253100]

As Harbor v1.8 introduces the supporting to replicate images from DockerHub to Harbor, the image whose architecture is amd64 and OS is linux(if found) or the first image will be replicated. This is the limitation for now.

[darkl0rd]

See also #6207.

[Richard-Payne]

As Harbor v1.8 introduces the supporting to replicate images from DockerHub to Harbor, the image whose architecture is amd64 and OS is linux(if found) or the first image will be replicated. This is the limitation for now.

That doesn't really answer my question. I have no interesting in Dockerhub->Harbor replication. Is there are plan for when the api will be fixed to support manifest lists and if so, what are the projected timescales?

[darkl0rd]

Prior to v1.8, I could at least push manifest lists. We just couldn't replicate them.
After the upgrade to v1.8, an error is thrown when trying to push a manifest list, effectively breaking all our multi-arch images, which previously worked.

[ddompe]

Hi, is anyone working on this? Is on the roadmap? I will have to abandon our Harbor deployments and use something else if the manifest supports doesn't come soon, we are held back to pre 1.8 versions because of this. If not full support for it, maybe just a google on to let publish manifest knowing that the replication won't work.

[l0nax]

We are currently facing the same problem as @ddompe

[Jack47]

We are currently facing the same problem as @ddompe

Hopes to see this feature on roadmap

[sampsonhuo]

We are currently facing the same problem as @ddompe

[iiwoai]

We are currently facing the same problem as @ddompe

[frankcorneliusmartin]

There is already a pull request for this: https://github.com/goharbor/harbor/pull/10448

[ddompe]

Thanks @frankcorneliusmartin , but that PR is not reverting the code at https://github.com/goharbor/harbor/pull/7666 that blocks uploads of manifests.

[dajose]

@ywk253100 is there a progress with this? we are looking forward it

[dajose]

does this relates with #10877 ?

[xaleeks]

Manifest lists push / pull will be supported on the upcoming 2.0 release. We will also allow manipulation of the manifest list as a whole as well as manipulation of the individual images referenced within. When you pull a manifest list, it will correctly match to image within based on the OS / platform of the client and only pull that image. Please contact me for a early build to test if you guys have interest, thx

[ddompe]

@xaleeks What is the timeline for 2.0? If we can run an early build that would be great. Do you have nightly builds? BTW, I run on a K8S cluster, I think 2.0 will have an operator, does it?

[xaleeks]

please see official support stance here https://github.com/goharbor/harbor/issues/8268, multiarch images in the form of manifest lists are now supported in v2.0. 2.0 GA is end of this month, and you'll likely see an RC build much sooner than that.

other relevant issues on manifest list replication and scanning here
https://github.com/goharbor/harbor/issues/11136
https://github.com/goharbor/harbor/issues/10064

[xaleeks]

@ddompe The operator is another deployment of Harbor that we've been working on, we housed it in a separate repo and will cut a beta release very soon. This is not part of v2.0 and technically it has its own release cadence, but will obviously taper to the main release. Please read about the Operator roadmap here - https://goharbor.io/blog/introducing-the-harbor-operator/