Harbor: Import existing Harbor registry from S3

[LunaticZorr]

What can we help you?

We've lost our Harbor setup which was using S3 for storage, and have created a new one. We're now trying to import all our images from S3 into the new setup.
We've created the new setup using the same S3 configuration, but the images do not show up. I can push new images to the registry and these show up withing S3.

Is there any way of importing the S3 images from S3? Even if not directly from Harbor. Is there any way of getting them back in?

Thank you!

[FrederikNS]

We just upgraded from harbor v1 to harbor v2, and the database migrations failed. We decided that we had all the data available in S3, so we could probably delete the harbor database and recreate it, and simply hook it up to the existing S3 bucket. But yeah, now we have a completely empty harbor, we can push images, and they show up in S3, but the images that are already present in S3 are not showing up and harbor, and pulling the image simply yields a unknown: repository <repo-name> not found.

[LunaticZorr]

We just upgraded from harbor v1 to harbor v2, and the database migrations failed. We decided that we had all the data available in S3, so we could probably delete the harbor database and recreate it, and simply hook it up to the existing S3 bucket. But yeah, now we have a completely empty harbor, we can push images, and they show up in S3, but the images that are already present in S3 are not showing up and harbor, and pulling the image simply yields a unknown: repository <repo-name> not found.

Yep, same situation we faced. We ended up using a small bash script on everyone of our servers that pushed every image locally stored to the new registry. This way we at least recovered every working image we had. But many Dev and archived images were lost.

[FrederikNS]

It seems that there used to be a feature where you could set the SYNC_REGISTRY environment variable to true, and harbor would go and scan S3, however it was removed back in february without any explanation: https://github.com/goharbor/harbor/commit/bd204464f33d8d61f156f14c5a0da91296f8eb90

Would this feature be possible to ressurect?

[FrederikNS]

We desperately wiped our entire harbor registry and installed the latest v1 version (1.10.2) instead.

We manually recreated all the projects (not repositories) we had previously and set the environment variable SYNC_REGISTRY=true on the core container and then restarted it. This resulted in harbor starting up and scanning our entire registry, so we now have all our images present again. User accounts and similar settings are of course still lost though.

We will retry the upgrade to version 2 again.

[FrederikNS]

Interestingly as long as we stay on version 1 we have all our repositories and image tags available when we follow the above mentioned procedure. However as soon as we upgrade to version 2, the repositories are kept, but all the image tags are lost.

[reasonerjt]

@FrederikNS
In v2.0 Harbor uses its DB as the source of truth for tags. So it seems in your migration v1.10 -> v2.0 the tags are failed to populate into the DB.

After you sync the registry in 1.10 as described in https://github.com/goharbor/harbor/issues/11773#issuecomment-641418113, did you try to upgrade again? Was is successful and could you provide the log of harbor-core so we can check?

[FrederikNS]

Yes, we did try the upgrade again, but that resulted in all the artifacts going missing... I do unfortunately not have the logs available.

[bitsf]

Is it possible that querying S3 is very slow, and db sync is not finished after upgrade.
Maybe you can try use a small set of images on S3, testing if upgrade still fail.

[reasonerjt]

IMO we may start by providing a external tool in future to help user check the consistency, and sync data.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.