Harbor: Overwritten tags will not be GCed.

Created on 29 Sep 2017  ·  12Comments  ·  Source: goharbor/harbor

This is a limitation of registry:
https://github.com/docker/distribution/issues/2212

A PR has been there for 2months but still pending:
https://github.com/docker/distribution/pull/2302

Currently we can treat it as limitation, if user keeps hitting it we'll consider fixing it.

Some solution by community.
https://github.com/mortensteenrasmussen/docker-registry-manifest-cleanup

backlog candidat1.8.0 dependencexternal kincustomer-found kinlimitation

Most helpful comment

Managed to get the mortensrasmussen clean up method to work:

1 - Disable authentication by editing common/config/registry/config.yml. Comment out
following section with hash.

auth:
token:
issuer: harbor-token-issuer
https://docker.iotechsys.com/service/token
rootcertbundle: /etc/registry/root.crt
service: harbor-registry

2 - Edit docker-compose.yml file to expose registry port:

registry:
ports:
- 127.0.0.1:5000:5000

3 - Stop and restart harbor for changes to take effect

4 - Run cleanup to remove unused manifests

docker run --network="host" -it -v /data/registry:/registry -e REGISTRY_URL=http://127.0.0.1:5000 mortensrasmussen/docker-registry-manifest-cleanup

5 - Run normal cleanup to remove blobs now no longer associated with a manifest

docker run -it --name gc --rm --volumes-from registry vmware/registry-photon:v2.6.2-v1.4.0 garbage-collect /etc/registry/config.yml

6 - Back out config/compose file changes and restart harbor

All 12 comments

Hello, is there any chance of fixing this?
The solution suggest won't work as we can't access directly the docker registry API.
If we can't clean overwritten tags the registry will continue to grow until there is no space left.

Looks like docker/distribution#2302 was merged in January.

Managed to get the mortensrasmussen clean up method to work:

1 - Disable authentication by editing common/config/registry/config.yml. Comment out
following section with hash.

auth:
token:
issuer: harbor-token-issuer
https://docker.iotechsys.com/service/token
rootcertbundle: /etc/registry/root.crt
service: harbor-registry

2 - Edit docker-compose.yml file to expose registry port:

registry:
ports:
- 127.0.0.1:5000:5000

3 - Stop and restart harbor for changes to take effect

4 - Run cleanup to remove unused manifests

docker run --network="host" -it -v /data/registry:/registry -e REGISTRY_URL=http://127.0.0.1:5000 mortensrasmussen/docker-registry-manifest-cleanup

5 - Run normal cleanup to remove blobs now no longer associated with a manifest

docker run -it --name gc --rm --volumes-from registry vmware/registry-photon:v2.6.2-v1.4.0 garbage-collect /etc/registry/config.yml

6 - Back out config/compose file changes and restart harbor

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Pending for the new release of docker registry.

docker run --network="host" -it -v /data/registry:/registry -e REGISTRY_URL=http://127.0.0.1:5000 mortensrasmussen/docker-registry-manifest-cleanup

Hi SteveOss,

I tried the steps you given, but always get error:
Could you please help what is problem? thanks in advance

# docker run --network="host" -it -v /data/ic/harbor/registry:/registry -e REGISTRY_URL=http://127.0.0.1:5000 mortensrasmussen/docker-registry-manifest-cleanup
curl: (7) Failed to connect to 127.0.0.1 port 5000: Connection refused
Could not contact registry at http://127.0.0.1:5000 - quitting``

3 - Stop and restart harbor for changes to take effect
finally, I executed the docker-compose down and then docker-compose up, it works now.

Thanks!

@wwyhy Glad you got it working :)

This will be included in Harbor gc schedule? Thanks

PR(#7170) has been merged into master, close it as fixed.

Hi @SteveOss I tried what you said, half of the images are cleaned. But the other half got error like: Failed to clean manifest 45bd95d24e36cdd78d11eda9a02c57bde166cee2932023e2118356536e356c17 from repo pfic/pfic-dev with response code 404
Do you have any solutions for that?

怎么关闭接收???

---Original---
From: "sunyao"notifications@github.com
Date: Mon, May 27, 2019 00:14 AM
To: "goharbor/harbor"harbor@noreply.github.com;
Cc: "Subscribed"subscribed@noreply.github.com;
Subject: Re: [goharbor/harbor] Overwritten tags will not be GCed. (#3344)

Hi @SteveOss I tried what you said, half of the images are cleaned. But the other half got error like: Failed to clean manifest 45bd95d24e36cdd78d11eda9a02c57bde166cee2932023e2118356536e356c17 from repo pfic/pfic-dev with response code 404
Do you have any solutions for that?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

mramanathan picture mramanathan  ·  3Comments

izhichao picture izhichao  ·  3Comments

abououdine picture abououdine  ·  3Comments

Hotege picture Hotege  ·  3Comments

reasonerjt picture reasonerjt  ·  3Comments