Bfg-repo-cleaner: GitLab won't update repo size

[gpassero]

I ran bfg-repo-cleaner as explained in https://rtyley.github.io/bfg-repo-cleaner/ and apparently it worked. However, the git repo size in GitLab web interface still shows 250MB, when it should be <50MB. I also noticed that a txt file with 26MB was not removed from the repo, even the cleaner said it would. I then removed the file with a new commit.

I can't find some files in git history anymore - the ones deleted with --delete-folders option. A file stripped with --strip-blobs-bigger-than is still there in the history, but with no content.

I tried running git gc in the server, but gitlab repo is still 250MB.

Any help here?

[javabrett]

Is your assertion that the GitLab-hosted repo should shrink from 250Mb to <50Mb based on an observation of that size-change in the local clone where you ran BFG? In other words, do you see what you expect in terms of files cleaned-up and shrink in the local clone where you ran BFG, but just not in GitLab?

There's a lot of things that could be going-on in the GitLab repo that might cause this. What does git ls-remote show?

[gpassero]

Actually it may be a problem with GitLab, not BFG. Unless this problem happens to other repositories (i.e. github or standard git).
I opened an issue in GitLab too.

[gpassero]

Does BFG clear out historical objects too?

[huoteng]

@gpassero I have the same situation with you.

Before I run bfg-repo-cleaner, the size of repo on GitLab is nearly 100M. After running bfg-repo-cleaner as explained in https://rtyley.github.io/bfg-repo-cleaner, the size of local repo reduce to 8M. Then I run git push to sync to remote repo, the size of repo on GitLab goes to 200M+. But when I clone the remote repo to a new folder, it's truely 8M.

Considering the situation is caused by the bug of GitLab?

[javabrett]

This is not a BFG issue, but one that will occur when working with "managed" remotes such as Gitlab etc. Those remote repositories typically have developer/management tooling such as pull-requests, code-review etc., which often creates refs at paths which are not cloned by-default by a git client. That means that you will see a small clone-size post-clean-up, but the remote on-disk size of the repository won't shrink.

At that point you might want to assess what your goal is in running BFG to shrink the repo. If it is to provide a cleaned-up, compact repo for new clones, then you might consider this complete - disk space is fairly cheap. You will of course run the risk of someone accessing those old trees/commits in history and getting confused because the history has been rewritten.

Take a look at admin: can't reduce gitlab repository size after accidental large file push and merge requests and Document how to use filter-branch to reduce the repo file size for info on how Gitlab can retain refs to old history e.g. in refs/keep-around/<id>. You might need to read-up on your options for cleaning-up these objects and getting Gitlab to gc afterwards.

[AlyShmahell]

@javabrett Do you have an idea of how to get Gitlab to gc?

[javabrett]

Did you try a web search for: gitlab force gc, and try some of those
instructions?

[AlyShmahell]

@javabrett Yeah, what I found works if you're running your own Gitlab server, I'm using repositories provided by Gitlab.com, so my quickest solution was to:

• reconstruct the entire history to exclude the unwanted objects while keeping relevant commits to the wanted objects.
• delete the repo on remote, and delete the origin on local repo.
• create a new remote repo.
• add the new remote repo as origin, and force push the commit history.

I will try to make a script to facilitate the process, then post it here for clarity.

[geberl]

Can confirm that this is not an issue with bfg but with GitLab. This is how I got it resolved: https://stackoverflow.com/q/31932574