Amber: Amber repository size is huge

[faustinoaq]

Description

Currently amber takes 7.0 MB on disk and amber_spec takes 33MB!!!

Steps to Reproduce

1. Create a new amber projects
2. Install dependencies
3. Check project size (aprox 61MB!!!)

Expected behavior: Project size around 20MB or less

Actual behavior: Project size above 50MB

Reproduces how often: 100%

Versions

0.3.0

Additional Information

Maybe amber using 7.0MB is ok but Why 33MB!!! for amber_spec?

[marksiemers]

@faustinoaq - For amber_spec this has a lot to do with files being committed that should have been gitignored, and the pack file has blown up. It took some work, but I got it down to around 7MB (and I think most of that is the dependency on amber):

$ git clone [email protected]:marksiemers/amber-spec.git
Cloning into 'amber-spec'...
remote: Counting objects: 340, done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 340 (delta 2), reused 12 (delta 1), pack-reused 324
Receiving objects: 100% (340/340), 7.34 MiB | 466.00 KiB/s, done.
Resolving deltas: 100% (22/22), done.
...
$ git count-objects -vH
count: 0
size: 0 bytes
in-pack: 340
packs: 1
size-pack: 7.35 MiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

I followed these steps:
http://www.ducea.com/2012/02/07/howto-completely-remove-a-file-from-git-history/
It involves using filter-branch - so history has been re-written. I'm not sure if there is any other way to get the repo size down though.

For amber-spec it may be worth considering the filter-branch approach. For amber itself, that may be a bit riskier.

I'm going to experiment with getting the size down anyway, just to see what is possible.

UPDATE: It doesn't look like any significant decreases in size can be had for the amber repo using the filter-branch approach - however, it can work for amber-spec

[marksiemers]

FWIW, I just wanted to see how small the current set of files is for both repos:
Steps - clone and cd; rm -rf .git; git init; git add .; git commit -m "..."; git gc

For amber:

count: 0
size: 0 bytes
in-pack: 365
packs: 1
size-pack: 325.02 KiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

For amber-spec:

$ git count-objects -vH
count: 0
size: 0 bytes
in-pack: 12
packs: 1
size-pack: 3.76 KiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

Of course, it is impossible to get it that small. It makes me wonder if it would be good (in general, not just with amber) - to have a new repo for the 1.0 release, and keep the 0.x code in an archived repo.

[eliasjpr]

Is this resolved? If so can we close this issue

[faustinoaq]

@eliasjpr this repo still is a bit heavy, about 7MB:

➜  Projects git clone [email protected]:amberframework/amber.git 
Cloning into 'amber'...
Enter passphrase for key '/home/main/.ssh/id_rsa': 
remote: Counting objects: 6612, done.
remote: Compressing objects: 100% (123/123), done.
remote: Total 6612 (delta 59), reused 94 (delta 33), pack-reused 6453
Receiving objects: 100% (6612/6612), 7.04 MiB | 607.00 KiB/s, done.
Resolving deltas: 100% (3594/3594), done.

And amber-spec is 7MB too:

➜  Projects git clone [email protected]:amberframework/amber-spec.git          
Cloning into 'amber-spec'...
Enter passphrase for key '/home/main/.ssh/id_rsa': 
remote: Counting objects: 387, done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 387 (delta 10), reused 29 (delta 7), pack-reused 353
Receiving objects: 100% (387/387), 7.36 MiB | 505.00 KiB/s, done.
Resolving deltas: 100% (40/40), done.

I think is better than 33MB but still huge for a repository with source code < 400KB (amber) and < 12KB for amber-spec.

Maybe we can remove old tags or something similar, WDYT?

[faustinoaq]

@eliasjpr Why not recreate amber-spec?, I mean That repo only have 16 commits, 3 releases and source is < 12KB but git clone is 7MB weight.

We have 2 passing PRs on https://github.com/amberframework/amber-spec/pulls and 0 issues

[faustinoaq]

to have a new repo for the 1.0 release, and keep the 0.x code in an archived repo.

@marksiemers I agree, something like amberframework/amber-pre-alpha

[marksiemers]

@faustinoaq and @eliasjpr One of those PRs is from me, and it substantially changes things. If we recreate amber spec, I would likely try to save that code, and reintroduce the changes as a PR - rather than merge it in now and re-create.

[elorest]

Amber can be cleaned up whenever we close all the PR's and delete their branches. I've experimentally gotten the size down to a MB.

https://github.com/amberframework/ambertrimmed

[faustinoaq]

Amber size is increasing, now amber repo is over 8MB 😅

amber_spec was solved by garnet_spec 🎉

[marksiemers]

@faustinoaq - With history, it is going to grow.

If we want to pause on branches and PRs, it sounds like @elorest can get the size down to ~1MB

This should be doable, we'll just have to do work in forks for a little while.

[eliasjpr]

Can we apply https://github.com/amberframework/ambertrimmed to this repo?

[eliasjpr]

What is causing the repo size increase? Is the tags, specific commits?

[faustinoaq]

@eliasjpr We must merge all PRs and delete all branches before trimming this repo

[elorest]

@eliasjpr Amber trimmed was a test of deleting all branches except master. Cleaning and pushing up to a repo without locked refs. It's a couple days behind right now too. Should be deleted soon.

https://github.com/amberframework/ambertrimmed only exists as an example.

[eliasjpr]

@elorest thanks. That is awesome if we can get it down to that size

[faustinoaq]

Done! 🎉