Azure-devops-docs: Any limits on Git LFS

[shanehsu]

Are there any limits on single file size or total storage size for Git LFS?

---

文件詳細資料

⚠ 請勿編輯此區段。 這是 docs.microsoft.com ➟ GitHub 問題連結的必要項。

• ID: 96e60bfd-765f-81ad-312d-f46ef9a66ddc
• Version Independent ID: c7ea62ca-1ba4-9a63-30d5-4788f4dce0c4
• Content: Git limits - Azure Repos
• Content Source: docs/repos/git/limits.md
• Product: devops
• Technology: devops-code-git
• GitHub Login: @steved0x
• Microsoft Alias: sdanie

[shanehsu]

Can you guys get back to me here when you got a reply from the dev team? Putting a feature on hold for this.

[bluewwork]

tutoril pls,with vid in windows 10

[macjustice]

It appears that LFS is also affected by the 5GB push limit. I'm opening a support request to see if there's any way to increase that limit for LFS usage.

[arammaliachi]

@macjustice thank you for spotting this, the change has been submitted and is under review by the Content Development team. In short:

Git LFS pushes don’t count towards the 5GB push limit. There is no limit for the size of the files in LFS. The 5GB limit is only for files in the actual repo, not blobs stored as part of LFS.

[ssa3512]

@arammaliachi I am not able to push a 342 MB file with git lfs push. Azure devops returns a HTTP 413 (Request Entity Too Large).

[ssa3512]

I was able to determine a ~120MB file size limit using this method:

• Create new azure devops git repository
• Run the following powershell in an emtpy folder

git init
git lfs install
git remote add origin [remote url]
git lfs track *.bin
git add -A
git commit -m "Initial Commit"

for ($i = 10; $i -le 350; $i += 10) {
    Write-Host $i
    fsutil file createnew $("testfile_" + $i + "MB.bin") ($i * 1024 * 1024)
    git add -A
    git commit -m "Added $i MB file"
    git push origin master
}

This generates arbitrary files in increasing 10MB steps. A 120MB file will push fine, but a 130MB file and anything larger fails with an HTTP 413 error.

This seems like a crazy low limit for Git LFS as GitHub supports files up to 2GB.

[arammaliachi]

@ssa3512 HTTP 413 is usually a grumpy proxy or any other networking/machine limitation. I have seen tickets for that. Please open up a support request for the Azure DevOps support team and send the ID to my email: [email protected]

[shanehsu]

@macjustice Sorry to bother, but is there any update on increasing the total push size beyond 5 GB? Also, I'd like to know if there is a soft or hard limit on total LFS object size.

[arammaliachi]

@shanehsu product group is not currently considering to increase the 5GB limit/push operation for Git repositories that do not implement LFS. The 5GB limit is hard for files that are not tracked by LFS.

I don't think we have a size limit per LFS object.

[macjustice]

Hello, in my testing I was able to upload multiple files above the 5GB
threshold using LFS. I haven’t run into a limit on total LFS object
storage, either, but I haven’t pushed it that hard yet.

On Mon, Dec 23, 2019 at 9:53 AM 徐鵬鈞 [e-mail] wrote:

@macjustice https://github.com/macjustice Sorry to bother, but is there
any update on increasing the total push size beyond 5 GB? Also, I'd like to
know if there is a soft or hard limit on total LFS object size.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/vsts-docs/issues/4179?email_source=notifications&email_token=AAEDGS2XFB4AV5ILFLUZZ3DQ2D3H7A5CNFSM4HLTLIDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHRTSRA#issuecomment-568539460,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEDGS2YUJ3OBNBMFDZHKL3Q2D3H7ANCNFSM4HLTLIDA
.

[arammaliachi]

@shanehsu; @macjustice; @ssa3512; I think we have a size limit per LFS object of 128MBs, will confirm and update the documentation accordingly.

[arammaliachi]

@shanehsu ; @macjustice ; @ssa3512 Running the following command within the affected repository may get you unblocked:

git config http.version HTTP/1.1

Then retry the push.

[ssa3512]

@arammaliachi that worked for me. I was able to push the file I couldn’t previously.

[dshakya]

I am trying to push a file over 5GB with LFS. It starts to push using http/1.1 but restarts after approximately 2 mins. On a faster upload connection, i get upto 1.4Gb and on slow upload speed only get upto about 112MB and then restarts. The time is about 2 mins on both speed. Any help is appreciated. I can see the file being listed using git lfs ls-files

[dshakya]

I did a test by creating a VM on Azure. Clone the repo and added the 5GB file and pushed using faster connection. This time, it went above 3GB and then crashed at 2 mins mark. (Again!) Is there something I need to tweak so that the push does not restart at 2 mins?

05:18:03.288870 trace git-lfs: tq: starting transfer adapter "basic"
05:18:03.289855 trace git-lfs: HTTP: PUT https://.visualstudio.com//info/lfs/objects/918f62a074bef6d1f3ffc4117ee226093bad7d0ce3bec8d7735fc3682923841e
05:20:08.434420 trace git-lfs: HTTP: 503B | 19 MB/s
05:20:08.436423 trace git-lfs: tq: retrying object 918f62a074bef6d1f3ffc4117ee226093bad7d0ce3bec8d7735fc3682923841e: Fatal error: Server error: https://.visualstudio.com//info/lfs/objects/918f62a074bef6d1f3ffc4117ee226093bad7d0ce3bec8d7735fc3682923841e from HTTP 503
05:20:08.436423 trace git-lfs: tq: enqueue retry #1 for "918f62a074bef6d1f3ffc4117ee226093bad7d0ce3bec8d7735fc3682923841e" (size: 5402327976)
05:20:08.436423 trace git-lfs: tq: sending batch of size 1
05:20:08.436423 trace git-lfs: api: batch 1 files

........AFTER MULTIPLE RESTART......

Uploading LFS objects: 0% (0/1), 8.7 GB | 34 MB/s

[JSalazar88]

@shanehsu ; @macjustice ; @ssa3512 Running the following command within the affected repository may get you unblocked:

git config http.version HTTP/1.1

Then retry the push.

This appears to work for us as well

[norlin]

Any confirmed answer on this?
It's weird that there is no info about how much data I can store in total with git+lfs with Azure

[tedvanderveen]

Same problem here. Mostly HTTP 413 errors and sometimes 503 error. Relatively small number of files (35) total size around 250Mb. Nothing too special.

[moyukhbiswas]

@shanehsu ; @macjustice ; @ssa3512 Running the following command within the affected repository may get you unblocked:

git config http.version HTTP/1.1

Then retry the push.

I was facing the same issue and this works. Also, there seems to be more cases of this failure from my geolocation (India), so that might be something to note.

[pri-kise]

Is there any progress or news? We need to know if there are any limits on single file size or total storage size for Git LFS, to decide if we are going to use Azure DevOps with Git LFS.

[ssa3512]

Is there any progress or news? We need to know if there are any limits on single file size or total storage size for Git LFS, to decide if we are going to use Azure DevOps with Git LFS.

There are no limits for Git LFS, however there is a bug surrounding HTTP/2 that requires you to change a git config setting on any client that needs to push files larger than a certain threshold (seeemingly around 120MB)

[MattJeanes]

The git config http.version HTTP/1.1 fix worked for us too uploaded a blob of around 800MB, original error response was HTTP/2.0 413 Request Entity Too Large using GIT_CURL_VERBOSE=1 and GIT_TRACE=1

[pri-kise]

Is anyone else having the problem that the upload restarts after some time?
Is there any timeout ?

EDIT: Setting the git config for this repository via git config http.version HTTP/1.1 didn't helped.

[kwaclaw]

Yes git config http.version HTTP/1.1 is the solution.
My uploads failed at about 140MB.

[FlsZen]

This still seems to be an issue, luckily git config http.version HTTP/1.1 is still the workaround.