Git-lfs: Credentials not found using Username / Password in remote

[pascalberger]

If I have a remote like https://user:[email protected] git lfs still throws an error that credentials are not found. A normal git fetch works fine.

Trying to using this in a VSTS with the OAuth token passed from VSTS. See Microsoft/vso-agent-tasks/issues/962.

[technoweenie]

Cool, thanks for the report. This is a pretty low priority bug though. Putting authentication info in your git config is a pretty terrible idea, security-wise.

[pascalberger]

@technoweenie I'm with you regarding using auth in the config and security. But my use case is that I need to fetch lfs objects on a build server (VSTS). I only have the OAuth token available and therefore temporarily change the remote url to contain the OAuth token to access the repo on the server.

The other option would be to use a dedicated service account only for this and have its credentials stored permanently on the build agent which I think is worse security-wise.

[blazespinnaker]

This breaks for me as well. I'm just entering my credentials at the command line. Also, my repo isn't exact super duper secret territory considering it's all open source anyways... I honestly can't imagine why anyone would try to hack it.

[whoisj]

Cool, thanks for the report. This is a pretty low priority bug though. Putting authentication info in your git config is a pretty terrible idea, security-wise.

Is there a recommended solution to the problem presented?

Is there something the GCM could be providing here?

[andyneff]

This breaks for me too... My scenario is less security sketchy though... I have to have a username and password on the lfs server, but I have a "read only" permission account that I actually WANT to be public. That's the username and password in plain text that I want to have in the .lfsconfig file.

Also, I noticed the lfs.url will parse the username only if its supplied but neither if both are supplied. e.g.

url = http://[email protected]:1234/stuff.git/info/lfs

Works

For what its worth, my workaround is to set the store credential helper before hand...

git config --global credential.helper "store --file=/pw.txt"
echo "https://user:[email protected]:1234" > /pw.txt

[WestonThayer]

@pascalberger were you ever able to get this working?

@andyneff I tried that trick in VSTS to no avail. Are you on VSTS too?

[pascalberger]

@WestonThayer No, using a service account for just this is no option for us. Therefor unfortunately no git-lfs for us until this get fixed.

@technoweenie Any chance to have the priority of this increased, since it seems like I'm not the only one with a use case requiring this?

[andyneff]

@WestonThayer It should work on VSTS too. (I created an account real quick and tested it in a docker. It DOES work) If you are still having problems, it might be on how your VSTS account is setup (send me a private message on gitter if you want to discuss the VSTS part further).

If you are using a service like VSTS, there is no need to set a custom lfs url, The credential storage will work for the normal git url AND the git lfs url, which will be based off of the same https url from VSTS

In another scenario where you need to use an external lfs server, keep in mind what the FULL lfs url normally have something AFTER the *.git directory. This of course could vary from server to server, that's all beyond my scope.

For example, a github.com url would be https://github.com/username/projectname.git, and you can NOT set the lfs.url to the exact same, because that's not the lfs url. The lfs url is https://github.com/username/projectname.git/info/lfs.

@pascalberger Why can you not use the store credential in your situation? If you have a script on your build server, leave the url untouched in whatever script you are using, and just set the store credential (temporarily even, if that's all you want) and use a plain text file?

[pascalberger]

@andyneff I don't have user / pwd credentials but only an OAuth token to access the repo during the build. Of course I could create some service user, store his credentials on the build server and this way access the repository. But speaking about security this would make completely no sense, contrary to temporarily put an OAuth token in the remote.

[andyneff]

@pascalberger Used OAuth is the username, and the <OAuth token> is the password. (Tested, and works)

echo https://OAuth:[email protected] > /pw.txt

Notice: there is NO path in that line, just the server

Edit: I was using a "Personal access token" for this. If a build token doesn't work this way, sounds like a VSTS issue, and even if THIS issue was fixed, it wouldn't help.

Also, I'm only testing all of this on my local machine, I was assuming once you get it working locally, the solution can be transferred to your build system

[WestonThayer]

@pascalberger I chatted with Andy offline, we were on different pages. This is a VSTS build server issue (where they execute Git in a special context) as far as I can tell. Things that work on my local machine don't work on the VSTS build server.

[technoweenie]

Hey all, thanks again for the feedback. I've added this to the list for Git LFS v1.2. Though I really want to get a new release out next week, so it may have to wait for Git LFS v1.3 (unless someone steps up to do it). Here's a comment where I'm discussing next week's release: https://github.com/github/git-lfs/issues/844#issuecomment-204080138

[jonmagic]

I've been trying to replicate this on master (with help from @technoweenie and @LizzHale) and haven't been able to yet. Here are the tests we wrote showing that it uses the user:pass pair from the url whether it is in the git config for lfs.url or remote.origin.url.

Wondering if you can provide more details to help me replicate this? Or wondering if it got inadvertently fixed in the changes to master since the version the error was reported on?

[WestonThayer]

I was able to get my scenario to work by switching my VSO (now VSTS) build agent from running under a service account to my personal account. When it was running via the service account, I assume Windows wouldn't let Git see the git.config for some reason. It was confusing that remote.origin.url didn't work either.

In any case, my fix was to run the build agent as a Windows service with my credentials, then install Git Credential Manager for Windows. After that it just worked, so I stopped poking at it.

[pascalberger]

My original scenario was like this:

• Using VSTS Build (vNext), with an Agent run under a service account (and no Credential Manager)
• Updating repo remote during build with git remote set-url (using this script: https://github.com/iozag/vsts-git-tasks/blob/master/Tasks/EnableGitRemoteAccess/EnableGitRemoteAccess.ps1) to add user OAuth and as password the provided OAuth token from VSTS build system.
• Calling git lfs fetch

It returned a Git credentials for https://mytenant.visualstudio.com/DefaultCollection/MyProject/_git/MyRepo/info/lfs/objects/841e3a4174433932fc9e0a7f506cfda7f240b640199a656632300a971d11bd58 not found. error.

If I manually call git remote -v in the Git repository on the agent after running the build it shows the remotes being updated with username/password.

Also standard git operations (like git fetch) worked with the modified remote url.

[technoweenie]

These tests prove that configuring your git remote or lfs.url values with a username/password in the url work just fine. Let me know if there's some other configuration I should try.

@WestonThayer mentioned that the service account didn't have permissions to access the git config. Unfortunately, this is a hard requirement for Git and Git LFS to work successfully, so you'll have to work those permissions issues out on your own.

One thing to keep in mind is that LFS will _only_ pass the credentials from the git remote to the LFS endpoint if the hosts match, as this could leak your credentials to other systems. I'm assuming this doesn't effect you since both the Git and LFS servers are on https://mytenant.visualstudio.com. You can confirm by comparing git remote show origin and git lfs env output.

[jonmagic]

One thing to keep in mind is that LFS will not pass the credentials from the git remote to the LFS endpoint if the hosts match

Should this be unless the hosts match?

[pascalberger]

If it is a permission issue with accessing the git config, wouldn't standard git operations, like git fetch also fail?

[technoweenie]

I'm going to need exact settings that are used, so we can replicate it in our tests:

• git lfs env
• git config --list

You can of course change host names and credentials, since that stuff will be fake in our tests anyway.

[technoweenie]

One more thing that would help: Running the failing fetch or push operation with GIT_TRACE=1. I updated and pushed the tests @jonmagic and I were working on to #1136.

[pascalberger]

Here are the settings:

• git lfs env: https://gist.github.com/pascalberger/9ccf1439a2cafa211eb5f2279d781ea8
• git config --list: https://gist.github.com/pascalberger/7b05f650b8f4bb3f49fe64625f00c316

One more thing: If I manually run git lfs fetch on the machine the login screen appears asking for credentials. Running git fetch works without asking for credentials.

[technoweenie]

@andyneff was able to provide me an example that I could check out. Running with GIT_TRACE=1 showed me this:

trace git-lfs: HTTP: POST https://USER:[email protected]/repo.git/info/lfs/objects/batch
trace git-lfs: HTTP: 200
trace git-lfs: HTTP: {"objects":[{"oid":"0000000000000000000000000000000000000000000000000000000000000000","size":1,"_links":{"download":{"href":"https://git-lfs-server.com/repo/info/lfs/objects/0000000000000000000000000000000000000000000000000000000000000000","header":{"Accept":"application/vnd.git-lfs"}}}}]}
trace git-lfs: HTTP:
Username for 'https://git-lfs-server.com':

The LFS client is authenticating https://USER:[email protected]/repo.git/info/lfs/objects/batch properly. However, the LFS server is not returning authenticated urls to the requested objects, like https://git-lfs-server.com/repo/info/lfs/objects/0000000000000000000000000000000000000000000000000000000000000000. This is undesirable, but should be easy enough to fix.

[pascalberger]

And here my git lfs fetch with GIT_TRACE=1: https://gist.github.com/pascalberger/093fce9eea909db09968305b486d1083

The output is until the first credential dialog appears (if I cancel the credential dialog two others appera until the command exits)

[technoweenie]

Cool thanks. Sorry for handling this poorly. I thought I knew exactly where the bug was. But if I'd have assumed ignorance and asked up front for debug data (like GIT_TRACE), I'd have solved this really quickly in Dec. A fix will be coming in v1.2 next week.

[andyneff]

@pascalberger @WestonThayer

With git lfs 1.2.0, I'm able to get my scenario working where I have my "oauth" key in .lfsconfig. Are you able to get it working in your scenario of git clone https://user:[email protected]?

[foresterre]

I have the same issue on _git lfs 1.2.0_ on _windows 8.1_.

git lfs env and git config --list:
https://gist.github.com/foresterre/1c44f08cff4b31c77e0c8097fb506f04

I enter my credentials on the command line.

[andyneff]

@foresterre If you are supposed to enter your password on the command line is more about not having the password helper working. See here

[RagtagShawn]

What exactly is the work around if you're not on v1.2? I have to support a remote contractor using SourceTree on Windows and they (Atlassian) have not moved SourceTree to v1.2 yet on Windows. This user is not very technical so I'd like to keep it as simple as possible. Their current setup is using their normal GitHub credentials via https and keeps getting the authentication failure when they do a big push with a lot of LFS tracked files up to the remote at the end of the push command.

[andyneff]

@RagtagShawn The same workaround should still work

git config --global credential.helper "store --file=/pw.txt"
echo "https://user:[email protected]:1234" > /pw.txt

Where user is their username, pass is their password, blah.com:1234 would be the hostname and port of the external lfs repo, and /pw.txt would be the location for storing their password, in plain text, for all to see, that's why this isn't a great solution ;) But this solution/issue is geared towards more automated procedures (where prompting ONCE was a problem) than dealing with "being asked for a password multiple times", that sounds like the credential helper isn't set up.

If they are using windows, I think the windows credential helper is good at remembering the password after prompting only once.

[RagtagShawn]

@andyneff Thanks for the quick reply.

No the credential helper is not setup as described in that link because like I mentioned the user is using SourceTree and the embedded version of Git/LFS that comes with it. I am trying to keep the user's setup as easy as possible. Also they are not getting prompted for their credentials just the push keeps failing with the credential error and a git lfs pull command will not work due to the same error.

Normal git commands are working and the push appears to be partially working because every time we re-initiate the push the output indicates LFS tracked files that are being skipped because they are presumably already pushed up to the LFS server.

Anyway I am not a Git super user, so how exactly do I find the "would be the hostname and port of the external lfs repo"? Our repository is hosted on GitHub.

[andyneff]

@RagtagShawn I don't know anything about SourceTree, but it sounds like you are not using the feature that this issue is about (username AND password in the url). This issue was a "it didn't work at all" kind of thing. You might try the chat room or asking SourceTree for help.

[RagtagShawn]

@andyneff You're right it was not exactly the same issue, but it was the same symptoms. I got around it by configuring the user's SourceTree to use the latest versions of Git for Windows and LFS. Not a big deal, but I was hoping to avoid extra installation steps for supporting less technical and remote users. Regardless your replies got me on the right track. Thanks!

[kkrugler]

I recently ran into this issue, when a few files were added to the GitHub repo that require git-lfs. Previously I'd configured my URL in .git/config to be something like https://<username>@github.com/<organization>/<reponame>.git, which was necessary once I enabled 2FA in GitHub (and I didn't want to jump through the password helper hoops).

The git-lfs URL in the .git/config file looks similar:

[lfs "https://<username>@github.com/<organization>/<reponame>.git/info/lfs"]`
    access = basic

But doing a pull returns Error downloading object: <path> (<hash>) for every file in GitHub that was added using LFS.

Running git lfs logs last says:

Git credentials for https://<author>@github.com/<organization>/<reponame>.git not found:
'git credential fill' error: exit status 1

If I put my access token in the URL (so now it's https://<author>:<access token>@github.com/<organization>/<reponame>.git) then I can clone, and it all works. But it's weird that regular git is fine with just my username, but LFS needs my access token.

Finally, after the error, I had to explicitly run git lfs pull to pull down the files that failed during the git pull. Seems like git pull only runs git lfs pull if it has non-LFS files to pull.

[technoweenie]

Git credentials for https://<author>@github.com/<organization>/<reponame>.git not found:
'git credential fill' error: exit status 1

That's an error with Git. Maybe you don't have your git credential helper set up properly? Check out https://help.github.com/articles/caching-your-github-password-in-git/ for some help there.

You can also run GIT_TRACE=1 git pull to get some more details about why your git credential helper didn't work.

EDIT: If you're having credential helper problems, please post a new issue so we can help you out. I've locked this issue since the original request has been fulfilled, and individual requests for help don't need to email everyone else on this thread.