Rubberduck: Github Integration Setup

[brab]

I'm having trouble setting up an existing VBA project to use a brand new Github repo using Rubber Duck v2.0.3b. I haven't been able to find instructions to set up source control, and after muddling through the GUI a bit and not being very successful, I tried using the "initialize new repository from project" button and then dropped down to the Github desktop shell (via powershell) and added the remote and branch ref for master manually. The project files had been copied to the folder and committed so I git pushed and everything went up to Github.

I should note that I'm very experienced with Git, but not at all experienced with Windows, Excel or VBA. I'm trying to on board someone to my team that has been writing VBA without any source control for some time.

At this point the RD GUI isn't seeing any branches, nearly all of the buttons are disabled and I can't get it to recognize changes to the VBA code. I've tried using both git and https endpoints but that doesn't seem to matter.

I'd be just fine with wiping the repository and starting again if that's necessary but I'm hoping that this is recoverable and I'm just missing settings somewhere.

Any help would be appreciated. Thanks in advance!

[Hosch250]

First, you need to initialize a new repo or import an existing repo (only do the later if you are in blank project). This should create a "master" branch and an initial commit for you to work from. Rubberduck relies heavily on changes being done from its UI--it won't pick up changes until you close and reopen the project if you do it through the command line. It also doesn't pick up changes to the code automatically because of the way the VBE works--it will only pick up the changes if you click the Refresh button or do a reparse (and I've been having a bit of trouble getting normal reparses to consistently refresh the Source Control UI).

We are working on writing tutorials on how to use it for our new website that will be launched along with the stable release of 2.0.

[retailcoder]

I need to make this wiki page more visible - @Hosch250 is it still all valid?

[Hosch250]

Yep. My notes in the new website are a bit more comprehensive though (if incomplete).

[brab]

You're very quick; thank you! I'm going to take a look again this afternoon and try and get things running so I'll get back to you after that.

@Hosch250 could you please link to your notes on the new website?

[Hosch250]

I'm still working on them--I haven't even committed them to the repo.

[brab]

That's fair; I am using the bleeding edge release, after all.

When I click "Ok" in the clone dialog as per the wiki page linked to above, Excel crashes with the message, "Excel has stopped working," and no more information. Excel crashes and restarts and nothing has been cloned to the local directory on my computer.

I'm using Windows 7, VBA 7.0, and MS Office Pro Plus 2010. Any suggestions?

[Hosch250]

Actually, no. We keep getting reports about that, but I can't reproduce them :/ Are you cloning a GitHub or BitBucket repo? Is it public or private? In the last release, I fixed a crash caused by cloning a private BitBucket repo, and I've successfully cloned public GitHub and BitBucket repos multiple times. I cannot test cloning a private GitHub repo because I don't pay for my account, but it _shouldn't_ happen... Also, is the repo you are cloning empty? That _shouldn't_ be an issue, but I haven't tested it in a while.

[brab]

I'm cloning a private Github repo.

Since there are now unlimited private repos I've created a private one and added you, @Hosch250, as a collaborator. Go nuts.

The repository I tried to clone isn't empty since the instructions on the wiki page said to create a readme file anyway. I tried using the repo that somehow magically got everything I wanted in it yesterday, but on a different computer with a completely fresh install of RD v2.0.3b.

[Hosch250]

I just cloned it and committed a new file from Rubberduck. Everything worked as expected.

[retailcoder]

@brab so it worked, only, off a fresh install? this looks like a possible glitch with the source control config file then. do you still have the source control configuration file for the failing install?

[brab]

I haven't had it work successfully for me yet, no. I'll try the new repo on the new machine in a little bit and get back to you; just heading into a meeting.

I'll also grab the gitconfig for the existing, failing repo and post it here.

[brab]

I wasn't thinking before. There isn't a gitconfig for the failing repo because nothing has been cloned.

I just tried to connect the private repo that I created earlier and I got the same failure. I also created a public repo, tried that and got the same error again.

I wonder if there's a logging in to Github step that I'm missing? I've provided my username and email in the Source Control settings but I haven't added an RSA ID to the new machine or logged in. Public repos shouldn't need this step, but could it be a requirement of Rubber Duck? I've installed Github desktop so I can drop into powershell and inspect what's going on but so far I haven't done anything that would mutate state via the command line.

[Hosch250]

No, this shouldn't matter. If it detects there is a 401 error, it will display the login screen. Hmm, that would be an interesting bug if GitHub returned the 401 error in a different language--we hardcode the value to tell it apart from other errors. It wouldn't ask for it for a public repo, anyway, though.

How exactly are you doing this? You do have an empty project in the VBE, right?

[retailcoder]

@brab nah, I was referring to the SourceControl.config file, which is an XML file that contains information about what repositories Rubberduck knows about (nothing sensitive, just project names and paths).. @Hosch250 where's that file saved again?

[Hosch250]

Right by the other config file in C:\Users{username}\AppData\Roaming\Rubberduck.

[brab]

Right now I'm just opening Excel, pressing Alt-F11 and then opening the Rubberduck -> Source Control menu option. From there I'm clicking on the clone button, entering https://github.com/brab/rubberduck-test-public into the Remote Path text input and clicking Ok. Then I get the error and Excel crashes.

Here's the config after performing the above:

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
  <SourceControlSettings>
    <UserName>brab</UserName>
    <EmailAddress>[email protected]</EmailAddress>
    <DefaultRepositoryLocation>C:\Users\User\Documents\GitHub</DefaultRepositoryLocation>
    <Repositories />
    <CommandPromptLocation>cmd.exe</CommandPromptLocation>
  </SourceControlSettings>
</Configuration>

[Hosch250]

That looks about right. Which Excel screen do you see when you open it, the default workbook one?

[brab]

When I open Excel I see the default workbook, similar to your screenshot above. When I open VBE I see an empty main pane and on the left I see a "Project - VBAProject" pane with "VBAProject (Book1)" expanded with three sheets and a "ThisWorkbook" object. Below that I see a "Properties - Sheet1" pane with a number of properties enumerated below.

[Hosch250]

Yup, that sounds exactly right.

[Hosch250]

Oh wow, I just duplicated this with the release build, but not with the debug build.

[Hosch250]

Interestingly enough, I can't duplicate it with the latest build compiled into a release, although I can reproduce it with the last release. Please try this release and see if it works: https://1drv.ms/u/s!Arb7hwtzEkcogtA8EGYQ7-3T47XCTg

[brab]

Thanks!

When I use that release I get a "Rubberduck Add-In could not be loaded" error when I launch the VBE. The last two code identifying lines of the traceback read:

2015\Projects\Rubberduck\RetailCoder.VBE\Root\RubberduckModule.cs:line104
C:\Users\hosch\Documents\Visual Studio 2015\Projects\Rubberduck\RetailCoder.VBE\Extension.cs:line47

I'm actually on my way out of the office now and will be back on Monday (I'm in Canada), so this is going to have to wait on my end. Thank you both very much for your help so far.

[retailcoder]

@brab not fair! I only have Friday off! :'(

[Hosch250]

Thanks for the help, but those two lines are useless--I'd need the full stack trace (or at least the first two lines would be better than the last two lines). At this point, I have no idea what is going on with any of this...

[brab]

@retailcoder extra-long weekends are great. I highly recommend requesting an extra day off far in advance. ;)

@Hosch250 that's too bad, sorry about that. I was being lazy and not logging in to github on the windows machine and typing instead. I won't have access to a windows comp until Monday so I'll take a screenshot then.

Have a good weekend you two!

[brab]

@Hosch250 @retailcoder good morning. I hope you both had a good weekend.

Here's the screenshot of the custom build failing when opening VBE with a new workbook:

[Hosch250]

@brab Yep, known issue, and fixed in the latest release.

[brab]

Fantastic. I was able to open VBE and clone my rubberduck-test-public repo!

When I try it with a private repo it complains about my credentials. It's not asking for a 2FA code so I'm thinking that might be the problem? Thankfully I don't actually need editor access to this repo immediately and my new team members don't have 2FA enabled, so if that is the problem then I'll just make sure they don't set up 2FA.

PS. I really like the new Source Control "Manage" menu, it's much clearer!

[brab]

I just did a quick search of your issue tracker and didn't find any requests for this, so I'll mention it here and you can tell me what to do with it.

It would be ideal if instead of using my github.com credentials for authentication if I could setup an RSA ID keypair and connect via the git protocol. This gets around the 2FA issue (I'd prefer everyone on my team to be using 2FA) and is generally more secure.
[/end feature request]

Thoughts?

[retailcoder]

@brab we're using the LibGit2Sharp API to integrate/wrap Git (as does Visual Studio) - does VS deal with 2FA? We (@ckuhn203 actually) even made a PR on LibGit2Sharp to enable using a SecureString instead of storing credentials as plain strings... I like the idea, but I believe this would be best handled in the LibGit2Sharp repository rather than Rubberduck's.

[rubberduck203]

Let me take a look at LibGit2 a little bit to see if it's supported before we open any issues over there. Be right back.

[rubberduck203]

Does this look relevant to anyone? I've never used 2FA with Git.

https://github.com/libgit2/libgit2sharp/issues/1256

I've never used a private GitHub repo. I do know LibGit2Sharp supports SSH certs, but I don't know how using it works, let alone what changes we'd need to make to RD.

[Vogel612]

@ckuhn203 seems relevant.

Regarding ssh-certs. What we'd need to use those is a path to the ssh-cert to use. In general we could assume that %USER_DIRECTORY%/.ssh contains a key, but that's only convention and there may be mutliple keys.
We have to match the public-key signature that's authenticated against github to the corresponding private key and possibly unlock the private key to authenticate our interactions against the repo.

This should be properly handled with LibGit2Sharp though.

[Hosch250]

When you clone a private repository, you need to enter your username/password and click OK, and Rubberduck will use them to gain access to your repository. We only display that message on 401 errors, so if we don't have your credentials, we can't do anything.

[retailcoder]

@ckuhn203 seems if we use SSH, there's nothing more to do for 2FA. See http://stackoverflow.com/a/31305980/1188513:

https://github.com/blog/1614-two-factor-authentication#how-does-it-work-for-command-line-git

How does it work for command-line Git?

If you are using SSH for Git authentication, rest easy: you don't need to do anything. If you are using HTTPS Git, instead of entering your password, enter a personal access token. These can be created by going to your application settings page.

[rubberduck203]

@Brab, if I'm understanding correctly, it should work if you enter the token instead of a password. Can you try that please and let us know.

[brab]

@ckuhn203 I understand the excerpt in the same way. If that's the case, obviously a bit of documentation would handle the problem, at least in the short term.

I don't have time to give it a try today, but I've added @ckuhn203 @Vogel612 and @retailcoder as contributors to the private repo I setup for debugging this issue; if one of you has time to try things out, please by all means go ahead. Otherwise I should have some time early next week.

Edit I've just now had a minute to come back and read my comment and realized that earlier I missed the point of your request, @ckuhn203. I still don't think I'll have a chance to test it out before next week though so I'll get back to you. Thanks for the suggestion though, that would solve my problem if it works.

[brab]

@ckuhn203 using an authentication token worked just as expected. There's even a bit of text on Github's "Personal access token" page that describes using access tokens in this way:

So it seems that a bit of text on your (new and upcoming) Git integration instructions page would help to cover the use cases where a user has 2FA setup, or is used to using RSA ids to authenticate with Github and other Git servers.

It seems that I have to enter the access token each time I perform a command that interacts with the remote server. Since access tokens are long and hard to remember strings, this is inconvenient. Is Rubberduck supposed to save my credentials somewhere so I don't have to keep re-entering them? Especially since this is an access token and not my password I would have no issue with it being stored.

[retailcoder]

Rubberduck is supposed to keep the SecureString around in memory, and reuse it whenever it needs to supply a password. If there's no difference between a password and a token, then storing the token in plain text anywhere would amount to storing the password in plain text in that same somewhere, which would defeat the purpose of the SecureString support we PR'd into LibGit2Sharp.

Can you paste the token into the password box?

[Hosch250]

This is actually a bug in Rubberduck. We are _supposed_ to check whether we have credentials and use them every time, but we don't.

[brab]

@retailcoder thanks for that info. It seems that it mostly works during a session but after restarting I have to re-enter the access token.

@Hosch250 good to know. I assume there's an issue tracking this bug so I can watch it?

[Hosch250]

Actually, no, we just discovered it when you made that comment.

[Hosch250]

Oh, yes, we don't keep the token across sessions, maybe I'm remembering the in-session token request thing wrong.

[brab]

In this case I think I'd prefer that my password (access token) is encrypted (it sounds like this is happening with SecureString :+1:) and stored on disc so I don't have to re-enter it every time I open VBE.

[rubberduck203]

Without checking the current code (and it has changed a bit since I touched it last), RD will prompt for credentials anytime the plug in connects to a "new" repository. It definitely prompts for credentials the first time you access a repository after just opening the VBIDE. As Mug pointed out, we didn't want to persist sensitive info on the file system.

[retailcoder]

Relevant: http://stackoverflow.com/a/147775/1188513

That said, we can't encrypt the SecureString, we don't even have access to the password string itself (which is the whole point ;0)

[brab]

Yes, I completely understand not wanting to store passwords on disc. This is why using RSA IDs over SSH is the preferred standard for secure communication via Git.

In my case, the access token (password) is long and "hard to remember" so I'll have to use a service like cryptobin.co each time I need to log in on a machine that I don't have configured with my password managers (and never will). I don't use Windows, so for me this is just access to the tools my team use, not regular development.

Since this is a tool I don't plan to use daily I can handle this, but I'm sure you'll receive more requests at some point to make authentication more persistent.

I'm going to close this issue because you have all answered my questions and concerns. You're an amazingly responsive group! Thank you all for all of your help.

[retailcoder]

@brab thanks! not sure if that's considered cheating, but that's why we thank "Duga the chatbot" in our About box:

:smiley:

[brab]

Definitely not cheating. That's just good process!