Renovate: Support private go modules on github.com

[rarkins]

What would you like Renovate to be able to do?

Successfully fetch private Go modules/repositories on github.com

Describe the solution you'd like

Renovate should set whatever files or environment variables necessary such that go get can retrieve the information it needs to update go.sum.

Additional context

The lookup works because Renovate accesses GitHub's API itself, but go get fails because it uses native git authentication.

[rarkins]

These discussions give some ideas:

• https://stackoverflow.com/questions/47617432/docker-change-gitconfig-with-token-for-private-repo-access
• https://stackoverflow.com/questions/53663603/how-to-authenticate-a-private-go-module-using-go-1-11-and-google-app-engine-stan

Assuming we run go via the docker image, it looks like we can hopefully get auth working if we pass through the token we have to the docker image.

Possibilities:

• Write it to a config file and map it into the container into /home/ubuntu/.gitconfig
• Build a git credential helper into the renovate/go image and then pass it via ENV at runtime

I lean towards the first .gitconfig file as most reliable, although it would be nice to avoid writing to any file in the actual host file system

[rarkins]

The git credential helper is not a good idea too because we may ultimately have more hosts than just github.com.

Here's how the .gitconfig should look:

[url "https://USERNAME:[email protected]/"]
  insteadOf = https://github.com/

[rarkins]

This type of approach works:

docker run --rm -e GITHUB_TOKEN renovate/go bash -c "git config --global url.\"https://[email protected]/\".insteadOf \"https://github.com/\" && cat /home/ubuntu/.gitconfig"

[moorara]

This type of approach works:

docker run --rm -e GITHUB_TOKEN renovate/go bash -c "git config --global url.\"https://[email protected]/\".insteadOf \"https://github.com/\" && cat /home/ubuntu/.gitconfig"

Is this where this should be added?
https://github.com/renovatebot/renovate/blob/76242888dcb314aa336ac0cac4c245be75479c40/lib/manager/gomod/artifacts.js#L89

[rarkins]

@moorara yes, this is the call it needs to be added to - although also when we go mod tidy too. We may need an alternative naming to GITHUB_TOKEN though to handle cases like GitHub Enterprise where there's one token for the private github host and another for github.com.

[rarkins]

@emman27 is also interested in a solution to this problem. I'll copy/paste my comment fro https://github.com/renovatebot/docker-go/pull/9#issuecomment-476499218:

I'm trying to think up the advantages and disadvantages of various approaches to this problem:

Approach 1: make no changes to the image itself

• In this approach we'd run like docker run --rm -e GITHUB_TOKEN renovate/go bash -c "git config --global url.\"https://[email protected]/\".insteadOf \"https://github.com/\" && go get
• Advantage: we can easily add as many different tokens as we need (e.g. GitHub.com, GitHub Enterprise, and others)
• Disadvantage: the run string looks pretty long

Approach 2: Pre-configure a git config file and then just supply the tokens in env

• Advantage: seems cleaner
• Disadvantage: although it would work cleanly for github.com right now, it would be the hardest approach for managing multiple/optional host authentications

Approach 3: This PR with a custom entrypoint to detect env and then set it

• Advantage: It makes the go get command from Renovate look normal, it handles optionality of the token
• Disadvantage: I can't see how we can scale this to multiple and custom platforms in future, e.g. for https://github.mycompany.com etc. Also we need to overcome the backwards compatibility problem using either a more advanced entrypoint script or a custom tag.

[emman27]

Thanks @rarkins.

Some quick thoughts:

Approach 1: Configuration should not be done in plain text (the existing JSON file), as a github token constitutes sensitive information. Also requires a fair bit of git and docker know-how to setup for an enduser who may not understand this.

I kind of like approaches 2 and 3 - I take it that you're scheduling these containers to run on some environment where you have privileged docker access? Would a user-uploaded gitconfig file mounted as a volume be another option worth exploring?

[rarkins]

For the app, we definitely don't need any user-uploaded file. Secret can always be encrypted, so there is no concern about tokens in plain text.

In 100% of cases right now we will have a github.com token available (we need one for release note retrieval anyway) so we can solve the immediate/most common problem pretty fast. I'm just trying to think how to design it to be future proof in case of custom github or gitlab hosts, or gitlab.com or bitbucket.org modules, etc.

[moorara]

Here is how I can get the approach 3 working:
https://github.com/renovatebot/docker-go/pull/9#issuecomment-476960043

[renovate-bot]

:tada: This issue has been resolved in version 15.3.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[emman27]

@rarkins it seems that in order for what's just been released to work, I still need to check in a github token into my repo in plaintext? Any options for encryption so that we don't store this token in plaintext anywhere?

[rarkins]

@emman27 you don't need to check in a github token - Renovate will reuse the existing token for github.com. If you are running via the app, that means it uses the same token as Renovate uses to access the current repository, so you need to make sure Renovate is installed into the source repo too (Renovate can be disabled, but it must be installed in order to have permissions to read the contents).

Renovate does support encrypted variables too using the encrypted object however that should never be necessary here.