Renovate: Private ruby gem: Host key verification failed

[Scritik]

Which Renovate are you using? CLI, App, or Pro

App

Which platform are you using? GitHub, GitLab, Bitbucket Azure DevOps

Github

Have you checked the logs? Don't forget to include them if relevant

Details

WARN: Failed to generate Gemfile.lock (unknown error)(branch="renovate/activerecord-6.x")
{
  "err": {
    "killed": false,
    "code": 11,
    "signal": null,
    "cmd": "docker run --rm -v /mnt/renovate/gh/xxxxx/the-main-repo:/mnt/renovate/gh/xxxxx/the-main-repo -w /mnt/renovate/gh/xxxxx/the-main-repo renovate/ruby:latest bash -l -c \"ruby --version && gem install bundler -v 2.0.2 --no-document && bundle lock --update activerecord\"",
    "stdout": "ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]\nSuccessfully installed bundler-2.0.2\n1 gem installed\nFetching [email protected]:xxxxx/the-private-gem.git\n\n\n\n\n",
    "stderr": "Host key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (3/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (4/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nGit error: command `git clone '[email protected]:xxxxx/the-private-gem.git'\n\"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\"\n--bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has\nfailed.\n",
    "message": "Command failed: docker run --rm -v /mnt/renovate/gh/xxxxx/the-main-repo:/mnt/renovate/gh/xxxxx/the-main-repo -w /mnt/renovate/gh/xxxxx/the-main-repo renovate/ruby:latest bash -l -c \"ruby --version && gem install bundler -v 2.0.2 --no-document && bundle lock --update activerecord\"\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (3/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (4/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nGit error: command `git clone '[email protected]:xxxxx/the-private-gem.git'\n\"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\"\n--bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has\nfailed.\n",
    "stack": "Error: Command failed: docker run --rm -v /mnt/renovate/gh/xxxxx/the-main-repo:/mnt/renovate/gh/xxxxx/the-main-repo -w /mnt/renovate/gh/xxxxx/the-main-repo renovate/ruby:latest bash -l -c \"ruby --version && gem install bundler -v 2.0.2 --no-document && bundle lock --update activerecord\"\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (3/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nRetrying `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` due to error (4/4): Bundler::Source::Git::GitCommandError Git error: command `git clone '[email protected]:xxxxx/the-private-gem.git' \"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\" --bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has failed.\nHost key verification failed.\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nGit error: command `git clone '[email protected]:xxxxx/the-private-gem.git'\n\"/usr/local/lib/ruby/gems/2.6.0/cache/bundler/git/the-private-gem-2ec564e1b1d2af42a20b1e5230aee9d6876377b3\"\n--bare --no-hardlinks --quiet` in directory /mnt/renovate/gh/xxxxx/the-main-repo has\nfailed.\n\n    at ChildProcess.exithandler (child_process.js:294:12)\n    at ChildProcess.emit (events.js:198:13)\n    at ChildProcess.EventEmitter.emit (domain.js:448:20)\n    at maybeClose (internal/child_process.js:982:16)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)"
  }
}

What is your question?

Hi,

I tried again to add renovate for a ruby project and the gemfile seems to be parsed correctly now.
The issue is that we have a private ruby gem, hosted on a private repo. I gave renovate access to this repo too but it seems that it can't fetch the content. Am I missing something?

Thanks

[rarkins]

Do you mean it's a gem hosted on a private github.com repository? And that you've installed Renovate into that repo too, so that Renovate has access? And by failure to you mean the "bundle update" command fails?

[Scritik]

Yep, this is a gem hosted on a private Github repo. I added access to this repo in Renovate (but it's not configured). The bundle update tries to fetch the repo and fails (Host key verification failed). You have all the details in the logs sections of my first message.

[rarkins]

This means that the GitHub token we are granted for Renovate should be enough for Bundler, but right now we are not passing it to Bundler. We need to work out the best way to tell Bundler "use token X for github.com" or "use token X for github.com/foo" in case there might be different tokens one day.

According to , we can do it with bundle config https://github.com/bundler/bundler.git username:password or export BUNDLE_GITHUB__COM=username:password. Hopefully this works with the app's tokens using x-access-token:${token}

[rarkins]

I've moved this into the main Renovate repo as it will require code changes, not config changes.

[Scritik]

Thx! It seems to be the latest issue before we can replace Dependabot with Renovate 🤞

[Scritik]

Hi @rarkins, any news on this?

[rarkins]

I think the next step is to be able to reproduce this. @Scritik is it possible you can create two dummy repos where repo A depends on some public ruby gems and also on repo B? You can have repos A and B public for now, what I would do is then:

(a) Fork these two and confirm it works while B is public
(b) Make B private and confirm this "breaks" the current Renovate, and
(c) Test out the environment variable approach and see if it unbreaks it

[Scritik]

@rarkins I played a little bit with two repos in public and I think that we have other issues too.

https://github.com/dakis/test-renovate-rb-1 (the gem)
https://github.com/dakis/test-renovate-rb-2

I installed the bot on the second repo, and this is what I have:

{
  "config": {
    "bundler": [
      {
        "packageFile": "Gemfile",
        "manager": "bundler",
        "registryUrls": [],
        "deps": [
          {
            "depName": "test-renovate-rb-1",
            "managerData": {
              "lineNumber": 2
            },
            "currentValue": "1.0.0",
            "datasource": "rubygems",
            "updates": [],
            "warnings": [
              {
                "updateType": "warning",
                "message": "Failed to look up dependency test-renovate-rb-1"
              }
            ]
          }
        ],
        "compatibility": {
          "bundler": "2.0.2"
        }
      }
    ]
  }
}

The first thing I noticed in my tests (and looking at Renovate's code), is that I need to specify a version in the Gemfile. It shouldn't be mandatory, we have the lockfile for this.

Then, it seems that the git repository as a source are not supported at all. Correct me if I'm wrong, but I don't see anything related to a git source here: https://github.com/renovatebot/renovate/blob/master/lib/manager/bundler/extract.ts

[Scritik]

@rarkins Have you had the chance to take a look at this?

[rarkins]

@Scritik thanks for following up on this with the excellent analysis.

The first thing I noticed in my tests (and looking at Renovate's code), is that I need to specify a version in the Gemfile.

Currently Renovate doesn't support lockfile-only updating for Bundler, i.e. our approach is to update the Gemfile and then use bundler to update the lock file. If you are already going to be using Renovate to keep this dependency up to date, I don't see a reason why not to put it in the Gemfile though (it will be updated in the same PR as the lock file).

It shouldn't be mandatory, we have the lockfile for this.

Agreed, but it's just a matter of implementation time for us. If a dependency version is unspecified in the Gemfile but has e.g. 1.0.0 in the lock file, are you aware of a bundler command that lets us update to a specific version? e.g. if both 1.0.1 and 1.1.0 exist then some users may want separate PRs for patches vs minor updates. Often it's the case that package managers update to the latest instead of letting you specify exactly.

Then, it seems that the git repository as a source are not supported at all.

Yes I think you're right - we need to add git repository awareness to the bundler extract and update functions. I have created a separate issue here: https://github.com/renovatebot/renovate/issues/4789

[Scritik]

@rarkins A specific version, I'm not sure but it seems that we can update based on the type of the update (patch/minor/major): https://bundler.io/v2.0/bundle_lock.html

I made a quick test and it's working well.

[bilby91]

Hello,

I'm trying to get renovate working on a project that has a dependency on a private repository. This repository is protected with HTTP basic authentication. The bundler configuration bundle config .... doesn't seem to work.

Renovate is trying to call https://my.private.registry/api/v1/gems/my-gem.json, is this API supported by rubygems ?

Thanks

[rarkins]

@bilby91 what does the Gemfile entry look like for that package?

[bilby91]

@rarkins My Gemfile looks like this:

source 'https://rubygems.org'

gem 'rake', '0.9.2.2'

source 'https://my.private.registry'do
  gem 'my-gem', '1.0.0'
end

I think my issue is that my private repository doesn't expose the /api/v1/gems/:gem.json API. I was able to add the HTTP Basic authentication using hostRules. The username and password field seems to be missing in this documentation -> https://docs.renovatebot.com/configuration-options/#hostrules

I'll try to send a PR to add them.

[rarkins]

Do you know what api format/syntax your registry supports? Sounds like we need to add support for it

[bilby91]

@rarkins Thanks for the quick reply!

I'm not using a "real" gem repository, we are basically using gem generate_index and then upload the artifacts to an S3 bucket. Maybe that is a use case that you would like to support ? Is geminabox gem server supported ?

Also, is there any way to skip dependencies ? I think it would be beneficial for us if we can introduce renovate in our team initially if we can skip our private gems.

Thanks

[rarkins]

You can definitely skip dependencies. Easiest way would be with the ignoreDeps option if there's not too many, or with a package rule with packagePatterns if they follow a naming convention.

[bilby91]

@rarkins Awesome!

Will it avoid this path https://github.com/renovatebot/renovate/blob/f0d4995fdac22611a7d5e5f20acc0963d800e67f/lib/datasource/rubygems/get.ts ?

[rarkins]

Yes if a dependency is ignored then it won’t ever get looked up

[zharinov]

@Scritik Hi, I managed to obtain this PR with forks of your example using this Renovate branch.

What it can do:

• Recognize git and github options, looking for new versions in tags (also I'm thinking about releases too).

What it can't do (yet):

• Infer available versions from gemspecs
• Update lockfiles for SSH-based git urls
• Support for sources like gemfury, github packages, etc

[Scritik]

@zharinov Great! That a really good first step to solve our issue. Thanks :)
Is it working with private repos?

[zharinov]

Is it working with private repos?

No, but it seems to be the proximate next step

[micheelengronne]

If anybody asks, passing the credentials through the ENV variable (so BUNDLE_GITHUB__COM=username:password) doesn't work.

[micheelengronne]

Waiting for https://github.com/renovatebot/renovate/pull/5269

[rarkins]

@micheelengronne GitHub has quite a few ways to authenticate - OAuth tokens, App tokens, Personal Access Tokens, etc - can you verify if none of GitHub's tokens ever work with BUNDLE_GITHUB__COM= or if you just tried a subset?

[micheelengronne]

In fact I tested with my own Geminabox. So, BUNDLE_MY__HOST__NAME=username:password and it still doesn't work. In my dev and prod environment, bundle can fetch my Geminabox gems that way.

[rarkins]

Do you mean you tested with Renovate?

Renovate filters the list of env variables by default unless you set trustLevel=high in your bot config (e.g. config.js or RENOVATE_TRUST_LEVEL=high in env

[micheelengronne]

oh ok, I didn't know that. I will try again.

[micheelengronne]

It doesn't seem to work either. I have INFO: RubyGems lookup failure: authentication failed and:

       "registry": "https://rubygem.domain.name",
       "err": {
         "name": "HTTPError",
         "hostname": "rubygem.domain.name",
         "method": "GET",
         "path": "/api/v1/gems/rubocop-performance.json",
         "protocol": "https:",
         "url": "https://rubygem.domain.name/api/v1/gems/rubocop-performance.json",
         "gotOptions": {
           "path": "/api/v1/gems/rubocop-performance.json",
           "protocol": "https:",
           "hostname": "rubygem.domain.name",
           "hash": "",
           "search": "",
           "pathname": "/api/v1/gems/rubocop-performance.json",
           "href": "https://rubygem.domain.name/api/v1/gems/rubocop-performance.json",
           "headers": {
             "user-agent": "https://github.com/renovatebot/renovate",
             "hosttype": "rubygems",
             "accept": "application/json",
             "accept-encoding": "gzip, deflate"
           },
           "hooks": {
             "beforeError": [],
             "init": [],
             "beforeRequest": [],
             "beforeRedirect": [],
             "beforeRetry": [],
             "afterResponse": []
           },
           "retry": {"methods": {}, "statusCodes": {}, "errorCodes": {}},
           "decompress": true,
           "throwHttpErrors": true,
           "followRedirect": true,
           "stream": false,
           "form": false,
           "json": true,
           "cache": false,
           "useElectronNet": false,
           "baseUrl": "https://rubygem.domain.name/",
           "method": "GET",
           "gotTimeout": {"request": 60000}
         },
         "statusCode": 401,
         "statusMessage": "Unauthorized",
         "headers": {
           "server": "nginx",
           "date": "Mon, 24 Feb 2020 11:32:08 GMT",
           "content-type": "text/html;charset=utf-8",
           "content-length": "16",
           "connection": "close",
           "x-powered-by": "geminabox 1.1.1",
           "www-authenticate": "Basic realm=\"Gem In a Box\"",
           "x-xss-protection": "1; mode=block",
           "x-content-type-options": "nosniff",
           "x-frame-options": "SAMEORIGIN",
           "set-cookie": [
             "rack.session=7f8d96fe775170bbc1db46278a745d95cdbead1f6761bdd2d150b5add9043b0b; path=/; expires=Mon, 24 Feb 2020 11:48:48 -0000; HttpOnly"
           ]
         },
         "body": "Not Authorized.\n",
         "message": "Response code 401 (Unauthorized)",
         "stack": "HTTPError: Response code 401 (Unauthorized)\n    at EventEmitter.<anonymous> (/home/renovate/.config/yarn/global/node_modules/got/source/as-promise.js:74:19)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)"
       }

[micheelengronne]

It seems to break before bundler is called. So, the ENV auth for Bundler is not called yet.

[micheelengronne]

I also put that in my hostRules:

 hostRules: [
   {
     hostType: 'rubygems',
     domainName: 'rubygem.domain.name',
     username: 'USERNAME',
     password: 'PASSWORD',
   }
 ]

Maybe, the hostType is wrong.

[rarkins]

These are off-topic for the current issue, so I am going to minimize them to avoid confusion. Let's wait on @bilby91 to deliver #5269 and first and not spam the participants of this issue so hard that they unsubscribe. Private Bundler authentication is not supported until then.

[bilby91]

@rarkins When you get a chance take a look at my last question on the PR, I did some progress on the weekend but need to double check some behaviours.

[bilby91]

5269 Has been merged!