Danger: Git Fatal on Danger Branches

[raymondhoagland]

Report

What did you do?

Run danger on a pull request using Github/Jenkins.

What did you expect to happen?

Running Dangerfile without seeing git fatals.

What happened instead?

Every build that runs prints
fatal: Couldn't find remote ref refs/heads/danger_base and
fatal: Couldn't find remote ref refs/heads/danger_base
three times each, corresponding to the calls to git_fetch_branch_to_depth. This seems to be tied to the manually assigned refspec, which makes git look for refs/heads/danger_head & refs/heads/danger_base as remote branches, throwing fatals if those branches do not exist.

I've manually set refspecs for the job, +refs/pull/*:refs/remotes/origin/pr/* refs/heads/${ghprbTargetBranch}:refs/heads/danger_base refs/pull/${ghprbPullId}/head:refs/heads/danger_head, in my case using the ghprb plugin in Jenkins to provide refs/heads/danger_head & refs/heads/danger_base, but the manual refspec assigned to the fetch command seems to override this anyway.

Why would a remote danger_base or danger_head branch be expected? If it is not expected, do we need the refspec in the fetch? The refspec isn't included in the call to git_in_depth_fetch which makes me think we don't need it.

This isn't a hard error as Danger gets past it each time, but would rather not see fatals unless there is something actually wrong.

Your Environment

• Which CI are you running on?
  Jenkins
• Are you running the latest version of Danger?
  5.5.3
• What is your Dangerfile?
  ruby # please paste here, with 2-space identation, thank you! xcov.report( scheme: ENV["FL_XCOV_SCHEME"], workspace: ENV["FL_XCOV_WORKSPACE"], skip_slack: ENV["FL_XCOV_SKIP_SLACK"], derived_data_path: ENV["FL_XCOV_DERIVED_DATA"], only_project_targets: true ) xcode_summary.ignored_files="Pods/**" xcode_summary.report(ENV["FL_XCS_REPORT"])

[orta]

Danger uses those branches to simplify some internal logic around generating the diffs and keeping track of where master vs PR head is. They should be generated at runtime, which is why they're not in documentation anywhere. It seems strange that Danger cannot make those branches, is there some weird permissions thing?

[raymondhoagland]

I'm not sure - the issue only seems to pop up when using a shallow clone with a very specific refspec (it's a huge repo that can't be reasonably fully cloned very often). It seems like if the refspec doesn't include whatever commit Danger is looking for, it falls back to git_fetch_branch_to_depth as expected but in the process ignores removes anything besides what it's looking for along the way which results in this output.

I did start seeing failures pop up sporadically around this - my best guess for when/how it happened at the time was that a new commit was added to the base branch at the same time (or thereabouts) that Danger gets the commit from the API. The Github API, at least the Enterprise version that I'm running on, appears to report the new commit before it's actually available to fetch which means Danger doesn't start out with the commit. Because the commit isn't available for Danger, it starts trying to perform incremental fetches and ends up removing the danger_head and danger_base branches along the way because of the --prune in the refspec.

[orta]

You can "work around" this problem by switching to Danger JS which only relies on the GitHub API to generate all this metadata (so no need to have a local git setup with the same infra as the PR ) - I don't think it uses modern APIs in GitHub, so you should be fine with any enterprise build.

[raymondhoagland]

I'm actually using danger from fastlane, so I'm a little hesitant to make that switch

[dalemyers]

We are seeing this issue as well running on VSTS. Same Dangerfile, same problem. The log looks like:

fatal: Couldn't find remote ref refs/heads/danger_base
fatal: Couldn't find remote ref refs/heads/danger_head
fatal: Couldn't find remote ref refs/heads/danger_base
fatal: Couldn't find remote ref refs/heads/danger_head
fatal: Couldn't find remote ref refs/heads/danger_base
fatal: Couldn't find remote ref refs/heads/danger_head
fatal: Couldn't find remote ref refs/heads/danger_base
fatal: Couldn't find remote ref refs/heads/danger_head
From https://[redacted].visualstudio.com/[redacted]/_git/[redacted]
   1cc96ae9b7..9e854a4b96 [redacted] -> origin/[redacted]
bundler: failed to load command: danger (/usr/local/bin/danger)
RuntimeError: Cannot find a merge base between danger_base and danger_head.
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/scm_source/git_repo.rb:128:in `find_merge_base'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/scm_source/git_repo.rb:16:in `diff_for_folder'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/danger_core/dangerfile.rb:261:in `setup_for_running'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/danger_core/dangerfile.rb:271:in `run'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/danger_core/executor.rb:28:in `run'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/lib/danger/commands/runner.rb:68:in `run'
  /usr/local/lib/ruby/gems/2.5.0/gems/claide-1.0.2/lib/claide/command.rb:334:in `run'
  /usr/local/lib/ruby/gems/2.5.0/gems/danger-5.5.11/bin/danger:5:in `<top (required)>'
  /usr/local/bin/danger:23:in `load'
  /usr/local/bin/danger:23:in `<top (required)>'

The branches that it refers to are totally random however. They have nothing to do with the build being run. I think it was just the latest push to the repo.

[kgaidis]

Having a similar issue that sometimes is there, and sometimes isn't (has been happening on-and-off for the past year or so):

fatal: Couldn't find remote ref refs/heads/danger_base
fatal: Couldn't find remote ref refs/heads/danger_head
fatal: Couldn't find remote ref refs/heads/danger_base
...

RuntimeError: Cannot find a merge base between danger_base and danger_head.
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/scm_source/git_repo.rb:128:in `find_merge_base'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/scm_source/git_repo.rb:16:in `diff_for_folder'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/danger_core/dangerfile.rb:261:in `setup_for_running'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/danger_core/dangerfile.rb:271:in `run'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/danger_core/executor.rb:28:in `run'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/lib/danger/commands/runner.rb:68:in `run'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/claide-1.0.2/lib/claide/command.rb:334:in `run'
  /Users/behance/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/danger-5.6.2/bin/danger:5:in `<top (required)>'
  /Users/behance/.rbenv/versions/2.3.1/bin/danger:22:in `load'
  /Users/behance/.rbenv/versions/2.3.1/bin/danger:22:in `<top (required)>'

[JoeSSS]

It feels like danger does some black magic to get the base and the HEAD, which is not really compatible to what shallow clone is about. What helped to me is adding master additionally to refspec: +refs/pull/${ghprbPullId}/merge:refs/remotes/origin/pr/${ghprbPullId}/merge +refs/heads/master:refs/remotes/origin/master and then defining base and the head on my own while calling danger like: danger --head=\$(git rev-parse HEAD) --base=\$(git rev-parse origin/master) ....

[kgaidis]

Just putting an update for others who may view this in the future.

My issue was "fixed" by looking into shallow clone:
https://github.com/danger/danger/issues/768#issuecomment-398893158

We had shallow clone of depth 1, and once it was changed to shallow clone of depth 100 it has worked since. The 100 is just a random number to maximize the chance of this issue _not_ happening.

[bdrelling]

This is still an issue today. I'm lucky I discovered this by perusing the PR and seeing this comment:

https://github.com/danger/danger/issues/768#issuecomment-398893158

[fredoliveira]

For the folks who end up stumbling on this issue because they use Github Actions, changing the depth of the check-out fixed the issue for us. Here's our full danger action, which may help some folks:

name: Run Danger

on: [pull_request]

jobs:
  build:
    name: Run Danger
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code
        uses: actions/checkout@v2
        with:
          fetch-depth: 100

      # Setup ruby
      - name: Set up Ruby 2.6
        uses: actions/setup-ruby@v1
        with:
          ruby-version: 2.6

      # Install the right bundler version
      - name: Install bundler
        run: gem install bundler

      # Cache dependencies
      - name: Cache ruby dependencies
        id: cache-ruby
        uses: actions/cache@v2
        with:
          path: vendor/bundle
          key: union-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            union-gems-

      # Install dependencies on cache miss
      - name: Install ruby dependencies
        if: steps.cache-ruby.outputs.cache-hit != 'true'
        run: |
          bundle config path vendor/bundle
          bundle install --jobs 4 --retry 3 --without=documentation

      # Run danger
      - name: Run danger
        env:
          DANGER_GITHUB_API_TOKEN: ${{ secrets.DANGER_GITHUB_API_TOKEN }}
        run: |
          bundle config path vendor/bundle
          bundle exec danger

[james-gonzalez]

^^ I can confirm this worked for us! Thanks @fredoliveira

[tkotosz]

thanks @fredoliveira :bow:

[zaimramlan]

Appreciate it @fredoliveira 🙇‍♂️