Dependabot-core: dependabot update fails with `key not found: :vendor_dependencies`

[mobilutz]

All our dependabot watched repositories are failing since a couple of days ago with this error:

  proxy | time="2020-08-20T08:10:20Z" level=info msg="proxy starting" commit=c5f79a43943044613474bf0c4b592cd1afbc064e
  proxy | 2020/08/20 08:10:20 Listening (:1080)
fetcher | yarn config v1.22.4
fetcher | success Set "cafile" to "/etc/ssl/certs/ca-certificates.crt".
fetcher | Done in 0.02s.
fetcher | INFO <job_44416105> Starting job processing
fetcher | ERROR <job_44416105> Error during file fetching; aborting
fetcher | ERROR <job_44416105> key not found: :vendor_dependencies
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `fetch'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `initialize'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `new'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `job'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:90:in `file_fetcher'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:80:in `base_commit_sha'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:12:in `perform_job'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:29:in `run'
fetcher | ERROR <job_44416105> bin/fetch_files.rb:21:in `<main>'
fetcher | ERROR <job_44416105> key not found: :vendor_dependencies
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `fetch'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `initialize'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `new'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `job'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:94:in `raven_context'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:148:in `handle_file_fetcher_error'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:23:in `rescue in perform_job'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:11:in `perform_job'
fetcher | ERROR <job_44416105> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:29:in `run'
fetcher | ERROR <job_44416105> bin/fetch_files.rb:21:in `<main>'
fetcher | /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `fetch': key not found: :vendor_dependencies (KeyError)
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `initialize'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `new'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:94:in `raven_context'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:40:in `handle_exception'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:32:in `rescue in run'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:27:in `run'
fetcher |   from bin/fetch_files.rb:21:in `<main>'
fetcher | /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `fetch': key not found: :vendor_dependencies (KeyError)
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `initialize'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `new'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:94:in `raven_context'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:148:in `handle_file_fetcher_error'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:23:in `rescue in perform_job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:11:in `perform_job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:29:in `run'
fetcher |   from bin/fetch_files.rb:21:in `<main>'
fetcher | /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `fetch': key not found: :vendor_dependencies (KeyError)
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/job.rb:31:in `initialize'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `new'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:76:in `job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:90:in `file_fetcher'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:80:in `base_commit_sha'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/file_fetcher_job.rb:12:in `perform_job'
fetcher |   from /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:29:in `run'
fetcher |   from bin/fetch_files.rb:21:in `<main>'

It does not matter, if it is a ruby, javascript or python repository.
None of our 16 repos that are being watched, have any mention of vendor_dependencies in it.

We tried enabling and disabled dependabot wihtout any success ...

Package manager/ecosystem

ruby:bundler, python, javascriot

Manifest contents prior to update

Updated dependency

What you expected to see, versus what you actually saw

_exptected:_
dependabot creating new pullrequest for our repositories.

_actual:_
dependabot fails with the error from above.

Images of the diff or a link to the PR, issue or logs

[feelepxyz]

@mobilutz thanks for reporting! Looking into a fix now.

[feelepxyz]

@mobilutz this should be fixed now, could you try and manually trigger a "Bump now" from the dashboard?

[j0k3r]

I got the same issue.
Retried few minutes ago and got once that error:

  proxy | time="2020-08-20T12:20:16Z" level=info msg="proxy starting" commit=c5f79a43943044613474bf0c4b592cd1afbc064e
  proxy | 2020/08/20 12:20:16 Listening (:1080)
fetcher | yarn config v1.22.4
fetcher | success Set "cafile" to "/etc/ssl/certs/ca-certificates.crt".
fetcher | Done in 0.04s.
fetcher | INFO <job_44444580> Starting job processing
  proxy | 2020/08/20 12:20:23 [002] GET https://api.github.com:443/repos/j0k3r/banditore
  proxy | 2020/08/20 12:20:23 * authenticating github api request
  proxy | 2020/08/20 12:20:23 [002] 200 https://api.github.com:443/repos/j0k3r/banditore
  proxy | 2020/08/20 12:20:23 [004] GET https://api.github.com:443/repos/j0k3r/banditore/git/refs/heads/master
  proxy | 2020/08/20 12:20:23 * authenticating github api request
  proxy | 2020/08/20 12:20:23 [004] 200 https://api.github.com:443/repos/j0k3r/banditore/git/refs/heads/master
  proxy | 2020/08/20 12:20:23 [006] GET https://api.github.com:443/repos/j0k3r/banditore/contents/composer.json?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
  proxy | 2020/08/20 12:20:23 * authenticating github api request
  proxy | 2020/08/20 12:20:23 [006] 200 https://api.github.com:443/repos/j0k3r/banditore/contents/composer.json?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
  proxy | 2020/08/20 12:20:23 [008] GET https://api.github.com:443/repos/j0k3r/banditore/contents/?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
  proxy | 2020/08/20 12:20:23 * authenticating github api request
  proxy | 2020/08/20 12:20:24 [008] 200 https://api.github.com:443/repos/j0k3r/banditore/contents/?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
  proxy | 2020/08/20 12:20:24 [010] GET https://api.github.com:443/repos/j0k3r/banditore/contents/composer.lock?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
  proxy | 2020/08/20 12:20:24 * authenticating github api request
  proxy | 2020/08/20 12:20:24 [010] 200 https://api.github.com:443/repos/j0k3r/banditore/contents/composer.lock?ref=1513a9b3155aca29c33dcf61e1e1010a6af49871
fetcher | INFO <job_44444580> Finished job processing
  proxy | time="2020-08-20T12:20:30Z" level=info msg="proxy starting" commit=c5f79a43943044613474bf0c4b592cd1afbc064e
  proxy | 2020/08/20 12:20:30 Listening (:1080)
updater | yarn config v1.22.4
updater | success Set "cafile" to "/etc/ssl/certs/ca-certificates.crt".
updater | Done in 0.03s.
updater | INFO <job_44444580> Starting job processing
updater | ERROR <job_44444580> Missing environment variable DEPENDABOT_REPO_CONTENTS_PATH
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:72:in `block in environment_variable'
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:71:in `fetch'
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:71:in `environment_variable'
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/update_files_job.rb:52:in `repo_contents_path'
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/update_files_job.rb:15:in `perform_job'
updater | ERROR <job_44444580> /home/dependabot/dependabot-updater/lib/dependabot/base_job.rb:29:in `run'
updater | ERROR <job_44444580> bin/update_files.rb:21:in `<main>'
updater | I, [2020-08-20T12:20:35.942127 #71]  INFO -- sentry: ** [Raven] Sending event 65c47c0b5ba54026b1e4894c61df4377 to Sentry
  proxy | 2020/08/20 12:20:56 [002] POST https://sentry.io:443/api/1425239/store/
  proxy | 2020/08/20 12:20:56 [002] 200 https://sentry.io:443/api/1425239/store/

Retrying and it's fixed.
Thanks!

[addersuk]

@feelepxyz this is working now on Ruby repos, thanks 👍

[mobilutz]

@feelepxyz Thanks for the very quick fix. dependabot is creating PRs now again!