Renovate: Allow option to not scan API for all files

[leipert]

First of all: Thank you for the great project and really readable source code.

What would you like Renovate to be able to do?

I work for GitLab and am evaluating Renovate to update dependencies for GitLab CE. We have ~14000 files in the repository, and the getFileList method makes over 140 API requests to load the files. We only have one package.json and one yarn.lock. Even if we add the renovate.config files, it is an overkill!

Describe the solution you'd like
Either

• support a root-path-only option, which just scans the root dir. The GitLab API allows reducing the search to one branch, or
• support an option to directly specify the files it should download and disable scanning

Describe alternatives you've considered

• None so far. I have changed getFileList to return ['package.json', 'yarn.lock'] and it works wonderfully 👍

Additional context

[rarkins]

Hi @leipert, thanks for the positive feedback. Your suggestions sounds reasonable - I'll have a little think about it and then decide on the most appropriate approach to implement. Right now I like the idea of scanning the root directory only.

BTW you mention:

The GitLab API allows reducing the search to one branch

But I think we're doing that already? Instead the problem seems to be that we're recursing to the entire directory structure of that one branch.

Also, FYI I have been tracking a feature request for the GitLab API that would improve Renovate rebasing: https://gitlab.com/gitlab-org/gitlab-ce/issues/45035

[rarkins]

I think the best approach would be to allow disabling of recursive fetching in the file list. e.g. we would use this endpoint in the GitLab CE case: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/repository/tree?ref=master&per_page=100

[leipert]

Ah, sorry. I was mistaken. I meant reducing to one path:

GET /projects/:id/repository/tree?path=<path>

https://docs.gitlab.com/ee/api/repositories.html#list-repository-tree

---

Maybe it is also feasible to disable scanning somehow, like just doing it for the first time when onboarding (awesome feature btw!). That way you'd do 3 or 4 requests to get the config, and only do the scanning if necessary.

[rarkins]

@leipert no problem. Anyway, I should be able to have a new release out with this tomorrow, and you can hopefully go back to using the mainline build

[renovate-bot]

:tada: This issue has been resolved in version 12.63.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[rarkins]

@leipert please try running [email protected] or later while setting RENOVATE_DISABLE_FILE_RECURSION=true in env. It should skip the recursion flag when pulling the file list from GitLab and hence enable root-level file discovery only. For now I will keep this as an undocumented env variable until confirmation it's working as you need plus we are sure it's the best approach.

[leipert]

@rarkins It works lovely! Thank you. Seems a bit hacky, but I like it ;)

Regarding https://gitlab.com/gitlab-org/gitlab-ce/issues/45035, I'll leave a comment there to underline it's importance. Thanks for the quick adjustment!

[rarkins]

I've created an issue to have it added to permanent/documented config, now that you've confirmed it working: https://github.com/renovatebot/renovate/issues/2187