Config-help: Different concurrent PR limits per language / manager

Created on 5 Mar 2020 · 15Comments · Source: renovatebot/config-help

Which Renovate are you using?

WhiteSource Renovate App

"renovateVersion": "19.155.0"

Which platform are you using?

GitHub.com

Have you checked the logs? Don't forget to include them if relevant

DEBUG: PR hourly limit remaining: 5
DEBUG: Enforcing prConcurrentLimit (4)
DEBUG: 3 PRs are currently open
DEBUG: PR concurrent limit remaining: 1

What would you like to do?

I'd like to configure Renovate Bot with different concurrent PR limits per language / manager, i.e. allow 3 concurrent PRs for NuGet packages, but only a single one for npm packages.

I also have some other manager in the repo (e.g. Terraform) that I don't really care too much, but would like to have some global limit as well.

I tried the below, but it doesn't seem to work (i.e. more than one PR for npm is created):

{
  "extends": ["config:base"],
  "nuget": {
    "enabled": true,
    "registryUrls": [
      "https://api.nuget.org/v3/index.json"
    ]
  },
  "prHourlyLimit": 5,
  "prConcurrentLimit": 4,
  "js": {
    "managerBranchPrefix": "js-",
    "prConcurrentLimit": 1
  },
  "dotnet": {
    "managerBranchPrefix": "dotnet-",
    "prConcurrentLimit": 3
  },
 (...)
 }

pending-closure

Source

Greg-Smulko

All 15 comments

There's definitely no way to make limits per-manager today (e.g. additive), although I would have expected it to be possible to configure different limits per manager and get part of the effect. I suggest also trying with the manager names instead of language names (e.g. npm and nuget instead of js and dotnet) although it's unlikely to make a difference. Also note: if you group together dependencies with different limits then it will be non-deterministic as to which is chosen.

rarkins on 5 Mar 2020

Thanks for a super-prompt response!

So, basically, your suggesting that I should try:

remove "prConcurrentLimit": 4, from the above example

If it doesn't work, then:

move "prConcurrentLimit": 3 from dotnet section to the nuget section

Right?

Greg-Smulko on 5 Mar 2020

Also be aware that if you include config:base in presets then it also configures limits.

I think it would be a good step to remove the base limit and verify that the limit defined within the languages/managers sections is taking effect

rarkins on 5 Mar 2020

👍1

Also, do you testing against concurrent limit rather than hourly, because it's much easier to reproduce/verify against!

rarkins on 5 Mar 2020

👍1

Also be aware that if you include config:base in presets then it also configures limits.

So basically I should replace "extends": ["config:base"], with:

  "extends": [
    ":separateMajorReleases",
    ":combinePatchMinorReleases",
    ":ignoreUnstable",
    ":prImmediately",
    ":semanticPrefixFixDepsChoreOthers",
    ":updateNotScheduled",
    ":automergeDisabled",
    ":ignoreModulesAndTests",
    ":maintainLockFilesDisabled",
    ":autodetectPinVersions",
    "group:monorepos",
    "group:recommended",
    "helpers:disableTypesNodeMajor"
  ]

(copied from https://docs.renovatebot.com/presets-config/ and removed:

    ":prHourlyLimit2",
    ":prConcurrentLimit20",

)
?

Greg-Smulko on 5 Mar 2020

Yes, that would be effective to make sure you're not importing any limits that way.

rarkins on 5 Mar 2020

👍1

I'm afraid that neither of the below configs really work. :(
Could you advise @rarkins?

{
  "extends": [
    ":separateMajorReleases",
    ":combinePatchMinorReleases",
    ":ignoreUnstable",
    ":prImmediately",
    ":semanticPrefixFixDepsChoreOthers",
    ":updateNotScheduled",
    ":automergeDisabled",
    ":ignoreModulesAndTests",
    ":maintainLockFilesDisabled",
    ":autodetectPinVersions",
    "group:monorepos",
    "group:recommended",
    "helpers:disableTypesNodeMajor"
  ],
  "nuget": {
    "enabled": true,
    "registryUrls": [
      "https://api.nuget.org/v3/index.json"
    ]
  },
  "prHourlyLimit": 5,
  "js": {
    "managerBranchPrefix": "js-",
    "prConcurrentLimit": 1
  },
  "dotnet": {
    "managerBranchPrefix": "dotnet-",
    "prConcurrentLimit": 3
  }
  ]
}

{
  "extends": [
    ":separateMajorReleases",
    ":combinePatchMinorReleases",
    ":ignoreUnstable",
    ":prImmediately",
    ":semanticPrefixFixDepsChoreOthers",
    ":updateNotScheduled",
    ":automergeDisabled",
    ":ignoreModulesAndTests",
    ":maintainLockFilesDisabled",
    ":autodetectPinVersions",
    "group:monorepos",
    "group:recommended",
    "helpers:disableTypesNodeMajor"
  ],
  "nuget": {
    "enabled": true,
    "registryUrls": [
      "https://api.nuget.org/v3/index.json",
      "https://red-gate.pkgs.visualstudio.com/_packaging/Main/nuget/v2"
    ],
    "managerBranchPrefix": "nuget-",
    "prConcurrentLimit": 3
  },
  "npm": {
    "managerBranchPrefix": "npm-",
    "prConcurrentLimit": 1
  },
  "prHourlyLimit": 3
  ]
}

Greg-Smulko on 6 Mar 2020

Neither of the configs you have pasted in are valid JSON, so I have doubts in that aspect.

You're also mixing hourly and concurrent limits still.

Do this:

focus on concurrently PRs only
run at debug level logging
look for every reference to concurrent limiting in the logs
describe exactly what you expect to happen but isn't

Failing that, you'll need to reproduce the problem in a public repo on github.com so that one of us can take a look.

rarkins on 6 Mar 2020

Here is my full config:

{
  "extends": [
    ":separateMajorReleases",
    ":combinePatchMinorReleases",
    ":ignoreUnstable",
    ":prImmediately",
    ":semanticPrefixFixDepsChoreOthers",
    ":updateNotScheduled",
    ":automergeDisabled",
    ":ignoreModulesAndTests",
    ":maintainLockFilesDisabled",
    ":autodetectPinVersions",
    "group:monorepos",
    "group:recommended",
    "helpers:disableTypesNodeMajor"
  ],
  "ignorePaths": ["Source/Tests/Website.JavaScript.Unit.Tests"],
  "nuget": {
    "enabled": true,
    "registryUrls": ["https://api.nuget.org/v3/index.json"]
  },
  "js": {
    "managerBranchPrefix": "js-",
    "prConcurrentLimit": 1
  },
  "dotnet": {
    "managerBranchPrefix": "dotnet-",
    "prConcurrentLimit": 3
  },
  "masterIssue": true,
  "labels": ["renovate"],
  "automerge": false,
  "packageRules": [
    {
      "managers": ["npm"],
      "depTypeList": ["devDependencies"],
      "updateTypes": ["patch", "minor"],
      "automerge": true,
      "groupName": "npm dev dependencies"
    }
  ]
}

Even though there is no prHourlyLimit configured, in the logs there is still the mention of PR hourly limit and more than one js pr was created (even though there was "prConcurrentLimit": 1 set):

DEBUG: PR hourly limit remaining: 5
INFO: PR created(branch="renovate/dotnet-phantomjs-2.x")
INFO: PR created(branch="renovate/dotnet-selenium-packages")
INFO: PR created(branch="renovate/dotnet-system.io.compression-4.x")
INFO: PR created(branch="renovate/js-dagre-0.x")
INFO: PR created(branch="renovate/js-highcharts-react-official-2.x")
DEBUG: Reached PR creation limit or per run commits limit - skipping branch creation(branch="renovate/js-jsplumb-2.x")

The same happens with the below config:

{
  "extends": [
    ":separateMajorReleases",
    ":combinePatchMinorReleases",
    ":ignoreUnstable",
    ":prImmediately",
    ":semanticPrefixFixDepsChoreOthers",
    ":updateNotScheduled",
    ":automergeDisabled",
    ":ignoreModulesAndTests",
    ":maintainLockFilesDisabled",
    ":autodetectPinVersions",
    "group:monorepos",
    "group:recommended",
    "helpers:disableTypesNodeMajor"
  ],
  "ignorePaths": ["Source/Tests/Website.JavaScript.Unit.Tests"],
  "nuget": {
    "enabled": true,
    "registryUrls": ["https://api.nuget.org/v3/index.json"],
    "managerBranchPrefix": "nuget-",
    "prConcurrentLimit": 3
  },
  "npm": {
    "managerBranchPrefix": "npm-",
    "prConcurrentLimit": 1
  },
  "masterIssue": true,
  "labels": ["renovate"],
  "automerge": false,
  "packageRules": [
    {
      "managers": ["npm"],
      "depTypeList": ["devDependencies"],
      "updateTypes": ["patch", "minor"],
      "automerge": true,
      "groupName": "npm dev dependencies"
    }
  ]
}

Log:

DEBUG: PR hourly limit remaining: 5
INFO: PR created(branch="renovate/nuget-selenium-packages")
INFO: PR created(branch="renovate/npm-dagre-0.x")
INFO: PR created(branch="renovate/npm-highcharts-react-official-2.x")
INFO: PR created(branch="renovate/npm-jsplumb-2.x")
INFO: PR created(branch="renovate/npm-lodash-monorepo")
INFO: PR created(branch="renovate/npm-moment-duration-format-2.x")
INFO: PR created(branch="renovate/npm-prismjs-1.x")
INFO: PR created(branch="renovate/nuget-protobuf-net-2.x")
INFO: PR created(branch="renovate/npm-react-datepicker-2.x")
DEBUG: Reached PR creation limit or per run commits limit - skipping branch creation(branch="renovate/npm-react-expand-animated-1.x")

Greg-Smulko on 6 Mar 2020

I think we'll need a public repo with simplified config. Something that may be happening here is that the branches already exist from previous runs but not the PRs. In this case the limit is checked at branch creation, so if you have the branch already existing then the limit doesn't apply.

rarkins on 7 Mar 2020

Hi, I created two separate public repos to test both versions of the config (for managers and languages).

In both cases the PR limit doesn't seem to be taken into account at all.

Also, in both cases, there is a message "DEBUG: PR hourly limit remaining: 5", even though there is no prHourlyLimit specified anywhere.

First repo:

Second repo:

Thanks again in advance!

Greg-Smulko on 7 Mar 2020

👍1

@Greg-Smulko sorry for forgetting about this one. Is it still a concern for you and is there any new information to add?

rarkins on 20 Jul 2020

No worries @rarkins, frankly - I forgot about it a bit too.

Yes, it is still a concern and nothing new to add, but it starts to be more pressing, as we're finishing updating old dependencies and would like to enable Renovate auto-merge soon.

Your help is really appreciated, and if you need anything more from me at this point, please let me know.

Greg-Smulko on 20 Jul 2020

Also, in both cases, there is a message "DEBUG: PR hourly limit remaining: 5", even though there is no prHourlyLimit specified anywhere.

FYI this is a default setting of the hosted app. The open source software defaults to no limits but people installing it could end up with dozens or hundreds of PRs instantly which was a really bad experience, so we default to 5.

rarkins on 20 Jul 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed soon if no further activity occurs.