Dependabot-core: Missed updates for a large Lerna repo

[RomainMuller]

Hello,

I am surprised @dependabot did not submit PR's for a number of package.json files at are monitored in our repository, while there are newer versions available for certain of our direct dependencies, as can be seen in aws/aws-cdk#3545.

I want to count on @dependabot for not having to manually cater for dependency upgrades, so I'd like to get an understanding of why I didn't get automated PRs for those dependencies (even after manually clicking "bump now").

Thank,
Romain.

[greysteil]

This is likely because Dependabot's Lerna handling isn't perfect - it looks like Dependabot hadn't created a PR for you in 19 days, which is definitely suspicious. CC @feelepxyz.

[feelepxyz]

@RomainMuller taking a look!

[MerlinMason]

Possible related issue... If I run yarn upgrade-interactive it suggests the following upgrades:

devDependencies
   name                          range   from       to      url
❯◯ @sentry/cli                   latest  1.47.0  ❯  1.47.1  https://docs.sentry.io/hosted/learn/cli/
 ◯ airbnb-prop-types             latest  2.13.2  ❯  2.14.0  https://github.com/airbnb/prop-types#readme
 ◯ conventional-changelog-cli    latest  2.0.21  ❯  2.0.23  https://github.com/conventional-changelog/conventional-changelog/tree/maste
r/packages/conventional-changelog-cli#readme
 ◯ copy-webpack-plugin           latest  5.0.3   ❯  5.0.4   https://github.com/webpack-contrib/copy-webpack-plugin
 ◯ css-loader                    latest  3.1.0   ❯  3.2.0   https://github.com/webpack-contrib/css-loader
 ◯ cypress                       latest  3.4.0   ❯  3.4.1   https://github.com/cypress-io/cypress
 ◯ eslint                        latest  5.16.0  ❯  6.1.0   https://eslint.org
 ◯ intercom-client               latest  2.10.6  ❯  2.11.0  https://github.com/intercom/intercom-node
 ◯ jsdoc-export-default-interop  latest  0.3.0   ❯  exotic  thanksbox/jsdoc-export-default-interop#master
 ◯ style-loader                  latest  0.23.1  ❯  1.0.0   https://github.com/webpack-contrib/style-loader
 ◯ webpack                       latest  4.37.0  ❯  4.39.1  https://github.com/webpack/webpack
 ◯ webpack-bundle-analyzer       latest  3.3.2   ❯  3.4.1   https://github.com/webpack-contrib/webpack-bundle-analyzer

 dependencies
   name                          range   from       to      url
 ◯ d3                            latest  3.5.17  ❯  5.9.7   https://d3js.org
 ◯ i18next                       latest  17.0.6  ❯  17.0.9  http://i18next.com
 ◯ i18next-xhr-backend           latest  3.0.0   ❯  3.1.1   https://github.com/i18next/i18next-xhr-backend
 ◯ js-cookie                     latest  2.2.0   ❯  2.2.1   https://github.com/js-cookie/js-cookie#readme
 ◯ query-string                  latest  6.8.1   ❯  6.8.2   https://github.com/sindresorhus/query-string#readme
 ◯ react-transition-group        latest  4.2.1   ❯  4.2.2   https://github.com/reactjs/react-transition-group#readme

However there's been no dependabot PRs raised.

[feelepxyz]

@MerlinMason could you send over the repo this is happening on? If private just ping it over to [email protected] and I'll take a look.

@RomainMuller still not sure what's gone wrong on your repo, definitely looks like we're missing updates though. Guess it could have something to do with the large number of tracked manifest files (currently tracking 260 files on your repo), will keep digging.

[MerlinMason]

Thanks, email sent :)

[feelepxyz]

@MerlinMason after digging a bit further it seems we are not dealing with dependencies that are shared across multiple manifests in a lerna/yarn workspace setup where some manifests have different versions.

Working on fix for this 🐛

[RomainMuller]

@feelepxyz - yeah this mono-repo covers all of AWS CDK's packages, which is 100+ packages, and then Dependabot tracks both the package.json and package-lock.json files, so we're definitely not making it easy on you guys here 😬

[feelepxyz]

@RomainMuller can confirm we are getting a bunch of time outs trying to generate updates for your repo but also found some cases where we are not finding an update but should be.

Starting to look at speeding up these jobs but might take a while to properly fix this as we've done a fair amount of work already to make our updates performant.

[RomainMuller]

@feelepxyz Yeah we have a manual process to update our dependencies at this stage. We'd like to stop manually caring & let dependabot take care of this for us... But it's absolutely fine if it takes some time for you to figure out how to resolve those issues...

[Konard]

I have similar issues with my repositories:
https://github.com/linksplatform/Reflection.Sigil was updated and published to https://www.nuget.org/packages/Platform.Reflection.Sigil/0.0.4

But all direct dependants are still at 0.0.3.
https://github.com/linksplatform/Converters
https://github.com/linksplatform/Numbers
https://github.com/linksplatform/Unsafe

Even if I manually "bump" them at https://app.dependabot.com/accounts/linksplatform

[greysteil]

I'm going to rename this issue, as it refers to debugging for a JS customer using Dependabot on a (very) large monorepo.

@Konard if Dependabot is missing .NET updates that's likely a different issue. It also looks like it created a PR for you at @Konard it looks like Dependabot created a PR just fine for you at https://github.com/linksplatform/Converters/pull/10 so I'm guessing you got this fixed?

[Konard]

Wow, it is working now, thank you. I did nothing. But what was the problem?

[greysteil]

We didn't do anything on our side. 🤷‍♂

[RomainMuller]

I seem to have gotten a couple of updates popping on the aws/aws-cdk repository today. Have you been working towards a fix/mitigation for this problem, or was this just luck?

[greysteil]

I'm afraid that's likely to be luck 😬

[RomainMuller]

Seems like updates that affect a small number of packages in our mono-repo have more chances of success than those of broader reach. If that's of any help...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs within seven days. Thank you for your contributions.