Brew: Automatically create pull requests based on Homebrew/livecheck/Repology

[MikeMcQuaid]

Homebrew/livecheck provides various automated ways of detecting formulae updates.

Repology provides an API that provides details on whether a Homebrew package is outdated e.g. https://repology.org/metapackage/boost/versions

A separate Ruby application that can be trivially deployed on e.g. Heroku should be built to make use of these checks, verify that the results seem correct and open a pull request with relevant metadata on Homebrew/homebrew-core.

No need to say "I'm planning on working on this" or similar in this issue but feel free to open a PR that doesn't work (yet!) and ask for help.

[BenMusch]

Hey Mike! I'm interested in this concept, but it seems like the work that goes into making a separate application (creating a repo, hosting the app, storing secret keys) would be something better suited for a member of the homebrew team to work on. Is that correct, or is it possible for an outside contributor to work on this?

[scpeters]

My first thought when I read this was that it could be a command-line application. For example, brew livecheck already has a command-line interface, and brew bump-formula-pr can create pull requests from the command-line, so I imagine that a solution could work in that same style, which wouldn't require storing secret keys or hosting an app.

[BenMusch]

That makes sense, thanks! I'll look into this and comment back here if I make significant progress on this concept

[MikeMcQuaid]

@scpeters Adding this logic to e.g. brew bump-formula-pr (or another command that wraps/feeds it) would be a good starting point, I agree. Eventually I do think an application that does this without human intervention would be preferable.

@BenMusch Given the above I think you should be able to get started. It's worth noting that even an application wouldn't need to store any secret keys; you could currently write an application to submit these pull requests as yourself as there's no special permissions required to submit PRs. Given e.g. Heroku's free tier it should be pretty easy to get started on a simple server for this in a public repo both of which could be migrated to Homebrew at a later stage.

[jeduardo824]

Could I start to work on this?

[zachauten]

@jeduardo824 Sure! I’ve been looking into it as well, let me know if you have any questions/ideas!

[MikeMcQuaid]

Yes please to either of you!

[zachauten]

@MikeMcQuaid What should the threshold for an upgrade be? I imagine there may be situations where we might not want to automatically bump major versions of a package, and bumping hundreds of packages every time there’s a new patch version could create a lot of noise.

[MikeMcQuaid]

What should the threshold for an upgrade be?

A new version being available.

bumping hundreds of packages every time there’s a new patch version could create a lot of noise.

I'm not sure I understand this, can you elaborate?

[zachauten]

Sorry, never mind that part about patch versions; My thinking was that creating a large amount of PRs at once to update formulae by a small amount would be annoying to manage, but that would only be the first time you run it.

[larson-carter]

Ok, I just jumped into this issue. So you want to automatically open a PR against Homebrew/homebrew-core if it detects a package being out of date?
@MikeMcQuaid

EDIT: typo

[MikeMcQuaid]

@larson-carter I would start with the "a separate application should be built to make use of these checks" and not automatically open any PRs yet.

[larson-carter]

@larson-carter I would start with the "a separate application should be built to make use of these checks" and not automatically open any PRs yet.

Ok, I'll start to work on it!

[larson-carter]

@MikeMcQuaid I have started work on this automated checker. Please see the link attached here with my current progress. Do you have a tutorial on how to update packages manually?

https://github.com/larson-carter/Homebrew-Package-Checker

[MikeMcQuaid]

Nice start @larson-carter and sorry for the delay! I've edited the issue above to note this will need to be written in Ruby for us to maintain, unfortunately. brew bump-formula-pr is the best way to submit package version updates. Thanks!

[larson-carter]

@MikeMcQuaid

Hey no worries about the delay.

I'll re write it in Ruby, then we can probably work something out since it will be nested under a completely new repo also for transferring ownership to the homebrew org. I'll also take into account about > brew bump-formula-pr

[zachauten]

I have a pretty basic solution here: https://github.com/zachauten/bump. It collects formulae that match between repology and livecheck, plus livechecks that aren't guessed, and runs bump-formula-pr on them.

Still lots of edge cases & peculiarities that could be ironed out.

[MikeMcQuaid]

@zachauten Nice! Is that something you're using yourself for version bumps? If not, that would be an interesting starting point.

[zachauten]

@MikeMcQuaid I've used it on one formula at a time, testing. Going to try a "real" run tonight.

[MikeMcQuaid]

@zachauten Nice, let us know how it goes!

[zachauten]

@MikeMcQuaid So the run yesterday picked up 91 outdated formulae that matched between livecheck and repology. About half passed the CI.

[alecclarke]

@MikeMcQuaid I'm not sure if you're still looking for a solution on this one, but I created a proof-of-concept project/repo that can leverage GitHub action's scheduling trigger to update Livecheck formulae automatically.

Currently the code invokes brew bump-formula-pr using the --dry-run flag so as to not make any noise from creating PRs. Also, I'm using a .brew_livecheck_watchlist list to keep the action run time low, but that can easily be replaced with adding --all to the brew livecheck call.

Would this approach be acceptable to what you're looking? If so I'm happy to keep working away on this.

[MikeMcQuaid]

@alecclarke Yeh, we're still looking for a solution. Can you link to some output of your GitHub Action?

[alecclarke]

@MikeMcQuaid sorry for the delay.

Here's a link to the latest Action run. You can see the output under Run brew bump section.

This is running off the formulae listed in .brew_livecheck_watchlist. Some of these have open PRs to bump the version already and that's also shown in the Run brew bump output.

[ran-dall]

@MikeMcQuaid We could probably extend the ML system we use for cask to support something like this. The only thing I see a problem with is that it isn't Ruby-based but more C and R based more than anything else. However, it can do a lot more than just make a PR when there's a new version of something available.

[larson-carter]

@MikeMcQuaid

We could possibly just expand on this project: https://github.com/MLH-Fellowship/0.0.1-API-Parser

All we need to do is have the bot open a PR against the formulae.

[BaffourAdu]

@MikeMcQuaid My team and I part of the MLH Fellowship (https://github.com/orgs/MLH-Fellowship) have come up with a possible solution to this issue. It checks for outdated formulas and automatically creates pull requests to update them. The solution works although there are a few edge cases and issues we need to address. You read more about the project here https://github.com/MLH-Fellowship/0.0.1-API-Parser

[MikeMcQuaid]

Nice work so far @alecclarke! Are there any incremental changes to Homebrew/brew or Homebrew/homebrew-livecheck that you can think of making based on your work so far?

@MikeMcQuaid We could probably extend the ML system we use for cask to support something like this. The only thing I see a problem with is that it isn't Ruby-based but more C and R based more than anything else. However, it can do a lot more than just make a PR when there's a new version of something available.

@ran-dall We would want something that's open source, can run on GitHub Actions and we have multiple maintainers familiar enough with the languages used to be able to have a non-trivial bus factor. I think that may rule out the current ML system, sorry. It may be there's room to improve and combine some elements on both systems?

We could possibly just expand on this project: https://github.com/MLH-Fellowship/0.0.1-API-Parser

@larson-carter @BaffourAdu Nice work so far! I'll review this and provide feedback through the MLH channels.

[alecclarke]

Thanks, @MikeMcQuaid !

Are there any incremental changes to Homebrew/brew or Homebrew/homebrew-livecheck that you can think of making based on your work so far?

Would it be reasonable to add more details to the json output of Livecheck?

Specifically, if we could add the new formula version's url to the output, or in the cases where a git tag based strategy is used, add the revision + tag - that would remove the need for brew bump to access Formula and be run as an external brew command.

[MikeMcQuaid]

Would it be reasonable to add more details to the json output of Livecheck?

I think so! @samford has been working on similar stuff and they may have some ideas here?

[samford]

I've been improving the debug and JSON output in Homebrew/homebrew-livecheck#860, so you may be interested in that.

In addition to the existing changes in that PR, I'm in the process of adding the following information to the JSON output (to bring it in line with the information that PR presents in the debug output):

• URL
  
  
  
  • The unmodified URL from the formula or livecheckable
  
  
  • The processed URL (i.e., the output from preprocess_url), if it differs
  
  
  • Any generated URL used in the strategy, if it differs
  
  
• Strategy used
• Regex (from the livecheckable or a default regex used in a strategy, if any)

This requires passing information from the strategy methods (in livecheck/livecheck_strategy.rb) back to latest_version() (in livecheck/heuristic.rb) and back to print_latest_version() (in cmd/livecheck.rb). Currently this doesn't happen, so I'm having to modify everything in this chain to make this information available where the JSON is generated in print_latest_version() and provided to livecheck() (which outputs the JSON).

Beyond this, it may be worth mentioning that I'm keeping the default JSON output as minimal as possible and requiring use of another flag in addition to --json to get the lengthier output including the above information. I'm currently using the --verbose flag in the PR for this but there's debate about whether --debug --json would make more sense.

There are also some changes to field names in the aforementioned PR, so if you're currently working with livecheck's JSON output in some way, you'll want to pay attention to these forthcoming changes.

---

If there's any additional information that's available to livecheck and you could benefit from having in the JSON output, just let me know and I can work on including it.

I'm unsure of precisely what's meant by "the new formula version's url". If you mean the URL for the archive file of the new version (e.g., https://example.com/example-1.2.3.tar.gz), this isn't currently possible with livecheck. We only capture the version text like 1.2.3 and not a full archive URL, so we would have to make some fundamental changes and modify hundreds of livecheckables to be able to provide this information. If I've understood this correctly, I can try to iron out how to make it work and it may be something that our GSoC student (Nanda) could work on.

That said, this isn't going to be possible for 100% of formulae, let alone 100% of the formulae for which livecheck currently works. For some of the checks, the URL of the new archive isn't available where we're checking (whereas the version number is). For formulae with very consistent URLs and filenames, you could consider substituting the new version string in the URL for the interim time and checking whether that URL is good (e.g., using a HEAD request). [Some URLs redirect to a mirror, so the criteria for "good" is more than just a 200 response.]

Regarding the Git strategy, it uses git ls-remote --tags and parses the tags from the output. The commit hash is also available in this output, so I could modify the git_tags() function to parse it as well if that's what you mean by "revision".

[alecclarke]

@samford thanks for the details!

I'm unsure of precisely what's meant by "the new formula version's url". If you mean the URL for the archive file of the new version (e.g., https://example.com/example-1.2.3.tar.gz), this isn't currently possible with livecheck.

Yes, that's what I was wondering about. In the brew bump command I've been working on I was using the strategy of simply substituting the new version string in the URL (like you described) but I was hoping that their might be a more robust way of getting that value. If you think there's value identifying the new formula version's URL and adding it to the Livecheck output, I'd be happy to contribute to that effort.

Regarding the Git strategy, it uses git ls-remote --tags and parses the tags from the output. The commit hash is also available in this output, so I could modify the git_tags() function to parse it as well if that's what you mean by "revision".

Yes, that's what I mean by revision. If the Livecheck output included the tag and commit hash that would make bumping formulae with git based downloaded strategies much easier!

While it looks like you are already well underway on making some of these changes, let me know if/where I can help out with this work!

[MikeMcQuaid]

Beyond this, it may be worth mentioning that I'm keeping the default JSON output as minimal as possible and requiring use of another flag in addition to --json to get the lengthier output including the above information. I'm currently using the --verbose flag in the PR for this but there's debate about whether --debug --json would make more sense.

I'd be tempted to make the JSON always be the same unless it's significantly more expensive to generate. If so, --verbose would be preferable to --debug (which has other effects).

While it looks like you are already well underway on making some of these changes, let me know if/where I can help out with this work!

@samford Would be cool to get a WIP PR open at some point and then @alecclarke and others could take a look!