Nixpkgs: [RFC][Fun] Prototype Nix 1.12 fetchFromGitHubWithUpdater

[dtzWill]

Description

We needed another of these, right?

I was exploring using the update.nix approach for helping automate checking for github releases,
and here's what I came up with. It can certainly be improved but I like it so thought I'd share :).

So there are a number of quirks:

• Requires 1.12
• (Ab)uses 1.12 goodness to pull github release information into a nix expression,
  filters and sorts it, and outputs the latest release.
• Uses unauthenticated github API so can trigger rate-limiting per-IP if used too much
• The "update script" generated is a constant, but will be changed once the cached github response expires (or you run with --option tarball-ttl 0)
• The implementation needs to go somewhere else, but wasn't clear where it belonged so it's in an obviously bad place for now :).

It seems to work surprisingly well considering I'm mostly just pointing Nix at GitHub's API :grin:.

cc https://github.com/NixOS/nixpkgs/pull/21734

Let me see!

• Here's the implementation
• branch using it for a few of my projects.

V2

Updates when script is executed, still mostly nix

• Implementation glue
• Actual update checking

[jtojnar]

This looks great, though I would love if it were little more composable:

• We might want to use the updateScript but not fetchFromGitHub, for example, when there is a release tarball.
• It would be also nice to support GitLab, or even selfhosted instances: https://gitlab.gnome.org/api/v4/projects/GNOME%2Fpolari/repository/tags.
• I have a similar patch for fetching version data from GNOME release server, it would be spectacular if the updateScript could take arbitrary version resolution function.

There are some corner-cases even on GitHub:

• Dash-to-Dock extension has two release branches. The one we want prefixes tags with extensions.gnome.org-v, the other uses no prefix.
• Until recently, GNOME projects used PROJECT_NAME_MAJOR_MINOR_PATCH format for tags, https://github.com/GNOME/libgdata/releases, some project might do that still.

[dezgeg]

I have been experimenting with auto-updaters every now and then (https://github.com/dezgeg/nixpkgs/blob/a44210758527fe1f8bdad13b58b3ada209470298/pkgs/common-updater/scripts/list-upstream-versions for my attempts).

In retrospect, writing all of this in shell was a horrible idea. But my overall idea is that the updaters should have 'phases' just like the Nixpkgs stdenv has, with sensible defaults/heuristics for the phases (just like stdenvs by default assumes the package is built in a autoconf-compatible way).

I've thought of (at least) the following separate phases:

• list all possible upstream versions
• pick the preferred latest version from a list of versions
  
  
  
  • e.g. various packages have the convention that even version numbers are stable ones and odd versions are unstable ones
  
  
• actually update the files in the repo
  
  
  
  • this one is separate so that a dry-run-like feature is easy to have
  
  

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.