Nixpkgs: geckodriver update to 0.26

[mpickering]

geckodriver is currently stuck at 0.22 because the packaging changed to remove the Cargo.lock file. The recommended way to get the source now is to checkout the mozilla-central megarepo and build the testing/geckodriver folder.

It doesn't seem like a good idea to clone the whole mozilla megarepo to get the source so perhaps the only way to unstick this issue is to package the binary instead.

See #35301 for some discussion about this already.

Any opinion @das-g ?

[das-g]

Any opinion @das-g ?

IMO upstream should try to be more source-based downstream friendly, but I don't see how that mine opinion helps us here in any way. :stuck_out_tongue_closed_eyes:

It doesn't seem like a good idea to clone the whole mozilla megarepo to get the source so perhaps the only way to unstick this issue is to package the binary instead.

Is that the only feasible way? Packaging binaries when the source code is available always feels kinda icky to me. Could we e.g. fetch just the relevant information from the mega-repo (Is that only the Cargo.lock file, or is there more to it?) and use it to check out the right revision of the geckodriver repo?

[das-g]

Btw., should upstream issue https://github.com/mozilla/geckodriver/issues/1427 be re-opened (or, probably better, re-filed with the current versions) or is it already foreseeable that "just use the mega-repo and build testing/geckodriver with mach" would be the answer to the question asked there?

[mpickering]

I just checked again and the "source" from the github release doesn't actually contain the source code for geckodriver but just the files from the repo. So I don't know of a way of getting the source without cloning the megarepo.

I stumbled across this blog post where the problem of cloning mozilla-central is discussed - https://geekyshacklebolt.wordpress.com/2018/02/14/how-to-clone-a-large-mercurial-repository/

The mozilla megarepo is about 40gb to clone so that isn't an option and because mercurial doesn't support shallow clones, it seems the only option left is to package the binary.

[matt@nixos:~/nixpkgs]$ ls /tmp/nix-build-geckodriver-0.26.0-vendor.drv-0/source/
CODE_OF_CONDUCT.md  CONTRIBUTING.md  ISSUE_TEMPLATE.md  README.md

[mpickering]

The Arch maintainers just don't even bother with the "lock file".

https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/geckodriver

[mpickering]

There is an alternative which builds from the mercurial repo - https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=geckodriver-hg

[7c6f434c]

Note there is _also_ a mozilla-central mirror on GitHub: https://github.com/mozilla/gecko-dev/tree/master/testing/geckodriver

Unfortunately, hg convert which would be the most realistic real solution does not work with remote Mercurial repositorues.

[mpickering]

It doesn't appear that there is a Cargo.lock file in the github mirror, or upstream either. So even if we somehow manage to get the source, we won't be able to use the rust builder to build it without creating our own Cargo.lock?

[mpickering]

I looked at how homebrew got the source for geckodriver and it seems you can just download some of the subfolders. So I tried to package it but got stuck because apparently the lock file is not synced up with the cargo.toml file. If anyone wants to try to finish this then I have pushed my WIP branch here:

https://github.com/mpickering/nixpkgs/commit/c2d9fc943def903108400447cb7959cc56e589a9

I am going to see how easy it is to package the binary.

[symphorien]

how about just generating one lock file and adding it as a patch ? Random example:
https://github.com/NixOS/nixpkgs/blob/1ea4896ae9cb4cafdfc023bd57084200d685d829/pkgs/applications/science/misc/rink/default.nix#L13

Pro: every other solution seems so painful
Con: cargo must manually be run on the source by the maintainer on each update to get a new cargo.lock file.

[7c6f434c]

cargo must manually be run on the source

… which might be scriptable. And maybe some updates would even tolerate an older Cargo.lock

[das-g]

Pinging @jraygauthier, who's currently listed as maintainer of the geckodriver derivation.

[mpickering]

I didn't know how to generate a lock file and given packaging the binary was so simple I stopped trying to build geckodriver from source.

[symphorien]

To generate a lock file:
1/ clone the source
2/ run cargo check
Even if it fails to compile, it will create a lock file.

[jraygauthier]

@das-g : Nothing much to contribute / add here, if anyone want to jump in as maintainer, this is fine by me. Added this package as a dependency to selenium for some client side testing I had to support for some project. I no longer use the package and when I first added it, it was pretty much the monkey see monkey do way (never had the opportunity to dabble with rust and its package manager). It looks however like a good thing to keep in nixpkgs either built from source or using the prebuilt binary.

[alyssais]

cargo generate-lockfile is the command to just generate a lockfile and do nothing else.