Nixpkgs: caddy: add all addons

[davidak]

compile caddy with all addons listed on the download page.

Only middleware handlers which are invoked from the Caddyfile will be chained in, so small Caddyfiles are very fast and efficient.

source: Directive section on https://caddyserver.com/docs/caddyfile

a later improvement could be to select just the addons you want like vim_configurable.

@angus-g

[angus-g]

This does sound like a good idea to improve the usefulness of caddy, but it's quite non-trivial from what I can see. It looks like there's some code generation required (in https://github.com/caddyserver/caddydev/blob/master/caddybuild/build.go) to insert the custom directives into caddy.

If that's the case, it would probably be easier to start with desired addons and do the code generation at build time. I expect this would need a bit more Nix code than currently exists for the package in order to support hashes for caddy and all addons, then modifying and building the source after this. I'm not great with the language just yet, so I'm not even sure of the feasibility of this. (Anything's possible, right?)

[davidak]

I found a Dockerfile where someone installed caddy with the git addon. That is also the one i want to use.

https://github.com/jumanjihouse/docker-caddy/blob/master/Dockerfile.build#L34

You could write the nix-expression like for a general program, but the go infrastructure is different.
You should ask in the IRC or Mailinglist how to realize that best. I'm also not that experienced with nix, never packaged a go program.

[davidak]

you can override build phases like described here https://nixos.org/nixpkgs/manual/#ssec-controlling-phases

for example the preBuildPhases like here: https://github.com/NixOS/nixpkgs/blob/dfc6d42860b41042deeccb487c73adb138a36e48/pkgs/top-level/go-packages.nix#L642

then you can use caddyext install git in the build phase.

if you are experienced with go, you can do that. i could also figure it out and create a PR. how you like :)

[davidak]

we can also create an additional package of caddyext and use it as caddy buildInputs.

but i see 2 problems:

• the hash of the addon source is not checked
• reproducible builds are impossible since the source of the addons change over time

[0xABAB]

If the source of the addons changes over time, I would try to convince upstream to change their ways and otherwise completely ignore this project, for a lack of professional software development practices.

Sometimes "gratis" software is even too costly.

[davidak]

@0xABAB you may misunderstood me. caddy has it's own "package manager" for addons.

if i execute caddyext install git now and again in one year, i will get a different version. if we build our nix package this way, it will not be reproducible.

i asked the developer of caddyext if it can install from local directory. then we can clone the addons repository and checkout a specific version.

Caddy is free software.

[davidak]

in version 0.9 addons will be plugins. i havn't looked what that means for building from source.

https://forum.caddyserver.com/t/caddy-0-9-beta-version-available/146?u=matt

we should wait for 0.9 to integrate the fix for this issue into that.

[joepie91]

Any updates on this?

[bricewge]

It is now quite easy to add plugins in caddy: https://github.com/mholt/caddy/wiki/Plugging-in-Plugins-Yourself
But I don't know how to make the build reproducible. It seems that at most the major version can be specified , not a specific version or a commit. For example adding _ "gopkg.in/restic/caddy.v0" will for the moment use the v0.2 as explained here.

[bricewge]

I managed to add the restic plugin to caddy by adding thoses lines to the current derivation:

    preConfigure = ''
        substituteInPlace ./caddy/caddymain/run.go \
        --replace '// This is where other plugins get plugged in (imported)' \
        '_ "github.com/restic/caddy"'
    '';
    goDeps = ./deps.nix;

And adding deps.nix:

[
  {
    goPackagePath = "github.com/restic/caddy";
    fetch = {
      type = "git";
      url = "https://github.com/restic/caddy";
      rev = "v0.2.0";
      sha256 = "0wjhbnm405vdpf3jwi9dzhz6nd5rhmxqibsa1nx2iyq43bc3p6sk";
    };
  }
  {
    goPackagePath = "github.com/restic/rest-server";
    fetch = {
      type = "git";
      url = "https://github.com/restic/rest-server";
      rev = "v0.9.3";
      sha256 = "0f6i952iy4arnc96wpayvh9wa1mdw7vprwwbnjxwj09jvifqd0hp";
    };
  }
]

So it is now really easy to add some plugins to caddy! Whoever I don't know in which form this configuration should be available in NixOS? Maybe having two versions would be the easiest:

• caddy as it currently is, without any plugins
• caddy-full with all the plugins availaible

[davidak]

I think that 2 versions would be OK.

Another option would be to enable each plugin like described for Chromium and Firefox here https://nixos.wiki/wiki/Chromium

[JakeStanger]

The above method for me isn't working. Has anybody had any luck or got a better approach?

[CMCDragonkai]

According to the https://github.com/caddyserver/caddy/wiki/Plugging-in-Plugins-Yourself

The current derivation is at: https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/caddy/default.nix

All we need to do is is something like:

{ caddy }:

caddy.overrideAttrs (attrs: {
  preBuild = ''
    cat << EOF > caddy/main.go
    package main
    import "github.com/caddyserver/caddy/caddy/caddymain"
    import _ "github.com/techknowlogick/caddy-s3browser"
    func main() {
      caddymain.EnableTelemetry = false
      caddymain.Run()
    }
    EOF
  '';
})

Which I override the preBuild to write our own main.go to be used.

You can see that I added in:

    import _ "github.com/techknowlogick/caddy-s3browser"

However that dependency is not being brought into the buildGoModule.

Normally with buildGoPackage, I'd add extra things to deps.nix. However here, this is buildGoModule, so how do we add caddy-s3browser as an extra dependency to buildGoModule?

[CMCDragonkai]

There's a overrideModAttrs in the buildGoPackage function. However I can't seem to access that in the resulting derivation. As in pkgs.caddy.override cannot access overrideModAttrs and neither can pkgs.caddy.overrideAttrs.

Instead it seems like it is needed to override the go-modules derivation located in pkgs.caddy.go-modules.

No documentation on how to override the go-modules derivation to add an extra go package into the go-modules.

[CMCDragonkai]

I think what needs to be done is that the go-modules.buildPhase needs to be overridden. Like a preBuild hook.

Because that's when go mod download is called.

Either that or the go.mod must have a patch applied to add that in prior to the source being used. I'm wondering if go.sum also needs a patch.

This does not work even if I update go.mod and go.sum. I think this is not working for overriding caddy. It seems some thing more drastic required.

The buildGoModule builder needs to have a better way of adding extra go dependencies.

[CMCDragonkai]

Abandoned trying to do overrides. I just had to create my own Go package to do this. But in the future, caddy needs some work to be able to compile plugins more easily like how firefox/chrome does its plugins.

[ooesili]

@CMCDragonkai Hey there! I couldn't get overrides to work either, but I ended up with this and thought I'd share. I ran into some issues with nix caching old versions of the module after tweaking the derivation but leaving modSha256 the same as before. Nix thought it had already built those modules even though the inputs technically changed. It tripped me up for a while, maybe it happened to you too? Anyways here's my local fork of caddy:

with pkgs; buildGoModule rec {
  pname = "caddy";
  version = "1.0.3";

  goPackagePath = "github.com/caddyserver/caddy";

  subPackages = [ "caddy" ];

  src = fetchFromGitHub {
    owner = "caddyserver";
    repo = pname;
    rev = "v${version}";
    sha256 = "1n7i9w4vva5x5wry7gzkyfylk39x40ykv7ypf1ca3zbbk7w5x6mw";
  };
  modSha256 = "1xs5qf89sddj21vig4lmhdfkyccf2alrd39915spz845xhv3pl8w";

  overrideModAttrs = (old: {
    preBuild = ''
      go mod edit -require=github.com/tarent/[email protected]
      go mod edit -require=github.com/BTBurke/[email protected]+incompatible
    '';
  });

  preBuild = ''
      cat << EOF > caddy/main.go
      package main

      import (
        "github.com/caddyserver/caddy/caddy/caddymain"

        _ "github.com/BTBurke/caddy-jwt"
        _ "github.com/tarent/loginsrv/caddy"
      )

      func main() {
        caddymain.EnableTelemetry = false
        caddymain.Run()
      }
      EOF
  '';

  meta = with stdenv.lib; {
    homepage = https://caddyserver.com;
    description = "Fast, cross-platform HTTP/2 web server with automatic HTTPS";
    license = licenses.asl20;
    maintainers = with maintainers; [ rushmorem fpletz zimbatm ];
  };
}

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[davidak]

this is still important