Lego: Split into multiple Go modules?

[glasser]

Depending on the lego Go module ends up causing your project to depend on dozens of other modules which implement the APIs for different DNS providers (primarily). While their packages won't bloat your binary size if unused, it does cause your dependency graph to be larger, and if you do use one of those libraries it can affect what version of the library gets used in your project even if you don't use the relevant piece. (eg, when we added lego to our project it affected our AWS library version even though we aren't using either AWS provider.)

Would it be possible to split off providers into its own Go module? cmd would also need to be its own module because it depends on providers. That way, projects consuming lego as a library that implement their own provider or use one of the dependency-free built in providers won't end up with the 25+ provider libraries in their project.

[ldez]

Hello,

for now, we don't plan to do that, but we will keep that in mind.

[mholt]

I would be willing to help move the DNS providers to a separate repo where each one can be their own module.

Multi-module repositories are kind of tricky but once it's automated it shouldn't be too bad.

Caddy binaries are currently about 20 MB but without the DNS providers it would go down to around 12 MB, so they currently make up almost 50% of the size of the binary and the compile time...

[256dpi]

Another option would be to extract the "core" libraries into a separate core repository/module. This would be backwards compatible for users that already import this module. And then users that only require the library parts without commands and DNS providers can use the core module with a minimal set of dependencies.

[kroppt]

I considered using this as an alternative to using certbot. I found it annoying that installing certbot on Gentoo would also install 29 Python libraries.
I went to install the lego client from source, and there were 2GB of dependencies.
I would use the source packages and Go to cut that down if possible, but it's not.

[ldez]

2 GB of source's dependencies? It's 25,1 MB.

The dependencies in Go are managed really differently than Python but go modules system have some side effects.

[ldez]

To explain the lego context of the library if it is divided:

• CLI depends on core and providers
• providers depends on core

[kroppt]

~ > rm -rf go
~ > GO111MODULE=on go get -u github.com/go-acme/lego/v3/cmd/lego
~ > du -sh go/pkg/mod
2.0G    go/pkg/mod

This actually failed to compile. Nevertheless...

[ldez]

The go modules have some side effects because the real size of the source dependencies is lower:

$ go mod vendor
$ du -sh vendor/
31M     vendor/

[anderspitman]

@mholt it looks like Caddy/certmagic have settled on libnds for this, correct?

[mholt]

@anderspitman That's correct. They are general-purpose DNS provider packages that can also be used to set and delete TXT records for ACME challenges.