Mattermost-server: 5.21.0 won't build without makefile patch

[alerque]

Testing the recently tagged 5.21.0 release I'm having trouble building normally.

Cleaning
rm -Rf dist
go clean  -mod=vendor -i ./...
go: inconsistent vendoring in /home/caleb/projects/aur/mattermost/src/src/github.com/mattermost/mattermost-server:
        go.uber.org/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates go.uber.org/[email protected]
        go.uber.org/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates go.uber.org/[email protected]
        go.uber.org/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates go.uber.org/[email protected]
        golang.org/x/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates golang.org/x/[email protected]
        golang.org/x/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates golang.org/x/[email protected]
        golang.org/x/[email protected]: is explicitly required in go.mod, but vendor/modules.txt indicates golang.org/x/[email protected]

run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory

I gather the build wants something like this (but just this simple start this doesn't actually build):

--- Makefile    2019-12-28 12:38:41.471311251 +0100
+++ Makefile.new    2019-12-28 12:41:37.194305874 +0100
@@ -48,8 +48,8 @@
 # Golang Flags
 export GO111MODULE=on
 GOPATH ?= $(shell go env GOPATH)
-GOFLAGS ?= $(GOFLAGS:) -mod=vendor
+GOFLAGS ?= $(GOFLAGS:) -mod=mod
 GO=go
 DELVE=dlv
 LDFLAGS += -X "github.com/mattermost/mattermost-server/v5/model.BuildNumber=$(BUILD_NUMBER)"

Is this something than needs to be fixed in the packaging or is there a perhaps a caching problem with go having stashed something with builds from previous versions?

[agnivade]

This is a new feature with Go 1.14 that verifies the modules.txt in vendor mode. Can you try with master ? I remember fixing this.

[alerque]

@agnivade Maybe, I'll give it a shot.

[amyblais]

@agnivade Do we know if this is something concerning that also needs to be fixed for v5.21? Is there a PR / Jira ticket I can reference?

[alerque]

@agnivade It seems like I can build master normally, so yes this got fixed somewhere between the 5.21.0 tag and HEAD.

Might I suggest an urgent dot-release to account for this? It's going to be a blocker for Arch Linux packaging (which is what I was working on when I bumped into this) or at least require backporting whatever patches fixed it in order to release 5.21.

[agnivade]

I didn't realise Arch Linux has their own packaging for Mattermost. Is it a strict policy to build it always with the latest Go release ? FWIW, we haven't shifted our build pipelines to use 1.14 yet. Normal users can anyways get the 5.21 release from our servers.

In any case, I will leave the backport decision to @amyblais.

[alerque]

@agnivade Arch Linux has had packaging for Mattermost in AUR (Arch User Repository) since 0.6.0 and I would wager it is far more common to use this packaging that the "standard" upstream releases. There has been talk for a while now about moving it into the pre-built [community] package repository but that has been held up on some small issues.

Yes it is a strict policy to build with the latest Go :wink:! Arch is a rolling release and that's the only version of Go currently available. I think you'll also find this affects a number of other distros (Gentoo, OpenSUSE, Alpine, Nix, etc.).

Personally I think it's a mistake to think of people who use distro packages as "abnormal" users. In the Linux world normal users are those who reach for their distro packages first. Ubuntu's super long release cycles and locked versions of old packages has been pushing a lot of people away from that, but most other distros encourage native packages. If anything making sure things work for distro packagers would be the best way to ensure the most adoption and ease of use for normal end users.

[agnivade]

Thanks for the info ! Sorry if I implied users of distro packages as "abnormal", that wasn't my intention :) That is something I am very passionate about. From my knowledge of https://github.com/mattermost/mattermost-server/issues/4176, I just assumed that there was no distro packaging anywhere. Pardon my ignorance.

[alerque]

@agnivade Roger that.

Arch & Debian kind of come at this issue from opposite ends of the spectrum.

• The Debian philosophy is typically to cater to the _lowest_ common denominator. This gets aggravated even farther in Ubuntu because it is technically downstream of Debian plus they have LTS cycles. The net effect can be some staggeringly old libraries that seem to never get updated because they would break some other old software that's also held back because some other app can't handle newer libraries.

  Upshot: Everything is in a known state of of the gate, but newer software or anything under heavy development is often hard to get packaged at all.

• Rolling releases (Arch, PLD, Gentoo, OpenSUSE Tumbleweed) cater to to the _highest_ common denominator. As long as a basic core set of packages required to boot and run don't break, new libraries are typically packaged soon after they are released.

  Upshot: It's really easy to use the latest and greatest software, but some old projects that don't keep pace can eventually be kicked to the curb because it becomes too much of a burden to patch them to work with new libraries.

Different strokes ... as a software project unfortunately you have both problems to deal with.

[alerque]

@agnivade On the subject of "no distro packaging anywhere", Arch is not even the only distro with packaging. You can find a partial list on repology. Notably several BSD sources seem to have pretty up to date packaging and all the other *nix options are surprising current. Mac (Homebrew) and Windows (Scoop) packaging seems to have fallen behind. Also not shown on Repology but there are unofficial Debian/Ubuntu packages out there.

[alerque]

Any hope of a backport & patch version release or are Arch folks stuck waiting for the next full minor version release cycle?

[matoro]

Hey, I am also "packaging" this in the form of a Gentoo ebuild and have come across this issue with go 1.14.1. Will there be a point release that addresses this?

[amyblais]

This was discussed with devs and we feel that this is not a critical issue as this is related to dependencies and not the codebase.

Would you be open to updating to v5.22? We have just cut an RC-1 for v5.22 today.

[matoro]

This was discussed with devs and we feel that this is not a critical issue as this is related to dependencies and not the codebase.

Would you be open to updating to v5.22? We have just cut an RC-1 for v5.22 today.

Of course, when it gets released. Not a huge deal, just letting you know that there are downstream distro packagers that build from source (I also run in production at work from my own packages, so double invested in making sure things build properly).