Mattermost-server: pre-built binaries in source: app/plugin/jira/plugin_*

[onlyjob]

app/plugin/jira/plugin_* are pre-built binaries. It is a bad practice to commit pre-built files. Those files should be (re-)generated on build time.

[onlyjob]

Same applies for app/plugin/zoom/plugin_*.

[amyblais]

Hi @onlyjob - I apologize as I missed this Issue somehow - I will have an engineer take a look at this soon.

[jwilander]

Committing them hasn't caused any problems for us and it removes a build step that needs to be worried about as they are required for the Go code to compile. If it's causing issues for you or you have a good reason why we should remove them and add it as a build step instead, I'd love to hear it.

[onlyjob]

The problem is that your source distribution (tarball) is not pure source distribution.

This is not DFSG compliant. We have to throw away pre-built binaries and build everything from source in controlled environment. Since there is no straightforward way to build those plugins I just had to disable them...

[jwilander]

You should be able to easily build them yourself using the source:

• https://github.com/mattermost/mattermost-plugin-zoom
• https://github.com/mattermost/mattermost-plugin-jira

And packaging them like we do in our Makefile

https://github.com/mattermost/mattermost-server/blob/master/Makefile#L303

[onlyjob]

That describes current situation accurately yet the problem is presence of two large binary blobs committed to repository (not about how you make them). :)

[jwilander]

I'm not sure I see how it's a problem, just throw away the included plugin bits and re-build them yourself as part of your build process. Is it just that you need the source for the plugins included in the distributions?

[onlyjob]

That's exactly what I will have to do if committing binaries to Vcs does not bother you...

[grundleborg]

Hi @onlyjob . Thanks for reporting this and the other packaging/licensing related issues you have raised in other tickets. I'm a core developer at Mattermost and I'll be taking care of making sure these tickets get dealt with. I can't promise a quick resolution, particularly with the license issues as any changes will have to go through a bunch of people including lawyers, but I am hopeful that we can eventually bring all these tickets to a satisfactory conclusion that will keep the business and legal people here happy and also deal with the licensing and packaging concerns from a DFSG point of view.

On this issue specifically, we discussed this last week in our developers meeting and concluded that we should change this to remove the binary blobs from the mattermost-server repo and instead pull in the external plugin source repos as part of the build process. I think this change will address the issue fully, but please let me know if it doesn't. I can't promise an exact date by which it will happen but the decision has been agreed in the development team, so we will get it done.

[onlyjob]

Thanks @grundleborg.

FYI download on build is ever worse because it is a privacy breach and a potential security issue... It should be possible to reproducibly build in the secure/offline environment.

If external repository should be available on build time then perhaps using git submodules might be an answer. The only minor problem with submodules is that you'll have to (manually) generate comprehensive source tarball instead of relying on github tar.gz export which doesn't include submodules.

[grundleborg]

I think the plan is that building this repo will no longer include the plugins unless you also build them separately. However, we need to update our build server to build the plugins separately from source for the official release binaries.

[grundleborg]

This has now been fixed. Pre-compiled plugins are no longer part of this source repository.