Filebrowser: CI and release scheme

[eine]

A single master branch is used.

When regular commits are pushed to GitHub:

• CircleCI:
  
  
  
  • The backend (filebrowser/filebrowser) is built for a single platform.
  
  
  • A golang linter (gometalinter) is executed.
  
  
• hub.docker.com:
  
  
  
  • The built-in service is used to build a Docker image, tag it as latest and, if successful, publish it.
  
  

When tagged commits are pushed to GitHub:

• The three tasks above are executed, but the docker image is also tagged with the tag number.
• CircleCI:
  
  
  
  • The backend is built for multiple targets with goreleaser and a new release is pushed to GitHub Releases.
  
  

See:

• https://circleci.com/gh/filebrowser/filebrowser
  
  
  
  • https://github.com/filebrowser/filebrowser/blob/master/.circleci/config.yml
  
  
• https://hub.docker.com/r/hacdias/filebrowser/tags/
  
  
  
  • https://hub.docker.com/r/hacdias/filebrowser/builds/
  
  
• https://github.com/filebrowser/filebrowser/blob/master/.goreleaser.yml
• https://github.com/filebrowser/filebrowser/blob/master/Dockerfile

---

However, it seems that travis-ci.org was used until five months ago: https://travis-ci.org/filebrowser/filebrowser/builds @hacdias, why did you stop using travis-ci as a CI service?

---

The current approach has the following (minor) caveats:

• The backend is built multiple times: once in CircleCI for a single platform, several times in CircleCI if it is a release, and once in hub.docker.com if it is a release. Each of the procedures is completely decoupled from the others, so three build pipelines need to be separately maintained.
• The latest image published in hub.docker.com is not ensured to be functionally correct.
• The frontend must be built locally, in order to update rice-box.go with the artifacts.
• Tags are only applied to this repo (filebrowser/filebrowser). Since the organization was created the frontend (filebrowser/frontend) is not tagged anymore: https://github.com/filebrowser/frontend/releases

---

Moreover, the current proposal in #399:

• Builds the frontend, the backend and a docker image for each pushed commit in travis-ci.com. If successful, the image is pushed to hub.docker.com/filebrowser/filebrowser.
• Currently releases/tags are not considered.

So, the build task in CircleCI and the build task in hub.docker.com will be duplicated.

---

Since all of the steps can be executed in travis-ci.com, I propose to add missing features to #399, and drop CircleCI and hub.docker.com as CI services. These missing features are:

• [x] Add gometalinter to image filebrowser/dev.
  
  
  
  • [ ] Enable gosec with excluded options (those related to using MD5). See https://github.com/filebrowser/filebrowser/pull/399#issuecomment-410459808.
  
  
• [x] Add goreleaser to image filebrowser/dev.
• [x] Add a test stage in travis to execute linting before any build.
• [x] Add a deploy script to run goreleaser when tagged commits are pushed, and then upload the result to GitHub Releases.
• Build docker images:
  
  
  
  • [x] Tag images generated from regular commits as filebrowser/filebrowser:develop.
  
  
  • [x] Use goreleaser to build and tag images generated from tagged commits as filebrowser/filebrowser:<tag> and filebrowser/filebrowser:latest.
  
  
  • [ ] Extend goreleaser to build multiarch images. See goreleaser/goreleaser#707 and goreleaser/goreleaser#530. Close #440.
  
  
• [x] Modify publish.sh to tag both the backend repo and the frontend repo.

---

Alternatively, it seems that CircleCI allows to reproduce the builds locally (https://circleci.com/docs/2.0/local-cli/). Therefore, if anyone has enough knowledge to implement an equivalent workflow in CircleCI only, that would be equally acceptable.

[hacdias]

@1138-4EB I dropped Travis because it was a lot slower than Circle and in that time it was easier to get the different phases on Circle:

But I see now Travis is moving to GitHub Apps. We could try what you suggest and if it works, go for it. But we have to be careful about Caddy build server.

[eine]

@1138-4EB I dropped Travis because it was a lot slower than Circle and in that time it was easier to get the different phases on Circle.

I expected that. Indeed, CircleCI should still be much faster than Travis, because the latter needs to spin up a "virtual machine" for each stage and then run docker on top of it, while CircleCI is designed to run docker images directly.

However, I think that Travis is offering a better solution from the integration point of view:

• Stages provide a similar functionality to the phases in CircleCI.
• Being able to cache results from one stage to the next allows to build the frontend (rice-box) just once and share it with all the artifacts.
• Checks integration thanks to moving to GitHub Apps is a very interesting feature. See https://blog.travis-ci.com/2018-05-07-announcing-support-for-github-checks-api-on-travis-ci-com and e.g. https://github.com/Paebbels/JSON-for-VHDL/pull/8/checks
• I feel more comfortable with the Travis-GitHub Releases integration than having to create a GitHub API token for goreleaser to push artifacts.

BTW, I am setting goreleaser to build docker images with it: https://goreleaser.com/customization/#Docker It seems that, in the very near future, we will be able to create multiarch images: goreleaser/goreleaser#530 So we might close #440 with #399.

Anyway, should anyone know how to integrate everything in a CircleCI config which is equivalent to the .travis.yml I am writing in #399, please let me know.

[hacdias]

It sounds good to me. Thanks for all your work here 😃 It's being amazing!

[eine]

@hacdias just a quick update to let you know that this is almost ready. The only missing part is the Caddy build server issue. I registered, but there is so little info about the build process. So merging #399 will have to wait until we can find a solution.