Stackage: Docker image not guaranteed to be available for LTS right after release

[koterpillar]

Soon as I've seen the announcement of LTS 13, I've added it to the tests of a package I maintain. However, as the tests use stack docker, they failed because the fpco/stack-build:lts-13.0 image is still not built.

It seems that the LTS release procedure does not require the Docker image to be fully built before releasing the news, or the LTS itself, so there is a delay before stack docker is usable.

While the interval when the Docker image isn't available might be short, it will still cause problems if any automated processes grab the latest LTS version from Stackage and try to build something using Docker.

The release procedure should wait for the Docker image to be built (step 4 in the release procedure) before publishing the new LTS version to Stackage and making a blog post.

[DanBurton]

You are correct that the docker image has been treated as a second-class artifact of the lts release procedure. It may be worthwhile to reassess the release procedure.

However, note that step 4 of the LTS release procedure does not refer to the fpco/stack-build:lts-13.0 image, but rather the commercialhaskell/stackage:lts13 image. This is not the image which contains built haskell packages. It is the baseline from which packages are then built (step 5).

[koterpillar]

I see, however, are you saying that fpco/stack-build:lts-13.0 contains built Haskell packages? I thought there are no packages in that image, just tools required by Stack to build.

[DanBurton]

Sorry, I'm not sure what's in fpco/stack-build. What I do know is:

• the image used by the Stackage build server for building packages for lts-13 is commercialhaskell/stackage:lts13
• fpco also puts out a docker image with lts packages built into it. I don't know what this one is called but I believe @borsboom can tell you more about it.

[koterpillar]

You might be referring to fpco/stack-full, which says it's no longer updated. fpco/stack-build only has the tools for Stack to build any Stackage package.

[borsboom]

The fpco/stack-build images are built using the Dockerfiles and build.sh script in https://github.com/commercialhaskell/stack/tree/stable/etc/dockerfiles/stack-build. This is currently done manually by me after the snapshot is released. I usually do so pretty quickly, but occasionally (such as during the last two weeks while power was out a lot and Internet access was spotty due to windstorms) there's a longer delay.

I agree that it might be nice if this was done by Stackage curators as part of the snapshot release process. There's no reason it can't be done before the snapshot is live on stackage.org. Up to now, the concerns have been kept separate since Stack and Stackage are separate projects (albeit closely related).

@DanBurton What do you think about Stackage curators taking this on? It's usually just a matter of running ./build.sh --push lts-X.Y. For a new LTS major version, there's a bit more work involved, but it's mostly updating a few versions of things and referencing a new commit of debian-bootstrap.sh. _Very_ occasionally a change needs to be made for a new minor LTS snapshot, but we're talking like once every two years or so as far as I can recall. I do update the images when a new stack version is released, but the Stack maintainers can keep doing that as part of the Stack release process so that wouldn't fall on Stackage curators to manage.

[mihaimaruseac]

Is this still an issue at the moment?

[koterpillar]

Not at the moment, as lts-14.11 image is available. I'm not sure if the process has been updated or not, as it's been a week since the LTS release.

[mihaimaruseac]

A new LTS was just published and the docker image is not present so this means we still have to move the docker image generation in the LTS script.

[borsboom]

The Dockerfiles and scripts for building new images are here: https://github.com/commercialhaskell/stack/tree/master/etc/dockerfiles/stack-build

For a new LTS minor release, you basically just run.

./build.sh --push lts && ./build.sh --push --small lts

In most cases all this does is add a new tag to an existing image, so it doesn't take long.

Note that, as the script is currently written, the LTS release must appear on stackage.org (although that could be changed).

I'd be happy to move the script and Docker files into the stackage repo, or Stackage curators can run it from where it is now.

I'd just have to give curators permission to push to the Docker repos, which I'm happy to do.

New major LTS versions are a bit more involved since they often involve new GHC versions and bootstrap scripts, but that work can be done ahead of the LTS release instead of, as is the current situation, afterward. I'm fine with continuing to do this work, I'd just needs a heads-up that a new LTS is coming soon once the debian-bootstrap script is stabilized.

[DanBurton]

debian-bootstrap as used by nightlies has likely stabilized to the point where we will use it as is for LTS 15. The timing of LTS 15 is uncertain but I imagine it will be sometime in Nov or Dec.

[borsboom]

I'd love to get routine building of the Docker images for minor LTSs off my plate (but still happy to help with major LTSs and anything non-routine).

What, concretely, can I do to hand this over, and also to make sure I'm included when planning for a new major release? Looking around the repo and docs, some ideas:

• Add building and pushing the docker image to build-next.sh. This script appears to be the main step in releasing a new LTS version (major or minor).

• Give curators (anyone who would push a new LTS) permission to push to the fpco/stack-build Docker repo. I would need Docker Hub usernames for any curators.

• Update documentation with new processes:

  • Curator onboarding: give access to push to the fpco/stack-build Docker repo.
  • How to prepare a Dockerfile for a new major LTS snapshot.
  • How to prepare a Dockerfile if a new minor LTS snapshot needs an additional system library.

• Possible ideas: move the Dockerfiles to this repo instead of commercialhaskell/stack/etc/dockerfiles/stack-build. This would avoid build-next.sh next having to get the dockerfiles from that repo, somehow.

I'd be happy to make PRs for any of these, but would need some help from a curator to test those PRs.

Does this sound like a reasonable plan? I can take the lead on these steps, but who could help me out if I'm not sure how to proceed or need some help with testing?

[snoyberg]

This is now complete. With the LTS 15.4 release, this was pushed automatically by the build server:

Thanks @jeffhappily!