Official-images: Support manifest lists / manifest-tool

[tianon]

Ideally we'd like to start experimenting with manifest lists (likely using Phil's tool; https://github.com/estesp/manifest-tool), initially for allowing for latest and other similar tags to redirect to either Linux or Windows based images automatically as appropriate, but eventually for multi-architecture support as well.

I'm filing this issue so we can discuss obstacles and work on overcoming them. :smile:

• [x] library file changes
  
  
  
  • https://github.com/docker-library/go-dockerlibrary/pull/2
  
  
• [x] bashbrew changes (new subcommand for assembling and pushing manifest list objects)
  
  
  
  • https://github.com/docker-library/official-images/pull/3058
  
  
• [x] signing system (https://github.com/docker-library/official-images/issues/1516) support for manifest lists
  
  
  
  • [ ] scanning system support for manifest lists (decision was made to separate this to be a separate non-blocking task)
  
  
• [x] unexpected manifest-tool bug pushing to library/ (https://github.com/docker-library/official-images/pull/3437)
• [x] update Jenkins to build/push amd64 (https://github.com/docker-library/oi-janky-groovy/blob/7516f1609f1260f94e70192238c588bfb4f8a88b/multiarch/vars.groovy#L4)
• [x] update Jenkins to push manifest lists (https://github.com/docker-library/oi-janky-groovy/tree/06a6a7de82617c024fdd36f958316857b48e268d/put-shared), taking care with "heavy hitters"

[friism]

@tianon thanks for getting this started.

Note that tags are not actually redirecting - the manifest list is served to the engine on pull, and then the engine chooses what to get.

I think it's probably fine to still have linux, debian, winddows and so on tags, as long as latest and other tags that don't specifically include OS are manifest lists and do the right thing. Once there's good UX for this, and if we get a way in the engine to overide what manifest list entry to pull, we should consolidate.

[tianon]

Right, I don't mean that they redirect per-se, but we do have to push _something_ somewhere first in order to include it in a manifest list (and can't push the full list until all the relevant artifacts it needs to reference are pushed), so pushing becomes a bit complex. Sorry for not being more clear on that point. :innocent:

[StefanScherer]

I haven't used the manifest-tool yet. But without knowing all the details I would like to have an easy "the best tools go out of your way" style approach for multi os/architecture:

1. Travis/Circle/Autobuild do a Linux build with
   
   
   
   • docker build -t foo/bar:version and
   
   
   • docker push foo/bar:version
   
   
2. AppVeyor does a Windows build with the same commands
   
   
   
   • docker build -t foo/bar:version and
   
   
   • docker push foo/bar:version
   
   
3. I build on a Linux ARM machine and do the same, and so on...

And I'm done.

My build agents does not depend on each other or have to be called in a specific order.
All the magic happens while pushing into Docker Hub.

[tianon]

@StefanScherer yeah, I guess by delaying, I was kinda hoping Docker would grow support in docker push itself to append to an existing manifest list, because that'd definitely simplify this problem, but as-is the tooling isn't quite there yet, so each of those pushes will overwrite the existing manifest and you'll be left with only the final push's contents available :innocent: :disappointed:

[tianon]

Here's an initial teaser of integrating https://github.com/docker-library/go-dockerlibrary/pull/2, at least for generating the doc block I outlined in https://github.com/docker-library/go-dockerlibrary/pull/2#issuecomment-277853597:

$ bashbrew cat hello-manifest
Maintainers: Tianon Gravi <[email protected]> (@tianon), Joseph Ferguson <[email protected]> (@yosifkit)
GitRepo: https://github.com/docker-library/hello-world.git

Tags: 7.6-linux, 7-linux, linux
SharedTags: 7.6, 7, latest
GitCommit: bdee60d7ff6b98037657dc34a10e9ca4ffd6785f
Directory: hello-world

Tags: 7.7-nanoserver, 7-nanoserver, nanoserver
SharedTags: 7.7, 7, latest
GitCommit: 1f13a5bc3b787747eeefb3b0051d8d29f851260d
Directory: hello-world/nanoserver
Constraints: nanoserver

$ bashbrew cat --format '{{ range .Manifest.GetSharedTagGroups }}- `{{ .SharedTags | join "`, `" }}`:{{ "\n" }}{{ range .Entries }}  - `{{ .Tags | first }}`{{ "\n" }}{{ end }}{{ end }}' hello-manifest
- `7.6`:
  - `7.6-linux`
- `7`, `latest`:
  - `7.6-linux`
  - `7.7-nanoserver`
- `7.7`:
  - `7.7-nanoserver`

[friism]

@tianon have you made more progress on this? Asking for friend. (@dmp42 actually)

[tianon]

This has been a little bit side-tracked by officializing the multi-architecture work (which is definitely highly related): https://github.com/docker-library/go-dockerlibrary/pull/10 :innocent:

[tianon]

Filed https://github.com/docker-library/official-images/pull/3009 to start bringing some bits of this together -- there's a stub in there for where we plan to implement manifest lists, but for now we wanted to get at least _this_ much in so we can start moving from https://github.com/docker-library/oi-janky-groovy/blob/2c031cae0bd68ed7679e952c7aeffeb7cb081044/multiarch/vars.groovy as the source-of-truth for our current multiarchitecture support to the image manifest files themselves being the source-of-truth, as they should be (and as they are for the rest of what we do). :+1:

[tianon]

See https://hub.docker.com/u/trollin/ for where I've temporarily been pushing manifest-list-ified versions of the official images. :+1:

[vielmetti]

@tianon - is there a straightforward way to inspect the Docker hub to see what architectures these sample images point to?

[tianon]

@vielmetti the best I've seen so far is manifest-tool inspect <image>; I _think_ Docker Cloud might show arches on hover, but it's hard to explore on Docker Cloud, and the hover text isn't very granular

[StefanScherer]

@vielmetti another way is to use curl + jq if you just want to see the os/arch pairs: curl -sL https://registry.hub.docker.com/v2/repositories/trollin/golang/tags/latest | jq .

[StefanScherer]

@tianon what about extending trollin/golang (and others where applicable) for windows/amd64 with nanoserver as the default? 😄

[tianon]

@StefanScherer yes, that's the goal -- the plan is to get the existing cross-arch-only work into library/xxx first, then we'll start playing more with SharedTags (it's implemented in bashbrew, and I've done some successful initial testing of it, but the current priority is the other multiarch work)

[tianon]

Updated the OP checklist with a bit of news! The signing system is now updated to handle manifest lists appropriately, and we've applied a minor patch to our builds of manifest-tool to account for https://github.com/estesp/manifest-tool/issues/47.

We're really close to finally supporting this properly! :metal: :+1: :birthday:

[tianon]

So it begins: https://github.com/docker-library/oi-janky-groovy/commit/477c8aa43e3d37842dcb281b191313c9fbc9528a

As long as all goes well, we should be able to finally close this issue very shortly!

[tianon]

This is pretty much done now! Just a few stragglers left. Success!

root@rpi3:~# docker run -it --rm debian:jessie-slim dpkg --print-architecture
Unable to find image 'debian:jessie-slim' locally
jessie-slim: Pulling from library/debian
01c8fcecd171: Pull complete 
Digest: sha256:01f3ae3155dcdd3c86470cbcb81b1215dff357329e6750b1b4cf663e3c00a099
Status: Downloaded newer image for debian:jessie-slim
arm64

[friism]

@tianon wait, so this is rolled out to all the library images?

[tianon]

@friism yessir! There are a few still in queue (which still includes ubuntu, among a couple others, since it's a "heavy hitter"), but the majority have been pushed and the rest will be pushed before EOD!

[friism]

@tianon including windoze?

[tianon]

@friism well, we're not using SharedTags yet, but the functionality is implemented and manifest lists for multiarch was the prerequisite, so now we can update hello-world:latest to run on Windows as well too (this was all groundwork for that :+1:)

[friism]

ah, got it - really looking forward to golang et.al. supporting this

[mkumatag]

hurray..! @friism

root@wave0m:~# docker run -ti golang:1.8.3 go version
go version go1.8.3 linux/ppc64le
root@wave0m:~#

[SvenDowideit]

@tianon congrats!