Podman: `podman import` from a tarball doesn't preserve metadata

Created on 3 Nov 2019  路  3Comments  路  Source: containers/podman

/kind bug

Description

I am creating a container image using Buildah and setting several Label and Annotation values. After committing this image, it has every piece of metadata - labels and annotations - set properly. After buildah push oci-archive: one of the blobs in the TAR has all the metadata fields. Doing podman import of the TAR, adds the image to Podman without any metadata.

Steps to reproduce the issue:

  1. Create and commit the image using Buildah:
$ buildah from --name import-test fedora
Getting image source signatures
Copying blob d318c91bf2a8 done
Copying config f0858ad3fe done
Writing manifest to image destination
Storing signatures
import-test
$ buildah config --label foo=bar import-test                      
$ buildah config --annotation author="Eveline Raine" import-test
$ buildah commit --rm import-test import-test:latest
Getting image source signatures
Copying blob 2ae3cee18c8e skipped: already exists
Copying blob 5f70bf18a086 done
Copying config 1033ef08c1 done
Writing manifest to image destination
Storing signatures
1033ef08c1bc2f4bf3dba209319e0bebd33cad0f9bbeb83307f96e73e232626c
  1. Inspect the committed image:
    (All metadata is here)
$ podman image inspect import-test:latest
[
    {
        "Id": "1033ef08c1bc2f4bf3dba209319e0bebd33cad0f9bbeb83307f96e73e232626c",
        "Digest": "sha256:25f8f3db17a85465fef5596ab9a2f1bb8326e3e1e1c4d8ee661cd5b3c01d47c5",

        "RepoTags": [
            "localhost/import-test:latest"
        ],
        "RepoDigests": [
            "localhost/import-test@sha256:25f8f3db17a85465fef5596ab9a2f1bb8326e3e1e1c4d8ee661cd5b3c01d47c5"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2019-11-03T15:36:45.248850516Z",
        "Config": {
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "DISTTAG=f31-updates-candidatecontainer",
                "FGC=f31-updates-candidate",
                "FBR=f31-updates-candidate"
            ],
            "Cmd": [
                "/bin/bash"
            ],
            "Labels": {
                "foo": "bar",
                "maintainer": "Clement Verna <[email protected]>"
            }
        },
        "Version": "",
        "Author": "",
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 201095982,
        "VirtualSize": 201095982,
        "GraphDriver": {
            "Name": "overlay",
            "Data": {
                "LowerDir": "/home/evelineraine/.local/share/containers/storage/overlay/2ae3cee18c8ef9e0d448649747dab81c4f1ca2714a8c4550eff49574cab262c9/diff",
                "UpperDir": "/home/evelineraine/.local/share/containers/storage/overlay/36c18e6ac7c72103785b871be314bb848d51a10fd5c90a9cf9cc4bbe30037dba/diff",
                "WorkDir": "/home/evelineraine/.local/share/containers/storage/overlay/36c18e6ac7c72103785b871be314bb848d51a10fd5c90a9cf9cc4bbe30037dba/work"
            }
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:2ae3cee18c8ef9e0d448649747dab81c4f1ca2714a8c4550eff49574cab262c9",
                "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
            ]
        },
        "Labels": {
            "foo": "bar",
            "maintainer": "Clement Verna <[email protected]>"
        },
        "Annotations": {
            "author": "Eveline Raine"
        },
        "ManifestType": "application/vnd.oci.image.manifest.v1+json",
        "User": "",
        "History": [
            {
                "created": "2019-01-16T21:21:55.569693599Z",
                "created_by": "/bin/sh -c #(nop)  LABEL maintainer=Clement Verna <[email protected]>",
                "empty_layer": true
            },
            {
                "created": "2019-09-27T21:21:07.784469821Z",
                "created_by": "/bin/sh -c #(nop)  ENV DISTTAG=f31-updates-candidatecontainer FGC=f31-updates-candidate FBR=f31-updates-candidate",
                "empty_layer": true
            },
            {
                "created": "2019-10-29T03:23:37.355187998Z",
                "created_by": "/bin/sh -c #(nop) ADD file:298f828afc880ccde9205fc4418435d5e696ad165e283f0530d0b1a74326d6dc in / "
            },
            {
                "created": "2019-10-29T03:23:37.695123423Z",
                "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/bash\"]",
                "empty_layer": true
            },
            {
                "created": "2019-11-03T15:36:45.248850516Z",
                "created_by": "/bin/sh"
            }
        ]
    }
]
  1. Save image as an OCI archive using buildah:
$ buildah push import-test:latest oci-archive:import-test-latest.tar:latest
Getting image source signatures
Copying blob 2ae3cee18c8e done
Copying blob 5f70bf18a086 done
Copying config 1033ef08c1 done
Writing manifest to image destination
Storing signatures
  1. Untar and inspect the contents of the archive:
    (can still see the metadata in one of the blobs)
$ tar -xvf import-test-latest.tar
blobs/
blobs/sha256/
blobs/sha256/1033ef08c1bc2f4bf3dba209319e0bebd33cad0f9bbeb83307f96e73e232626c
blobs/sha256/1dbcab28ce46b65c0174e5e82658492107396fead31e9144c343e6bc96e471c7
blobs/sha256/315e857f93fb25bd2aec03f586c1fd88b614dc6fe9592efaa0e856ec73c6c617
blobs/sha256/6ccc4524a72eeb5401a4e349d3c87a4f3819d49d3d004ab6444490d75f31c343
index.json
oci-layout
$ cat blobs/sha256/1033ef08c1bc2f4bf3dba209319e0bebd33cad0f9bbeb83307f96e73e232626c | jq
{
  "created": "2019-11-03T15:36:45.248850516Z",
  "architecture": "amd64",
  "os": "linux",
  "config": {
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "DISTTAG=f31-updates-candidatecontainer",
      "FGC=f31-updates-candidate",
      "FBR=f31-updates-candidate"
    ],
    "Cmd": [
      "/bin/bash"
    ],
    "Labels": {
      "foo": "bar",
      "maintainer": "Clement Verna <[email protected]>"
    }
  },
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:2ae3cee18c8ef9e0d448649747dab81c4f1ca2714a8c4550eff49574cab262c9",
      "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
    ]
  },
  "history": [
    {
      "created": "2019-01-16T21:21:55.569693599Z",
      "created_by": "/bin/sh -c #(nop)  LABEL maintainer=Clement Verna <[email protected]>",
      "empty_layer": true
    },
    {
      "created": "2019-09-27T21:21:07.784469821Z",
      "created_by": "/bin/sh -c #(nop)  ENV DISTTAG=f31-updates-candidatecontainer FGC=f31-updates-candidate FBR=f31-updates-candidate",
      "empty_layer": true
    },
    {
      "created": "2019-10-29T03:23:37.355187998Z",
      "created_by": "/bin/sh -c #(nop) ADD file:298f828afc880ccde9205fc4418435d5e696ad165e283f0530d0b1a74326d6dc in / "
    },
    {
      "created": "2019-10-29T03:23:37.695123423Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/bash\"]",
      "empty_layer": true
    },
    {
      "created": "2019-11-03T15:36:45.248850516Z",
      "created_by": "/bin/sh"
    }
  ]
}
  1. Import this archive into Podman:
$ podman import import-test-latest.tar localhost/import-test:imported
Getting image source signatures
Copying blob e7eef1a8bab6 done
Copying config 0f9e3896bb done
Writing manifest to image destination
Storing signatures
0f9e3896bbe4da3dce0896220ea3f18474a4c8dc63f0c18b9181bc38381382e2
$ podman image list
REPOSITORY                                      TAG        IMAGE ID       CREATED          SIZE
localhost/import-test                           imported   0f9e3896bbe4   11 minutes ago   70.2 MB
localhost/import-test                           latest     1033ef08c1bc   17 minutes ago   201 MB
docker.io/library/fedora                        latest     f0858ad3febd   5 days ago       201 MB
registry.fedoraproject.org/f31/fedora-toolbox   latest     3a3fb0a29265   5 weeks ago      446 MB
  1. Inspect the imported image:
    (Metadata is gone)
$ podman image inspect import-test:imported
[
    {
        "Id": "0f9e3896bbe4da3dce0896220ea3f18474a4c8dc63f0c18b9181bc38381382e2",
        "Digest": "sha256:903f87d4945ff0a0245071e9193fc05e31ab59134d61f6d2e0a71c03b5476791",
        "RepoTags": [
            "localhost/import-test:imported"
        ],
        "RepoDigests": [
            "localhost/import-test@sha256:903f87d4945ff0a0245071e9193fc05e31ab59134d61f6d2e0a71c03b5476791"
        ],
        "Parent": "",
        "Comment": "imported from tarball",
        "Created": "2019-11-03T17:42:22.690014839+02:00",
        "Config": {},
        "Version": "",
        "Author": "",
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 70182995,
        "VirtualSize": 70182995,
        "GraphDriver": {
            "Name": "overlay",
            "Data": {
                "UpperDir": "/home/evelineraine/.local/share/containers/storage/overlay/e7eef1a8bab68cdd0b75d00deafb454f2a13c8eceef085ef1d72a1a81ee25a5f/diff",
                "WorkDir": "/home/evelineraine/.local/share/containers/storage/overlay/e7eef1a8bab68cdd0b75d00deafb454f2a13c8eceef085ef1d72a1a81ee25a5f/work"
            }
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:e7eef1a8bab68cdd0b75d00deafb454f2a13c8eceef085ef1d72a1a81ee25a5f"
            ]
        },
        "Labels": null,
        "Annotations": {},
        "ManifestType": "application/vnd.oci.image.manifest.v1+json",
        "User": "",
        "History": [
            {
                "created": "2019-11-03T17:42:22.690014839+02:00",
                "created_by": "/bin/sh -c #(nop) ADD file:e7eef1a8bab68cdd0b75d00deafb454f2a13c8eceef085ef1d72a1a81ee25a5f in /",
                "comment": "imported from tarball"
            }
        ]
    }
]

Describe the results you received:

Imported image is stripped of it's metadata - labels and annotations.

Describe the results you expected:

Imported image preserving all the metadata that was present in the committed image by Buildah.

Output of podman version:

Version:            1.6.2
RemoteAPI Version:  1
Go Version:         go1.13.1
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.13.1
  podman version: 1.6.2
host:
  BuildahVersion: 1.11.3
  CgroupVersion: v2
  Conmon:
    package: conmon-2.0.2-1.fc31.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.2, commit: 186a550ba0866ce799d74006dab97969a2107979'
  Distribution:
    distribution: fedora
    version: "31"
  IDMappings:
    gidmap:
    - container_id: 0
      host_id: 2505
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 2505
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  MemFree: 362450944
  MemTotal: 3137253376
  OCIRuntime:
    name: crun
    package: crun-0.10.2-1.fc31.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.10.2
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 2
  eventlogger: journald
  hostname: fossil.raine.ai
  kernel: 5.3.7-301.fc31.x86_64
  os: linux
  rootless: true
  slirp4netns:
    Executable: /usr/bin/slirp4netns
    Package: slirp4netns-0.4.0-20.1.dev.gitbbd6f25.fc31.x86_64
    Version: |-
      slirp4netns version 0.4.0-beta.3+dev
      commit: bbd6f25c70d5db2a1cd3bfb0416a8db99a75ed7e
  uptime: 74h 16m 27.83s (Approximately 3.08 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /home/evelineraine/.config/containers/storage.conf
  ContainerStore:
    number: 1
  GraphDriverName: overlay
  GraphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-0.6.5-2.fc31.x86_64
      Version: |-
        fusermount3 version: 3.6.2
        fuse-overlayfs: version 0.6.5
        FUSE library version 3.6.2
        using FUSE kernel interface version 7.29
  GraphRoot: /home/evelineraine/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 4
  RunRoot: /run/user/2505
  VolumePath: /home/evelineraine/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.6.2-2.fc31.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):

Buildah version:

Version:         1.11.3
Go Version:      go1.13
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.1-dev
CNI Spec:        0.4.0
libcni Version:  
Git Commit:      
Built:           Thu Jan  1 03:00:00 1970
OS/Arch:         linux/amd64
kinbug
Source
picture evelineraine

All 3 comments

The podman import command is the reverse of the podman export command, which only deals with the contents of a container's filesystem, discarding everything else.

If you use podman pull oci-archive:import-test-latest.tar:latest, it'll reconstruct the stored image using all of the information that push saved, give or take possible format conversion to or from OCI format, and that you might need to re-tag the image to give it the right name.

nalind picture nalind on 4 Nov 2019
❤1

Oh... I didn't think about pull for image tarballs 馃槄
Everything works just right now. Thank you!

evelineraine picture evelineraine on 4 Nov 2019

you could also use podman load

ThoTischner picture ThoTischner on 19 Aug 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

oblitum picture oblitum  路  5Comments
NooneXe picture NooneXe  路  3Comments
undecaf picture undecaf  路  3Comments
raukadah picture raukadah  路  4Comments
yangm97 picture yangm97  路  5Comments