K3d: [BUG] image import broken

[stephenc]

What did you do

• How was the cluster created?

  • k3d cluster create

• What did you do afterwards?

  • k3d image import foo:latest
  • docker save -o foo.tgz foo:latest && k3d image import foo.tgz

What did you expect to happen

The image should be imported

Screenshots or terminal output

$ k3d image import foo:latest
INFO[0000] Loading images into 'k3s-default'            
INFO[0000] Starting k3d-tools node...                   
INFO[0000] Saving 1 image(s) from runtime...            
ERRO[0000] Failed to start exec process in node 'k3d-k3s-default-tools' 
ERRO[0000] Failed to save image(s) in tools container for cluster 'k3s-default' 
ERRO[0000] Failed to load images into cluster 'k3s-default' 
ERRO[0000] error during connect: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/exec/f366e5cfff66e69246ea66bd0c9e108b7060024420c40cba4b75e9293b3c3abf/start": net/http: HTTP/1.x transport connection broken: unsupported transfer encoding: "identity" 
INFO[0000] DONE

and

$ docker save -o foo.tgz foo:latest && k3d image import foo.tgz
INFO[0000] Loading images into 'k3s-default'            
INFO[0000] Saving 1 tarball(s) to shared image volume... 
INFO[0005] Importing images into nodes...               
INFO[0005] Importing images from tarball '/k3d/images/k3d-k3s-default-images-20200925122056-file-foo.tgz' into node 'k3d-k3s-default-server-0'... 
ERRO[0005] Failed to start exec process in node 'k3d-k3s-default-server-0' 
ERRO[0005] Failed to import images in node 'k3d-k3s-default-server-0' 
ERRO[0005] error during connect: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/exec/8b81f2b679b85797f79ddf2ed297efec1200b2d7fc90cb297525f56954634b83/start": net/http: HTTP/1.x transport connection broken: unsupported transfer encoding: "identity" 
INFO[0005] Removing the tarball(s) from image volume... 
ERRO[0005] Failed to start exec process in node 'k3d-k3s-default-tools' 
ERRO[0005] Failed to delete one or more tarballs from '[/k3d/images/k3d-k3s-default-images-20200925122056-file-foo.tgz]' 
ERRO[0005] error during connect: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/exec/6ca2d896ab177b5951ca634dac149b390ca56afa00115538dea6c8243b1571ef/start": net/http: HTTP/1.x transport connection broken: unsupported transfer encoding: "identity" 
INFO[0005] Removing k3d-tools node...                   
INFO[0005] Deleted k3d-k3s-default-tools                
INFO[0005] Successfully imported image(s)               
INFO[0005] DONE                                         

Which OS & Architecture

• OS-X

Which version of k3d

$ k3d version
k3d version v3.0.2
k3s version latest (default)

Which version of docker

• output of docker version and docker info

$ docker version
Client: Docker Engine - Community
 Azure integration  0.1.14
 Version:           19.03.13-beta2
 API version:       1.40
 Go version:        go1.13.14
 Git commit:        ff3fbc9d55
 Built:             Mon Aug  3 14:58:48 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13-beta2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.14
  Git commit:       ff3fbc9d55
  Built:            Mon Aug  3 15:06:50 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

$ docker info
Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.1-tp-docker)
  scan: Docker Scan (Docker Inc., v0.3.2)

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 321
 Server Version: 19.03.13-beta2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 11.71GiB
 Name: docker-desktop
 ID: ZBXY:TXOP:ZFS5:DX6X:7OSG:JVMC:K3PF:ZH7N:PGD3:XCBD:PWSW:NJKH
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: gateway.docker.internal:3128
 HTTPS Proxy: gateway.docker.internal:3129
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

[stephenc]

$ docker images
REPOSITORY                                    TAG                  IMAGE ID            CREATED             SIZE
rancher/k3s                                   latest               74d7efe64615        3 days ago          152MB
rancher/k3d-tools                             v3.0.2               a4dd32227058        4 days ago          9.09MB
rancher/k3d-proxy                             v3.0.2               9399341d75fc        4 days ago          43.2MB

[stephenc]

Possibly related: https://github.com/golang/go/issues/40735

Also I downgraded to 1.7.0 and I am getting a similar error there

[stephenc]

Very likely related. I just had the same issue with a homebrew tap of k3d 1.7.0 which was working a while back.

Then I modified the tap to buoild with [email protected] and import worked

Then I created a modified tap of k3d 3.0.2 compiled with [email protected] and this bug seems to be gone

Unclear who maintains the homebrew formula... but likely it should be building with a pinned version of go

Here's my formula if you are following this rabbit hole:

https://github.com/stephenc/homebrew-helm-legacy/blob/master/Formula/k3d%403.0.2.rb

brew install stephenc/helm-legacy/[email protected]

[iwilltry42]

Hi @stephenc , thanks for opening this issue! And even more thanks for solving it already :wink:
Really nice investigations you did there :+1:
There is no clear maintainer of that homebrew formula, I'm afraid, but please feel free to submit a PR there applying your changes!