containerd crashes when address field for gRPC sockets is too long

[elliott-davis]

When the file path for address in [grpc] or [debug] is longer than 67 characters, containerd fails to start. I discovered this behavior when I was calling dockerd directly with --exec-root but I can reproduce with just containerd given the config file dockerd writes out.

Steps to reproduce the issue:

1. Create a config file like:
   ```root = "/mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon"
   state = "/hab/svc/builder-worker/data/947098412724445184/dockerd/var/run/docker/containerd/daemon"
   no_subreaper = false
   oom_score = -500

[grpc]
address = "/foo/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/dockerd/var/run/docker/containerd/docker-containerd.sock"
uid = 0
gid = 0

[debug]
address = "/foo/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/dockerd/var/run/docker/containerd/docker-containerd-debug.sock"
uid = 0
gid = 0
level = "debug"

[metrics]
address = ""

[cgroup]
path = ""

[plugins]
[plugins.linux]
shim = "docker-containerd-shim"
runtime = "docker-runc"
runtime_root = "/mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/runc"
no_shim = false
shim_debug = true

2. launch containerd

docker-containerd -c containerd.toml --log-level=debug

I get the following output:

INFO[0000] starting containerd module=containerd revision=cfd04396dc68220d1cecbe686a6cc3aa5ce3667c version=v1.0.2
DEBU[0000] changing OOM score to -500 module=containerd
INFO[0000] loading plugin "io.containerd.content.v1.content"... module=containerd type=io.containerd.content.v1
INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"... module=containerd type=io.containerd.snapshotter.v1
WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module=containerd
INFO[0000] loading plugin "io.containerd.snapshotter.v1.overlayfs"... module=containerd type=io.containerd.snapshotter.v1
INFO[0000] loading plugin "io.containerd.metadata.v1.bolt"... module=containerd type=io.containerd.metadata.v1
WARN[0000] could not use snapshotter btrfs in metadata plugin error="path /mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module="containerd/io.containerd.metadata.v1.bolt"
INFO[0000] loading plugin "io.containerd.differ.v1.walking"... module=containerd type=io.containerd.differ.v1
INFO[0000] loading plugin "io.containerd.gc.v1.scheduler"... module=containerd type=io.containerd.gc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.containers"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.content"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.diff"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.events"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.healthcheck"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.images"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.leases"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.namespaces"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.snapshots"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.monitor.v1.cgroups"... module=containerd type=io.containerd.monitor.v1
INFO[0000] loading plugin "io.containerd.runtime.v1.linux"... module=containerd type=io.containerd.runtime.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.tasks"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.version"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.introspection"... module=containerd type=io.containerd.grpc.v1
containerd: failed to get listener for debug endpoint: listen unix /foo/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/var/run/docker/containerd/docker-containerd-debug.sock: bind: invalid argument

I expected to get:

INFO[0000] starting containerd module=containerd revision=cfd04396dc68220d1cecbe686a6cc3aa5ce3667c version=v1.0.2
DEBU[0000] changing OOM score to -500 module=containerd
INFO[0000] loading plugin "io.containerd.content.v1.content"... module=containerd type=io.containerd.content.v1
INFO[0000] loading plugin "io.containerd.snapshotter.v1.btrfs"... module=containerd type=io.containerd.snapshotter.v1
WARN[0000] failed to load plugin io.containerd.snapshotter.v1.btrfs error="path /mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module=containerd
INFO[0000] loading plugin "io.containerd.snapshotter.v1.overlayfs"... module=containerd type=io.containerd.snapshotter.v1
INFO[0000] loading plugin "io.containerd.metadata.v1.bolt"... module=containerd type=io.containerd.metadata.v1
WARN[0000] could not use snapshotter btrfs in metadata plugin error="path /mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" module="containerd/io.containerd.metadata.v1.bolt"
INFO[0000] loading plugin "io.containerd.differ.v1.walking"... module=containerd type=io.containerd.differ.v1
INFO[0000] loading plugin "io.containerd.gc.v1.scheduler"... module=containerd type=io.containerd.gc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.containers"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.content"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.diff"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.events"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.healthcheck"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.images"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.leases"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.namespaces"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.snapshots"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.monitor.v1.cgroups"... module=containerd type=io.containerd.monitor.v1
INFO[0000] loading plugin "io.containerd.runtime.v1.linux"... module=containerd type=io.containerd.runtime.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.tasks"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.version"... module=containerd type=io.containerd.grpc.v1
INFO[0000] loading plugin "io.containerd.grpc.v1.introspection"... module=containerd type=io.containerd.grpc.v1
INFO[0000] serving... address="/bar/a/var/run/docker/containerd/docker-containerd-debug.sock" module="containerd/debug"
INFO[0000] serving... address="/bar/a/var/run/docker/containerd/docker-containerd.sock" module="containerd/grpc"
INFO[0000] containerd successfully booted in 0.002605s module=containerd

Which I can get with the following config file:

root = "/mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/containerd/daemon"
state = "/hab/svc/builder-worker/data/947098412724445184/dockerd/var/run/docker/containerd/daemon"
no_subreaper = false
oom_score = -500

[grpc]
address = "/bar/a/var/run/docker/containerd/docker-containerd.sock"
uid = 0
gid = 0

[debug]
address = "/bar/a/var/run/docker/containerd/docker-containerd-debug.sock"
uid = 0
gid = 0
level = "debug"

[metrics]
address = ""

[cgroup]
path = ""

[plugins]
[plugins.linux]
shim = "docker-containerd-shim"
runtime = "docker-runc"
runtime_root = "/mnt/hab/svc/builder-worker/data/947098412724445184/dockerd/var/lib/docker/runc"
no_shim = false
shim_debug = true

containerd github.com/containerd/containerd v1.0.2 cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
```

Related: https://github.com/habitat-sh/builder/pull/356

[estesp]

On 64-bit Intel Linux the path length limit of a UNIX socket is 108, effectively followed a UNIX "standard" for portability between the BSDs, Solaris, etc.

Of course it would make sense not to crash here and output an error message if the length is too long.

/usr/include/x86_64-linux-gnu/sys/un.h:

/* Structure describing the address of an AF_LOCAL (aka AF_UNIX) socket.  */
struct sockaddr_un
{
    __SOCKADDR_COMMON (sun_);
    char sun_path[108];     /* Path name.  */
};

[elliott-davis]

oh wow - TIL. I also just now saw where docker has hit this before https://github.com/moby/moby/pull/13408

It would be really nice to throw a more friendly error but I also understand it's just a limitation of the underlying lib.

[mlaventure]

Just to be fair, it doesn't crash, it exits with a not so helpful error ;-)

I thought I had changed this error message, but it seems I only did it for the unix socket used to talk to the shim.

I'll make a quick PR for it.

[stevvooe]

I also think we have some redundancy in the socket name in moby. var/run/docker/containerd/docker-containerd-debug.sock could be reduced to var/run/docker/containerd/debug.sock. Can we do that without impact?

[mlaventure]

@stevvooe moby would have to check for both for a while, since updates do not necessarily requires to shutdown containerd