K3d: [BUG] Containers restart policy is not set to "unless-stopped" by default

[fabricev]

Looking at #317 , it seems that containers restart policy should be set to unless-stopped by default, but it's actually empty.

What did you do

• How was the cluster created?

$ k3d cluster create test

What did you do afterwards?

$ docker inspect k3d-test-server-0 | jq '.[0].HostConfig.RestartPolicy'
{
  "Name": "",
  "MaximumRetryCount": 0
}

$ docker inspect k3d-test-serverlb | jq '.[0].HostConfig.RestartPolicy'
{
  "Name": "",
  "MaximumRetryCount": 0
}

What did you expect to happen

$ docker inspect k3d-test-server-0 | jq '.[0].HostConfig.RestartPolicy'
{
  "Name": "unless-stopped",
  "MaximumRetryCount": 0
}

```shell
$ docker inspect k3d-test-serverlb | jq '.[0].HostConfig.RestartPolicy'
{
"Name": "unless-stopped",
"MaximumRetryCount": 0
}

## Which OS & Architecture

Linux 5.4.0-53 x86_64
Ubuntu 20.04.1

## Which version of `k3d`

$ k3d version
k3d version v3.1.3
k3s version v1.18.9-k3s1 (default

## Which version of docker

```shell
$ docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:52 2020
 OS/Arch:           linux/amd64
 Experimental:      false
Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:20 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[fabricev]

Got a working fix.

[fabricev]

@iwilltry42 : not sure why the tests don't pass...
_make e2e_ works locally : make.log

[iwilltry42]

Hi @fabricev , thanks for opening this issue!
And thanks for finding this dumb mistake of mine :grimacing:
No clue, why I put that if-conditional there and then never actually set the checked variable anywhere :thinking:
~Can you please create a PR out of you fix?~ (got confused, sorry)
As to the tests: sometimes Drone fails to create the multi-server cluster in time and then the deadline exceeds.. happens, but that's not too bad. However, we may actually want to add tests to detect a "Crash Loop" instead of waiting forever for the server to come up if it fails on first try :thinking:

[fabricev]

Hi ! What about limiting the multiserver test to 2 servers instead of 3 ? I guess it covers the multi-server scenario as good as for 3 but with less resources on drone (assuming the wait fail is due to drone choking a bit on resources).

[iwilltry42]

The problem with that was that 2 servers isn't enough for reaching the HA quota for the embedded db (dqlite/etcd).

[fabricev]

The tests don't seem to loop if the cluster create fails, are you suggesting to try "cluster create" max n times before failing (in the multi server scenario or for single master as well ) ? I guess that that would be a separate enhancement, correct ?

[iwilltry42]

What I mean with the "Crash Loop" detection is, that with restart=unless-stopped, k3d may wait forever for the server to be ready (only stopped by the timeout in the tests) instead of exiting early, if the server fails to start (due to some error), because the container will be restarting but still tell k3d that it's running (thus k3d does not abort waiting for the log message).
I just tried it locally and got a container into a "Crash Loop" (i.e. continuously restarting) with the unless-stopped parameter. This is the state it reports:

"State": {
            "Status": "restarting",
            "Running": true,
            "Paused": false,
            "Restarting": true,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2020-11-24T12:26:52.241769434Z",
            "FinishedAt": "2020-11-24T12:26:52.283700328Z"
        },

So I guess we'll need to add a check for "Restarting": true, in the waitForLogMessage part.

[fabricev]

OK, now I get it... Indeed, the crash loop could be exited earlier: I've given it a crack in the PR as suggested with node.status.

[iwilltry42]

Thanks @fabricev !