Kubevirt: Make virtctl console wait for console to become ready

[booxter]

When I start a VM, I would like to immediately connect to its serial console to see all the output it may produce during boot. But I can't do it right away, because I get the error:

[ihar@ws kubevirt]$ ./cluster/virtctl.sh console ovm-cirros                                                                                                                                     
Escape sequence is ^]                                                                                                                                                                                       
                     Can't connect to websocket (400): Unable to connect to VM because phase is Scheduling instead of Running                                                                               

To mitigate the impact of it, I need to use a "while true; do virtctl console ...; done" command to get connected as soon as possible.

While I appreciate that the infrastructure to provide console output may be not ready at this point, I think virtctl could spin / wait for console to become ready and connect me to it at the earliest possibility, so that I don't miss some of boot messages.

[rmohr]

hm, since the cluster will try to schedule a workload indefinitely, that could hang indefinitely too. We may be able to handle that scenario with extra timeout flags, or something like that. Thoughts?

[booxter]

Having a conservative timeout (5 minutes?) and ability to abrupt waiting (Ctrl+C) would be a good solution.

[fabiand]

Timeout sounds like a good start.
A beefed up version could use: kubectl wait --for=condition=available --timeout=60s resource-string(deployment)/foo from here

[shiywang]

@fabiand I would like to stab at this if no one else

[fabiand]

/assign @shiywang

[fabiand]

Sounds good @shiywang

Do you need anything to get started?

[SchSeba]

Note to developer:

The command virtctl console is inside the pkg/virtcl/console
To make the console to wait until we get interrupt or time out we need to check the error coming back from the function in line 97

con, err := virtCli.VirtualMachineInstance(namespace).SerialConsole(vmi)

If the error is "Unable to connect to VM because phase is Scheduling instead of Running" wait for a few second and retry.

We also need to add a signal interrupt just like in line 109

go func() {
        interrupt := make(chan os.Signal, 1)
        signal.Notify(interrupt, os.Interrupt)
        <-interrupt
        close(stopChan)
    }()

Another thing is we need to do is create a tick object that will make the time out for our loop in case the vm is stuck in a loop and cant go up.

And the last thing is to add a --timeout=number to the console command.
We can add a default value of 5 min if the parameter is not used

[SchSeba]

Looks like no one take this issue so I will work on this one.

[garimavsharma]

I'm seeing the same error when running the kubevirt tutorial [1]

[root@student001 ~]# ./virtctl console testvm
Escape sequence is ^]
Can't connect to websocket (400): Unable to connect to VirtualMachineInstance because phase is Scheduling instead of Running

What release is this fixed in ?

[1] https://github.com/scollier/kubevirt-tutorial

[SchSeba]

Hi @garimavsharma

Please try to use the v0.8.0 release.