Kubevirt: Rescheduling issue with VM/Pod

[x8091]

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug

What happened: When the worker node (which hosts the VM/pod) became Not Ready state (by poweroff for example), the VM didn't get rescheduling to another nodes, it just keep staying in Terminating state.

What you expected to happen: The VM terminate successfully and re-create on another node.

How to reproduce it (as minimally and precisely as possible): A cluster with 3 worker nodes, all of them can create VMs by kubevirt (tested by using node selector).

Anything else we need to know?: When testing with regular pods, rescheduling works fine.

Environment:

• KubeVirt version (use virtctl version): 0.18.1
• Kubernetes version (use kubectl version): 1.14.2
• VM or VMI specifications:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release): Ubuntu 18.04.2 LTS
• Kernel (e.g. uname -a): 4.15.0-51-generic #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
• Install tools:
• Others:

[cynepco3hahue]

In general, we can not know if it was the poweroff issue, or maybe kubelet crashed, so it possible situation when the node has NotReady state, but the VM that runs on this node still continue to write on the storage. In this case similar to k8s we do not restart the VM on the other node until an admin will remove it to avoid data corruption.
We have some solutions to make this process automate, but it depends on the cluster-api, see https://github.com/openshift/machine-api-operator for more details.

[kubevirt-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kubevirt-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[kubevirt-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[kubevirt-bot]

@kubevirt-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.