Velero: Add tests for our CLI commands

[carlisia]

When we create a new cli command, the way to test it is manually by running the binary. What if instead we were to test that the commands we expect are all there and are responding to and with what we expect, programmatically?

We can follow this example:
https://github.com/kubernetes/kubernetes/blob/2981fb7a0171798184b4f860b07870de0ff7b234/cmd/kubeadm/test/cmd/util.go

https://github.com/kubernetes/kubernetes/blob/2981fb7a0171798184b4f860b07870de0ff7b234/cmd/kubeadm/app/cmd/phases/util_test.go

This should be broken down into 1 issue per command as it is looking like a large endeavor. The first one would be the biggest since all/most of the framework for the test cases would need to be put together.

[nrb]

What kinds of tests are you looking for? End-to-end?

[carlisia]

Not end-to-end, scoped to the cli. Many of the cli cmds have a lot of logic in them.

[nrb]

Ok - can you add a little bit of a description? I'm not 100% what you're looking for here :)

[carlisia]

@nrb thanks for probing me for a description! Let me know if I can expand on what I have so far.

[nzoueidi]

@carlisia thanks a lot for opening this issue. I was wondering why there is no tests for Velero binary.
More like logic testing. I would like to help with implementing these tests and I would like also to update the second link in the issue.
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/cmd/phases/util_test.go

[carlisia]

@nzoueidi thank you, and I have updated the links. Let me know if you'd like to start with this or with #574 and I'll assign the issue to myself while you work on it.

[nzoueidi]

Sure! thanks @carlisia. I would like to start with this issue. :)

[carlisia]

Go for it!

[skhalash]

Is it still relevant? If yes, I would like to do it!

[carlisia]

Hey @skhalash yes it is! 🎉

[skhalash]

Hey @carlisia! Just want to clarify something before I start implementing it. I read the discussion in the thread and it's not really clear, what kind of test you have in mind. Most of the Velero CLI commands access a k8s cluster, so do you mean:
1) Unit testing the CLI logic by stubbing out/faking the client factory. If yes, are there any existing test utils for that?
2) More of e2e tests with a real cluster (e.g. a local kind cluster which is getting spinned up every time a test suite is run)?
In the kubeadm exampe you posted they only seem to run negative tests.

[eleanor-millman]

Closing because E2E tests will take care of this. @carlisia feel free to let us know if you disagree :)