Spark-on-k8s-operator: Upgrade CustomResourceDefinitions to use v1 apiVersion

[jutley]

The apiVersion apiextensions.k8s.io/v1beta1 has been deprecated since Kubernetes v1.16 in favor of apiextensions.k8s.io/v1. One of the new features in apiextensions.k8s.io/v1 is the ability to set defaults for fields.

As a result of not upgrading, every time this project has regenerated CRDs, they need to be manually updated in order to include protocol as one of the required fields in the ports container field, as done in #986.

This change would make the resulting CRDs incompatible with Kubernetes versions older than v1.16.

[liyinan926]

Thanks for creating this issue @jutley. Wondering would you be willing to send a PR for the fix?

[fisache]

I wonder whether the result of problem solving is k8s.io package including client-go version up newer than v1.16

[jutley]

@fisache Unfortunately upgrading the dependencies does not seem to help. Throughout the various minor versions, the definition of a ContainerPort remains constant--the protocol field is required, with a default, and this field is used to identify unique entries.

Since the definition remains consistent, the behavior from controller-gen is also consistent. Of course, updating the dependencies would be a good idea nonetheless.

@liyinan926 Currently I don't have enough time to get this PR set up, but may be able to revisit it in a couple weeks.