Serving: Installing bundled Istio frequently fails with "no matches for kind ..."

Created on 9 Oct 2018 · 6Comments · Source: knative/serving

Expected Behavior

kubectl apply -f third_party/istio-1.0.2/istio.yaml should succeed without error.

Actual Behavior

kubectl apply -f third_party/istio-1.0.2/istio.yaml often (but not always) spits out lots of errors.

Steps to Reproduce the Problem

kubectl apply -f third_party/istio-1.0.2/istio.yaml

Additional Info

This doesn't happen all the time, but there appears to be some race condition in applying the istio.yaml that causes it to frequently fail the first time on my local minikube or minishift environment. Applying it a second time always succeeds.

$ kubectl apply -f third_party/istio-1.0.2/istio.yaml namespace/istio-system created configmap/istio-galley-configuration created configmap/istio-statsd-prom-bridge created configmap/istio-security-custom-resources created configmap/istio created configmap/istio-sidecar-injector created serviceaccount/istio-galley-service-account created serviceaccount/istio-egressgateway-service-account created serviceaccount/istio-ingressgateway-service-account created serviceaccount/istio-mixer-service-account created serviceaccount/istio-pilot-service-account created serviceaccount/istio-cleanup-secrets-service-account created clusterrole.rbac.authorization.k8s.io/istio-cleanup-secrets-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-cleanup-secrets-istio-system created job.batch/istio-cleanup-secrets created serviceaccount/istio-citadel-service-account created serviceaccount/istio-sidecar-injector-service-account created customresourcedefinition.apiextensions.k8s.io/virtualservices.networking.istio.io created customresourcedefinition.apiextensions.k8s.io/destinationrules.networking.istio.io created customresourcedefinition.apiextensions.k8s.io/serviceentries.networking.istio.io created customresourcedefinition.apiextensions.k8s.io/gateways.networking.istio.io created customresourcedefinition.apiextensions.k8s.io/envoyfilters.networking.istio.io created customresourcedefinition.apiextensions.k8s.io/httpapispecbindings.config.istio.io created customresourcedefinition.apiextensions.k8s.io/httpapispecs.config.istio.io created customresourcedefinition.apiextensions.k8s.io/quotaspecbindings.config.istio.io created customresourcedefinition.apiextensions.k8s.io/quotaspecs.config.istio.io created customresourcedefinition.apiextensions.k8s.io/rules.config.istio.io created customresourcedefinition.apiextensions.k8s.io/attributemanifests.config.istio.io created customresourcedefinition.apiextensions.k8s.io/bypasses.config.istio.io created customresourcedefinition.apiextensions.k8s.io/circonuses.config.istio.io created customresourcedefinition.apiextensions.k8s.io/deniers.config.istio.io created customresourcedefinition.apiextensions.k8s.io/fluentds.config.istio.io created customresourcedefinition.apiextensions.k8s.io/kubernetesenvs.config.istio.io created customresourcedefinition.apiextensions.k8s.io/listcheckers.config.istio.io created customresourcedefinition.apiextensions.k8s.io/memquotas.config.istio.io created customresourcedefinition.apiextensions.k8s.io/noops.config.istio.io created customresourcedefinition.apiextensions.k8s.io/opas.config.istio.io created customresourcedefinition.apiextensions.k8s.io/prometheuses.config.istio.io created customresourcedefinition.apiextensions.k8s.io/rbacs.config.istio.io created customresourcedefinition.apiextensions.k8s.io/redisquotas.config.istio.io created customresourcedefinition.apiextensions.k8s.io/servicecontrols.config.istio.io created customresourcedefinition.apiextensions.k8s.io/signalfxs.config.istio.io created customresourcedefinition.apiextensions.k8s.io/solarwindses.config.istio.io created customresourcedefinition.apiextensions.k8s.io/stackdrivers.config.istio.io created customresourcedefinition.apiextensions.k8s.io/statsds.config.istio.io created customresourcedefinition.apiextensions.k8s.io/stdios.config.istio.io created customresourcedefinition.apiextensions.k8s.io/apikeys.config.istio.io created customresourcedefinition.apiextensions.k8s.io/authorizations.config.istio.io created customresourcedefinition.apiextensions.k8s.io/checknothings.config.istio.io created customresourcedefinition.apiextensions.k8s.io/kuberneteses.config.istio.io created customresourcedefinition.apiextensions.k8s.io/listentries.config.istio.io created customresourcedefinition.apiextensions.k8s.io/logentries.config.istio.io created customresourcedefinition.apiextensions.k8s.io/edges.config.istio.io created customresourcedefinition.apiextensions.k8s.io/metrics.config.istio.io created customresourcedefinition.apiextensions.k8s.io/quotas.config.istio.io created customresourcedefinition.apiextensions.k8s.io/reportnothings.config.istio.io created customresourcedefinition.apiextensions.k8s.io/servicecontrolreports.config.istio.io created customresourcedefinition.apiextensions.k8s.io/tracespans.config.istio.io created customresourcedefinition.apiextensions.k8s.io/rbacconfigs.rbac.istio.io created customresourcedefinition.apiextensions.k8s.io/serviceroles.rbac.istio.io created customresourcedefinition.apiextensions.k8s.io/servicerolebindings.rbac.istio.io created customresourcedefinition.apiextensions.k8s.io/adapters.config.istio.io created customresourcedefinition.apiextensions.k8s.io/instances.config.istio.io created customresourcedefinition.apiextensions.k8s.io/templates.config.istio.io created customresourcedefinition.apiextensions.k8s.io/handlers.config.istio.io created clusterrole.rbac.authorization.k8s.io/istio-galley-istio-system created clusterrole.rbac.authorization.k8s.io/istio-egressgateway-istio-system created clusterrole.rbac.authorization.k8s.io/istio-ingressgateway-istio-system created clusterrole.rbac.authorization.k8s.io/istio-mixer-istio-system created clusterrole.rbac.authorization.k8s.io/istio-pilot-istio-system created clusterrole.rbac.authorization.k8s.io/istio-citadel-istio-system created clusterrole.rbac.authorization.k8s.io/istio-sidecar-injector-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-galley-admin-role-binding-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-egressgateway-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-ingressgateway-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-mixer-admin-role-binding-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-pilot-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-citadel-istio-system created clusterrolebinding.rbac.authorization.k8s.io/istio-sidecar-injector-admin-role-binding-istio-system created service/istio-galley created service/istio-egressgateway created service/istio-ingressgateway created service/istio-policy created service/istio-telemetry created service/istio-statsd-prom-bridge created deployment.extensions/istio-statsd-prom-bridge created service/istio-pilot created service/istio-citadel created service/istio-sidecar-injector created deployment.extensions/istio-galley created deployment.extensions/istio-egressgateway created deployment.extensions/istio-ingressgateway created deployment.extensions/istio-policy created deployment.extensions/istio-telemetry created deployment.extensions/istio-pilot created deployment.extensions/istio-citadel created deployment.extensions/istio-sidecar-injector created horizontalpodautoscaler.autoscaling/istio-egressgateway created horizontalpodautoscaler.autoscaling/istio-ingressgateway created horizontalpodautoscaler.autoscaling/istio-policy created horizontalpodautoscaler.autoscaling/istio-telemetry created horizontalpodautoscaler.autoscaling/istio-pilot created mutatingwebhookconfiguration.admissionregistration.k8s.io/istio-sidecar-injector created unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "Gateway" in version "networking.istio.io/v1alpha3" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "attributemanifest" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "attributemanifest" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "stdio" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "logentry" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "logentry" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "metric" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "prometheus" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "kubernetesenv" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "rule" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "kubernetes" in version "config.istio.io/v1alpha2" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3" unable to recognize "third_party/istio-1.0.2/istio.yaml": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3"

kinbug

Source

bbrowning

👍10

Most helpful comment

@kacole2 In PR #2251 I have a fix for this where you first install the new istio-crds.yaml and then the istio.yaml. That PR isn't merged yet, and thus the docs aren't updated to use that new method yet. But, installing the CRDs first followed by the rest of Istio should resolve this.

bbrowning on 19 Oct 2018

👍4

All 6 comments

To make sure it works well, I think we'll need to:

make sure the k8s cluster is new enough to include the fix of https://github.com/kubernetes/kubernetes/pull/63068 (1.11+ I think)
improve the installation progress to watch a created CRD, and ensure it in Established condition before creating its CR (more like task of the istio folks though)

lichuqiang on 10 Oct 2018

Hmm - perhaps my cluster is not new enough to include the fix you mentioned. Until now I was under the assumption that any Kubernetes 1.11 is new enough, but it looks like we may need a more specific requirement for a good user experience.

With the split release yamls coming in Serving 0.2, this issue may become irrelevant anyway?

bbrowning on 11 Oct 2018

@bbrowning I'm getting the same types of errors with v1.11.2. I made an adjusted installation document to quickly follow, but I need to run the helm install twice, let it fail, remove CRDs, install it a third time and works. https://gist.github.com/kacole2/f761dac381b13a2e1b4dc9904d7559fc#install-istio-with-sidecar-injector

kacole2 on 19 Oct 2018

bbrowning on 19 Oct 2018

👍4

tl;dr I am going to move this out of the 0.2 project.

As this pertains to Istio installation, which isn't a release artifact we want to be in the business of supporting, I am going to remove this from the 0.2 tracking project.

We can and should strive to make our onboarding process as simple as possible, including the setup of our dependencies (e.g. Istio), but what we're shipping is really serving.yaml, not istio.yaml.