Tilt: Should Tilt allow deploys to non-local kubecontexts by default?

[jazzdan]

@thrawny in Kubernetes Slack had production downtime caused by inadvertently allowing Tilt to deploy dev configs to a production Kubernetes cluster. This is something we'd very much like to avoid happening to others.

A use case we want to support is for people to run Tiltfiles with a large number of services against a Kubernetes cluster that exists in the cloud, so that their laptop doesn’t melt. For this use case the kubecontext could be anything. However, it's my believe that comparatively few people are using Tilt in this way.

I think we should disable deploying to non-local (minikube, docker-for-mac, possibly others) kubecontexts by default. If Tilt is started with such a non-local kubecontext we could display a message like this:

It looks like you're running Tilt against a remote Kubernetes cluster (KUBECONTEXT_NAME_HERE). Tilt supports this, but we want to make sure you're not inadvertently deploying to prod! To enable remote Kubernetes deploys simply add `enable_remote_deploy()` to your Tiltfile.

Perhaps if you are using a remote kubecontext and you haven't set enable_remote_deploy() we still perform other local operations (docker builds, locals)?

I would love other's thoughts! @thrawny let me know if I got any details wrong.

[dbentley]

Idea:

allow_kube_context(['docker-for-desktop', 'minikube'])
block_kube_context('gke-useast-prod')

Not sure if we should default to blocking, or default to anything goes, but it's easy to add to your Tiltfile.

[jazzdan]

@dbentley Yeah I like that syntax. The default behavior is an interesting question.

[dexhorthy]

+1 on this, I cannot count the number of times I've switched kube contexts and accidentally deployed a dev stack to production with various k8s dev tools.

UX-wise I think I'd find the most useful function to be allow_kube_context rather than block_kube_context, defaulting to not blocking anything if allow_kube_context wasn't called.

[drubin]

I think the default should allow all + allow_kube_context it makes sense for it to be opt-in but also opt-in blocking is an end-less trying to find which contexts to block.

Whitelists also follow saner defaults in general.

[adamdecaf]

Why not set_kube_context("docker-for-desktop")? That's the only context I want tilt to apply into. If I wanted to test out a remote context it's easy enough to change the file.

[drubin]

This would be nice to I also set namespace too

[landism]

Given #1314, would it be more direct to solve this problem without environment names, via something like a Tiltfile directive allow_remote_k8s()?

1. Maybe more directly matches user intent (IIUC, "I don't want to accidentally touch remote docker / k8s")
2. Avoids the risk of unintentionally pushing images when Tilt fails to recognize that a cluster is local (as in #1314)

[jazzdan]

@landism yeah that makes sense to me

[jazzdan]

I took a pass at implementing this purely in Starlark. I'm curious if other people think this would help them out

https://gist.github.com/jazzdan/d30ef73482920beb3ac9fa2fcb2c3146

It makes use of the newly added JSON support.

[nicks]

i had an idea over the weekend: what if we checked to see if the existing resources in the cluster had a tilt-deploy-id label on them, and if they don't, we require some sort of intervention to patch over them?

[jazzdan]

aha! I like that.

[landism]

reopening because #1995 was reverted because it fails to recognize the latest docker-for-desktop as local