Python: How do I run kubectl apply-f using the API

[moluzhui]

I tried using the Kubernetes API python to execute the kubect apply command to deploy and update resources

the kubectl command is as follows:

kubectl apply -f export

export is a folder containing multiple yaml files, which is generated by kompose convert -o export

I only found an alternative API for kubectl create, but I didn't find apply

How do I implement kubectl apply to create resources and possible subsequent updates

If you know, please let me know. thank you in advance!

[Sunnykaby]

maybe you can use replace_XXX api

[MoShitrit]

You can look at that example as well https://github.com/kubernetes-client/python/blob/master/kubernetes/utils/create_from_yaml.py

[vishnu667]

@MoShitrit create_from_yaml works when we have to create a resource.
Let's say I want to update an annotation or the image create_from_yaml fails with a FailToCreateError

[vishnu667]

Trying to write a utility replace_from_yaml
But am hitting the error.

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "the name of the object (scheduler-deployment) does not match the name on the URL (airflow)",
  "reason": "BadRequest",
  "code": 400
}

The same payload works with kubectl replace -f not sure what am missing.

[MoShitrit]

Trying to write a utility replace_from_yaml
But am hitting the error.

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "the name of the object (scheduler-deployment) does not match the name on the URL (airflow)",
  "reason": "BadRequest",
  "code": 400
}

The same payload works with kubectl replace -f not sure what am missing.

What's the value of kind from this line, in the error you're getting?
https://github.com/vishnu667/python/blob/replace_from_yaml/kubernetes/utils/replace_from_yaml.py#L156

[vishnu667]

This is the payload Am passing

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment_time: '2020-05-22 12:39:38.324272'
  labels:
    app: scheduler
  name: scheduler-deployment
  namespace: airflow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: scheduler
  template:
    metadata:
      annotations:
        deployment_time: '2020-05-22 12:39:38.324286'
      labels:
        app: scheduler
    spec:
      containers:
      - args:
        - scheduler
        env:
        - name: AIRFLOW__KUBERNETES__NAMESPACE
          value: airflow
        image: airflow_base:local
        imagePullPolicy: IfNotPresent
        name: scheduler
      serviceAccountName: airflow-scheduler-serviceaccount

If you can point me in the right direction I might be able to write down both these functionalities replace and apply.

[MoShitrit]

The payload seems fine. But looking back at your code, I think I see what's missing.
You'll need to add a namespace argument to the call you make here https://github.com/vishnu667/python/blob/replace_from_yaml/kubernetes/utils/replace_from_yaml.py#L157
and in order to make sure you always have a value (which would align with how kubectl does it), you'll need to add an else block here like this:

else:
    namespace = "default"

namespace is a required parameter, and by the error message it seems like the function is trying to use airflow as the name of the deployment, rather than the namespace.
So I would try to be more explicit and see if it helps.

[roycaihw]

kubectl implements a three-way diff logic to perform a client-side apply. We should add example/documentation about how to use the new server-side apply feature instead of re-implementing client-side apply in this client.

[vishnu667]

@roycaihw Can you point me to the kubectl implementation Let me see if I can code the same logic in python.

[vishnu667]

@MoShitrit agree but for the yaml am providing am passing the namespace value.
Will add the else statement also.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[tvkit]

I tried using the Kubernetes API python to execute the kubect apply command to deploy and update resources

kubectl apply -f export

Why would you want to do that? :grin:

---

I'm using ytt to generate the config. It doesn't seem to have (or couldn't find) python integration. So I'm using a shell wrapper to run the command. Here's a decent example for an arbitrary shell operation. Essentially:

def run(cmd):
    proc = subprocess.Popen(cmd, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
    stdout, stderr = proc.communicate()
    return proc.returncode, stdout, stderr

I'm caching the ytt output into temp file, and subsequently running the kubectl shell command you referenced. Not ideal, but it gets the job done providing kubectl is available within the shell.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[solarmosaic-kflorence]

I'm not sure how people are using the python client without this feature... I'm trying to move our deployment process from command line tools (like kubectl) into a python lambda function and I'm surprised to find there is no easy way to replace a kubectl apply -f with a client call.

[tvkit]

@solarmosaic-kflorence ytt via python is a legit approach. It may not suit your peculiar needs, but it works well in our production environment, and is a widely adopted method for transforming yaml files.

[solarmosaic-kflorence]

@tvkit i'm sure it is, but it doesn't have a lot to do with the apply functionality that kubectl provides. And I'd rather not use python just to call a CLI command... kind of defeats the purpose of using python.

[solarmosaic-kflorence]

This ticket may be a duplicate of https://github.com/kubernetes-client/python/issues/1093

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.