Pipeline: Provide mechanism to set environment variables on TaskRuns

[poy]

Expected Behavior

Have similar API to knative-builds TemplateInstantiationSpec to provide environment variables to a TaskRun.

Actual Behavior

Use work arounds.

Steps to Reproduce the Problem

1.
2.
3.

Additional Info

[vdemeester]

@poy I am not entirely sure I follow but I guess, what you want is to be able to pass environment variable from the TaskRun (to the running pods) ; which is not possible as of today.

It could be something like the following…

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: read-repo
spec:
  taskRef:
    name: read-task
  env:
  - name: "FOO"
    value: "bar"

… or we could enhance podTemplate to have an env field (but I am not a huge fan of that, better abstracting it from podTemplate)

/kind feature

[poy]

Sorry for being vague, but you nailed it! I also prefer your first API suggestion.

[bobcatfish]

hey there @poy! could you describe a bit more about the use case for this?

today you could accomplish this by:

1. Declaring a parameter for your env var
2. Using variable substitution to use that param to populate an environment variable

We've tried to be very deliberate about what values provide at runtime: for example, if you have a step in Task X which requires environment variable FOO, we want to make sure that is as explicit in Task X as possible - if Task X doesn't mention it, but it's expected to be provided via the TaskRun, then users of the Task won't know how to use the Task

[poy]

@bobcatfish of course!

So we are using Buildpacks with Tekton. Traditional Buildpacks (In Cloud Foundry) allowed environment variables for settings.

We have no way of knowing what environment variables could be needed ahead of time as users can create their own Buildpacks. Therefore we need something more dynamic.

[poy]

@vdemeester @bobcatfish I took a swing at a PR for this so it was easier to visualize the API change: https://github.com/tektoncd/pipeline/pull/1657

[bobcatfish]

Thanks for the extra info @poy!

So we are using Buildpacks with Tekton. Traditional Buildpacks (In Cloud Foundry) allowed
environment variables for settings.

I'm not super familiar with Buildpacks, could you give an example of what an example Task + TaskRun would look like for this?

We have no way of knowing what environment variables could be needed ahead of time as users can create their own Buildpacks.

I'm a bit confused - how does the user provide the environment variables to the TaskRun if it's not known ahead of time what env variables are needed? (I think maybe an example might clear this up for me!)

Another couple ideas of how you could do this:

1. I think maybe you could have this same functionality if your Task had an array parameter which you could use to populate the environment variables (tho you'd have to do it for every step)
2. You could use the stepTemplate to set the environment varaiables

[bobcatfish]

You could use the stepTemplate to set the environment varaiables

I think this has the same magic problem that is making me hesitate to agree with adding env vars on TaskRuns - but at least it's a feature we already have :S

btw I'm not sure if you've already been but if you're able to attend our working group could be a great place to discuss a feature request / design like this in real time

[bobcatfish]

(p.s. @vdemeester I noticed you put this into the beta milestone so we might be on slightly different pages about this?)

[vdemeester]

@bobcatfish ah :sweat_smile: I did put it in the beta milestone just so that we discuss it while discussing beta, doesn't really mean I think it should go in for beta or not :angel:

[skaegi]

We use a well-known "named" ConfigMap to pass environment variable values into Steps. This works well when you know the environment variable name.

If you do not know the environment variable name in advance you can use envFrom or mount the ConfigMap as a volume and read and process the names and values as the first command in your Step. A similar approach is to mount an _env volume possibly generated in a previous Task and then in your Steps eval that before running your command.

[bobcatfish]

You could use the stepTemplate to set the environment varaiables

Just realized this is in the Task not the TaskRun so no help after all :woman_facepalming: but @skaegi 's suggestions sound legit! and/or array param maybe?

[vdemeester]

also, with embedded TaskSpec and PipelineSpec in the Run object & co, it's "kinda" covered, as you could "copy" a Task and embedded it with some changes :angel:

[poy]

@skaegi and @bobcatfish We actually use a secret today, however we were hoping to get away from managing the lifecycle of another object, however this is a valid option.

@vdemeester It seems weird to me that there are different API options if I use a Task vs embed it... But it also doesn't seem very clean to do so.

[vdemeester]

@vdemeester It seems weird to me that there are different API options if I use a Task vs embed it... But it also doesn't seem very clean to do so.

There is not a different API options depending of if you use a Task or embedded it. But as this is "hidden" from the user (aka your tool handle that), you could do something like the following (in go code) :

• get the task you want (in memory) — or even have your own defined in go code (using the Task struct), depends on how much you show the user / allow the user to modify/view.
• add environments in there (in memory).
• create a taskrun with embedded task, and use the task definition you just modify in the taskSpec.

You would use a given task as a "template" for your taskrun ; but all your taskrun would be "self-descriptive" and would not depend on any "task" per-se.

Not sure how clean or not it is :angel: it's just a possibility and a valid use-case I think :wink:

[sbwsg]

Just to try and move the conversation forward a bit more on this, there's another use case for the feature that I think is worth considering. Node.js libraries and apps commonly support a debug env var (e.g. DEBUG=express:router* means "print verbose logs from express:router modules") which a user may want to toggle on for specific runs of their tasks while they triage issues in their pipelines. I believe this is also true in Java land where env vars can be used to optionally tell an app to expose debugging ports while running.

I don't think it's great that in order to support those env vars every Java or node.js Task in the universe would need to individually expose their own debug param. This usage may not quite push the needle on deciding whether to add the feature or not but I wanted to point out that there are definitely use cases for custom runtime env vars beyond buildpack conf.

I'd also argue that, while it may not be _awesome_ to allow runtime configuration outside of a task's declared params / artifacts, so long as the TaskRun stores the env vars that were passed in there will at least be a clear record of their usage and they aren't totally silent.

And, finally, just to throw some more options out there, this could conceivably be configurable on the Task:

kind: Task
allowRuntimeEnvVars: true # but defaults to false!

Having that would make runtime env var support explicitly opt-in and catalog tasks could totally leave it out so we don't propagate an anti-pattern.

[skaegi]

I think at some point we have to recognize a Task author cannot make everything parameterizable. Not strictly for CI/CD and Tekton but when we hit these sorts of "debug" and "instrumentation" cases Kustomize is a pretty good solution.

[bobcatfish]

I just want to double check: are we sure that the array param won't work for @poy's use case?

[poy]

I think we could duct-tape this solution in, however it comes with two major drawbacks (as far as I can tell)... We would have to require the entrypoint to be a shell. This would allow us to prepend the command with the given array. BUT:

1. While I'm sure most containers will have a shell available in the path, I don't know that its guaranteed (containers generated from ko for example).

2. It also implies we have to "crack open" the container a bit to invoke the default entrypoint/command so we can prepend the environment variables.

Overall, while I agree this would likely cover many use cases, it doesn't feel clean and certainly doesn't cover all of them.

[cccfeng]

@vdemeester It seems weird to me that there are different API options if I use a Task vs embed it... But it also doesn't seem very clean to do so.

There is not a different API options depending of if you use a Task or embedded it. But as this is "hidden" from the user (aka your tool handle that), you could do something like the following (in go code) :

• get the task you want (in memory) — or even have your own defined in go code (using the Task struct), depends on how much you show the user / allow the user to modify/view.
• add environments in there (in memory).
• create a taskrun with embedded task, and use the task definition you just modify in the taskSpec.

You would use a given task as a "template" for your taskrun ; but all your taskrun would be "self-descriptive" and would not depend on any "task" per-se.

Not sure how clean or not it is 👼 it's just a possibility and a valid use-case I think 😉

This is really a way of implementation, but if I want to pull up taskRun or pipelineRun through tektoncd/trigger project, it seems to be a little difficult. IIRC resourceTemplates can only pull an instance like pipelineResource or pipelineRun, but can't dynamically modify the definition of task step's env. I agree with @sbwsg that the template writer cannot enumerate the parameters/envs that the template instance will pass in

[bobcatfish]

Thanks for the follow up @poy!

We would have to require the entrypoint to be a shell.

What about if the params could be a map or list of lists? So you could do something like:

apiVersion: tekton.dev/v1beta1
kind: TaskRun
spec:
    params:
    - name: settings
       value:
       - name: MY_ENV_VAR
          value: foo
---
apiVersion: tekton.dev/v1beta1
kind: Task
spec:
    params:
    - name: settings
       type: list-of-lists # probably something more complex with "schema" see #1393 
    steps:
    - image: my-image-with-no-shell
      envValueFrom: params.settings[*]

(Completely handwaving about the syntax above! I think that @skaegi is going to simultaneously hate the above syntax and have way better ideas)

If something like this would work for you @poy I'm happy to open another issue to explore further

[poy]

If those settings are made available to the underlying container via environment variables, that sounds great!

[shinji62]

@bobcatfish @dibyom I have a use case, I would like to propragate and environment variable to all my pod / container, for example a build number of something that I inject at the PipelineRun definition, similar to the pod Template.

Do you think that a reasonable case ?

[vdemeester]

Small updates on this after a slack discussion with @shinji62 :wink:
Overall I think we all agree on making runtime requirement in task as explicit as possible. But as @skaegi said

I think at some point we have to recognize a Task author cannot make everything parameterizable. Not strictly for CI/CD and Tekton but when we hit these sorts of "debug" and "instrumentation" cases Kustomize is a pretty good solution.

There is a few use-case where it would be useful to allow the user to "inject" environment variable in there tasks at runtime. The main one is : relatively generic task that might be able to act based on this. A make task would benifit from this as, it does execute something the user define in his repository (and thus, maybe it needs to env var to be set) — but it can be applied to lot's of build/deploy task from the catalog (from golang-test to ansible, …). Without this, we are making task less reusable and we are making the user experience a bit worse : the user needs to write specific tasks for his use case.

I'd also argue that, while it may not be awesome to allow runtime configuration outside of a task's declared params / artifacts, so long as the TaskRun stores the env vars that were passed in there will at least be a clear record of their usage and they aren't totally silent.

I tend to agree with @sbwsg here. I think this feature is needed one way or another. One thing we need to make sure though, is to "record" whatever env vars where set at runtime (in TaskRun status, somewhere else…)

[piyush-garg]

/assign

[shinji62]

@vdemeester Where are we with that?

[vdemeester]

@piyush-garg is working on this :angel:

[dbazhal]

@poy I am not entirely sure I follow but I guess, what you want is to be able to pass environment variable from the TaskRun (to the running pods) ; which is not possible as of today.

It could be something like the following…

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: read-repo
spec:
  taskRef:
    name: read-task
  env:
  - name: "FOO"
    value: "bar"

… or we could _enhance_ podTemplate to have an env field (but I am not a huge fan of that, better abstracting it from podTemplate)

/kind feature

it would be really useful in this form. Also it would be good to be able to use kubernetes valueFrom mechanism in such variables https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/

[piyush-garg]

I will prefer the way of having an env field in the pod template. Benefits are

1. We can specify a global level default
2. We can override default at taskrun and pipelinerun level by specifying in spec.
3. It requires fewer code changes as most fo the logic for the pod template is there to merge/override