Pipeline: Dynamic Input Parameters

[warrenbailey]

Expected Behavior

I'm building a release pipeline for one of our projects and ideally I want to pass a dynamically created value between two tasks. I want the first task to generate the 'release version' and do the git tags etc. Then the second tasks (which is Kaniko) to build the image and push it to the docker registry with the released version from the first task as a tag.

I know it's possible to have input parameters but the value of these need to be defined upfront in the pipeline run files but in this scenario they are not known at that point. I also had a look at pipeline resources but the only thing a task seems to be able to output is an image or file. It would be great if it could output a variable.

Actual Behavior

None - new feature suggestion

Steps to Reproduce the Problem

1. Create a Task that writes generates info required for a future task, e.g. this Task will write a version value to a file as json (this is a detail of the design and open to change)

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: create-version
spec:
  inputs:
    resources:
      params:
      - name: currentVersion
        description: The current version as just a single number for this trivial example
  outputs:
    resources:
    - name: version
      type: keyValue # also a detail of the design and open to change
  steps:
  - name: write-new-num
    image: ubuntu
    command:
    - /bin/bash
    args:
    - -c
    - "echo \"{ \\"version\\": $((5 + 3)) }\" > /workspace/version/keyvalue" 

1. Create another Task that uses that value

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: echo-version
spec:
  inputs:
    resources:
    - name: version
      type: keyValue
  outputs:
  steps:
  - name: echo-version
    image: ubuntu
    command:
    - echo
    args:
    - "${inputs.resources.version.version}" # also a design detail open to change

1. Link them together in a Pipeline:

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: demo-pipeline
spec:
  resources:
  - name: version-key-value
    type: keyvalue
  tasks:
  - name: make-version
    taskRef:
      name: create-version
    params:
    - name: currentVersion
      value: 5
    resources:
      outputs:
      - name: version
        resource: version-key-value
  - name: output-the-version
    taskRef:
      name: echo-version
    resources:
      inputs:
      - name: version
        resource: version-key-value
        from: [make-version]

1. Create a PipelineRun to run the Pipeline
2. The goal would be that the first Task could generate a Version which could be echoed by the next one

Additional Info

Related to functionality needed for #216

[abayer]

/assign

[bobcatfish]

@warrenbailey I added a "steps to reproduce" section which tries to sketch out the functionality you're looking for, is that accurate?

The example assumes a design we don't have to be tied to:

1. There's a keyvalue Resource type
2. Tasks can output keyvalue Resoruces by writing a json map to a certain location

[bobcatfish]

btw @abayer we can probably share some logic between this and https://github.com/knative/build-pipeline/issues/216 (the logic for how we get values from an executing container back to the controller)

[abayer]

@bobcatfish Yes! #216 is exactly the same sorta thing.

[warrenbailey]

@bobcatfish yes that's exactly what I want 👍

[fgiloux]

This would also be required to allow human interactions as part of a pipeline, something like the input step in Jenkins [1]. It would then be possible to create a task, which supports the human interaction (presenting a web form for instance) and populate the parameters according to the user input.

[1] https://jenkins.io/doc/pipeline/steps/pipeline-input-step/

[bobcatfish]

Wow I forgot about this issue! We ended up having the request again: #1273 and Tasks can now declare results. The next step is to link these in a Pipeline (https://github.com/tektoncd/pipeline/issues/1950) using a very similar syntax to @warrenbailey 's proposal above!

Closing this issue in favor of #1950