Task: Refactor vars and templating to fix some bugs for v3

[stephenprater]

If you are using dynamic variables, the shell invocations that populate those variables must not fail, regardless of the value of the templated variable.

Example:

tasks:
   release:
     vars:
       KIND: 'holepunch'
       RELEASED_TAG:
         sh:  git describe --abbrev=0 --tags --match "{{.KIND}}*"
     cmds:
        - echo "Doesn't matter won't run"

This task won't run because the sh command that populates RELEASED_TAG will fail when it is run with a KIND value of <no value> - ie git describe --abbrev=0 --tags --match "<no value>*"0

This feels like an undesirable behavior to me, since the only way to avoid this is to pass the variable in on the CLI, even though it has a static value.

Suggested solution:

Ignore errors on dynamic var assignments until the last expansion pass.

[smyrman]

@stephenprater, a possible workaround for you now could be to rely on the default function.

version: "2.6"
tasks:
   release:
     vars:
       RELEASED_TAG:
         sh:  git describe --abbrev=0 --tags --match '{{.KIND | default "holepunch"}}*'
     cmds:
      - echo '{{.RELEASE_TAG}}'

Or if you also need to use KIND in cmds, you would need to do this:

version: "2.6"
tasks:
   release:
     vars:
       KIND: '{{.KIND | default "holepunch"}}'
       # Workaround for go-task/task#218; dynamic variables are evaluated before static ones.
       RELEASED_TAG:
         sh:  git describe --abbrev=0 --tags --match '{{.KIND | default "holepunch"}}*'
     cmds:
      - echo '{{.RELEASE_TAG}}'
      - echo '{{.KIND}}'

Fix vars for v3?

@andreynering, maybe one could also consider getting rid of the whole multiple passes all-together, as the multiple passes is something that could easily introduce side effects, even today. It might also be easier to reason about if variables are simply passed in order.

Perhaps for v3 #195, it's a good idea to change vars and env to be a list so they can be processed in order. This would also make it easier to extend vars with more fields.

vars:
- key: KIND
  desc: The kind of release to do
  default: holepunch
- key: RELEASED_TAG
  desc: The final release tag to use (evaluated from KIND by default)
  defaultEval: git describe --abbrev=0 --tags --match "{{.KIND}}*"

It's worth thinking about the design a bit more, this is just an example. One could look on tools such as Tusk, Terraform or even Kubernetes manifests for inspiration.

[andreynering]

Yeah, I think since v3 is coming, we're open to discuss changes on how variables is implemented, since we definitely have bugs and undesired behavior on some use cases.

[andreynering]

I think that maybe we could have ordered maps without having to convert it to slices using new changes introduced by go-yaml/yaml v3:

https://blog.ubuntu.com/2019/04/05/api-v3-of-the-yaml-package-for-go-is-available

We have an issue blocking us from using v3: https://github.com/go-yaml/yaml/issues/450. But other than that, using ordered maps is probably better than slices, since it means less backward incompatibility between Task v2 -> v3.

[galindro]

One thing good that I'm trying to achieve is load envrionment variables exported by a sh file from a dependent task to the main task. For example:

vars.env

export MYVAR1=$(my command)

Taskfile.yml

version: '2'
tasks:
  load-vars:
    cmds:
      - . vars.env
  apply:
    deps:
      - load-vars
    cmds:
      - echo $MYVAR1

Do you guys think that will it be possible in v3? @andreynering

[andreynering]

Hi @galindro,

I don't think it'll work as you suggested.

But keep in mind global env: is already a thing, and there's this proposal to run something before every command.

[galindro]

I like the idea from https://github.com/go-task/task/issues/204. Is it being considered to be released on v3?

[andreynering]

@galindro Yeah, it's possible

[andreynering]

After #311 some of the confusion with variables should have been fixed. Task will now respect the order of declaration of variables.

If possible, give the v3 branch a try and give some feedback before the final release.

[marco-m]

Hello @andreynering I am going through the documentation https://taskfile.dev/ now that 3.0 is released.

Am I wrong or this part: https://taskfile.dev/#/usage?id=variables-expansion is not valid anymore after the merge of #311 ?

[andreynering]

@marco-m Thanks for letting me know, I just updated it.