Task: Variable propagation to subtasks

[atar-axis]

First of all: thank you for your efforts and sharing the software! I really like the simplicity of your tool.

Is there a reason why environment-variables aren't propageted into subtasks?

maintask:
  cmds:
    - echo "(maintask) A is $A and B is $B"
    - task: subtask
  env:
    A: a
    B: b

subtask:
  cmds:
    - echo "(subtask) A is $A and B is $B"

results in

echo "(maintask) A is $A and B is $B"
(maintask) A is a and B is b
echo "(subtask) A is $A and B is $B"
(subtask) A is  and B is 

I know i can propagate them by hand, but that's not very useful in my case since i need a specific variable in nearly every subtask (the variable is calculated dynamically at the beginning).
therefore my taskfile looks something like that at the moment:

one:
  cmds:
    - subtask a
      vars: {shared_dynamically_var: "one_content"}
    - subtask b
      vars: {shared_dynamically_var: "one_content"}
    - subtask c
      vars: {shared_dynamically_var: "one_content"}

two:
  cmds:
    - subtask a
      vars: {shared_dynamically_var: "two_content"}
    - subtask b
      vars: {shared_dynamically_var: "two_content"}
    - subtask c
      vars: {shared_dynamically_var: "two_content"}

[smyrman]

I suppose there are use-cases where not passing on the environment is useful. It might be an idea to look at changing this for v2 format, so that backwards compatibility for v1 is kept.

As a work-around, you can try:
```yaml:
maintask:
cmds:
- echo "(maintask) A is $A and B is $B"
- task subtask # issue subtask in a sub-process, A and B should now be present in _global_ environment
env:
A: a
B: b

subtask:
cmds:
- echo "(subtask) A is $A and B is $B"

```

[atar-axis]

thank you for your answer.

to be honest, I can't think of any use-case where this would be useful ;)
when invoking a task from within another task I would always expect that the environment-variables which are set for the parent task are propagated downwards, however...

your workaround - well, works :D - but, isn't there a way to set a real environment-variable inside the Taskfile? One which is accessible from all tasks? Something like the following in a Makefile

export A=a

target_one:
    echo $$A

[atar-axis]

okay, based on your workaround the following will do it for now, but i am not really happy :D i moved from make because i wanted to avoid all those workarounds^^

dummy_task_to_set_env:
  cmds:
    - task real_task
  env:
    A: a
    B: b

[andreynering]

@smyrman Recursive calls to task is possible and useful as a workaround, but it's not an ideal solution IMHO.

@atar-axis I don't know it propagating ENV to subtasks would be OK, it can introduce unintended side effects. (Example: GOOS, if unintentionaly set, will change the target OS for a Go program).

---

For v2, we can think about two different syntaxes for setting an ENV:

• In a task call:

foo:
  cmds:
    - task: bar
      env: {GOOS: linux}

• As a global for all tasks:

version: 2
env:
  FOO: bar
tasks:
  # ...

[atar-axis]

@andreynering Some thoughts on your suggestions:

foo:
  cmds:
    - task: bar
      env: {GOOS: linux}

this way GOOS: linux will be available from every subtask that is called within bar, right? That would be nice! I now use the workaround that was suggested above

dummy_task_to_set_env:
  cmds:
    - task real_task
  env:
    A: a
    B: b

The problem is that you cannot be sure that "task" is always available via "PATH", some may distribute the task-binary with their sourcecode, others tell the user to install it. Therefore on some machines it should be:

  cmds:
    - task real_task

and on others it should be

  cmds:
    - ./task real_task

Your suggestion would solve that problem.

---

As a global for all tasks:
version: 2
env:
  FOO: bar
tasks:
  # ...

Is also a good idea, but this way env wouldn't be changeable, right? So why don't allow both?

By the way: Both variations result in errors at the current version, therefore it shouldn't be necessary to limit them to "version: 2" - right? At least not in sense of downwards compatibility (supporting older Taskfiles on newer task-versions).

---

Another thought: Why don't leave it to the user to decide if the environment-variables are propagated downwards?

[andreynering]

@atar-axis Yes, maybe there should be an option for it. (propagate_env: true?).

By the way: both variations result in errors at the current version, therefore it shouldn't be necessary to limit them to "version: 2" - right? At least not in sense of downwards compatibility (supporting older Taskfiles on newer task-versions).

You're right there would be no backwards incompatibility for these keywords). But we're now pushing all new features to v2.

[atar-axis]

@andreynering: can you think of any reason why it would be dangerous to propagate local variables to subtasks? it would make life much easier.

whenever task A calls task B, every local variable from task A is inherited to task B.

the importance would be then:

• Variables given while calling a task from another.
• Environment variables
• Variables declared locally in the task
• Variables inherited from parents
• Variables available in the Taskvars.yml file

[smyrman]

@smyrman Recursive calls to task is possible and useful as a workaround, but it's not an ideal solution IMHO.

Completely agreed.

The problem is that you cannot be sure that "task" is always available via "PATH", some may distribute the task-binary with their sourcecode

FWIW, this is probably possible to configure/standardize per project; you could always extend _PATH_ in Taskvars.tml to include e.g. your $GIT_ROOT/bin/ or so. Having the task binary available in PATH is currently pretty much essential once you grow out of keeping everything in one Taskfile. E.g., we have a few things like this at our company:

## -- external tasks --

root/bin/mytool:
  dir: ../../
  cmds:
    - task bin/mytool
  status:
    - task --status bin/mytool

@andreynering: can you think of any reason why it would be dangerous to propagate local variables to subtasks? it would make life much easier.

@atar-axis, with the priority you list, it could be a problem if a variable is defined in Taskvars.yml, that you have overridden for a task without intending for the value to be overridden in sub-tasks.

I would vote for making it explicit in v1. This retains complete backwards-compatibility, and makes it a conscious decision that we are breaking the called task out of it's "sanboxed shell/variable environment" by passing along local variables and/or environment variables.

mytask:
  env:
    E1: test
    E2: test
  vars:
    V1: test
  cmds:
    - task: subtask
      inherit_local_vars: true
      inherit_local_env: true
      env:
        E2: override

This probably also means that we should allow the env as shown in the example. As pointed out by @atar-axis, we can add these options without breaking backwards compatibility. Global env values shown by @andreynering, would be introduced in v2.

the importance would be then:

1. Variables given while calling a task from another.
2. Environment variables
3. Variables declared locally in the task
4. Variables inherited from parents
5. Variables available in the Taskvars.yml file

I think it would probably be easier to reason about, and easier to implement, if the inherited variables where prioritized at position 2, right under call variables:

1. Variables given while calling a task from another.
2. Variables inherited from parents
3. Environment variables
4. Variables declared locally in the task
5. Variables available in the Taskvars.yml file

Implementation wise, this means we could at least in theory _compile_ the vars and env section of each cmd while preparing the task for execution, so that we simply "merge" local vars/env + what ever is set explicitly into the respective "env" and "vars" section of each cmd call.

Relevant code:

• 
  https://github.com/go-task/task/blob/e23a6dc9f16e759bd0a7082c83f937077814e50a/variables.go#L87

Priority order in v2

PS! There are ides to change the priority order for v2, so that deeper levels always override higher levels like they would normally do in a shell.

E.g. environment variables -> override by CLI variables -> overidden by Taskfile variables (now Taskvars.yml) -> Overriden by inherited variables I suppose -> Overidden by Call variables -> Overridden by Local variables.

Setting local _default_ values would still be possible and documented, so that you could still "override" values "the other way" like we do today. It could be that this will be restricted to setting a default value via the templating syntax, which would require at least #83 to be fixed. This ordering also allows the implementation to be further simplified by .e.g. allowing each variable to be evaluated only once, and pass it along to sub-tasks via some sort of context. See #57 or #76.

[andreynering]

Closing some old issues. If you think this one still makes sense, let me know

Propagation is still not supported, but you can concat a variable inside an env like this:

version: '2'

vars:
  FOO: foobar

tasks:
  do-something:
    cmds:
      - echo $BAR
    env:
      BAR: "{{.FOO}}"

If one think a global env map (like we have a global vars map) would be useful, feel free to open another issue

[sisp]

I'd like to pick up this old issue. I think setting an env map for a subtask (without automatic propagation further down) makes a lot of sense. Think of this use case (sketch):

version: '3'

tasks:
  app1:build:
    cmds:
      - task: .cra:build
        vars:
          WORKDIR: webapp1
        env:
          PUBLIC_URL: /webapp1

  app2:build:
    cmds:
      - task: .cra:build
        vars:
          WORKDIR: webapp2
        env:
          PUBLIC_URL: /webapp2

  .cra:build:
      dir: "{{.WORKDIR}}"
      cmds:
        - npm run build
      sources:
        - src/**/*
        - public/**/*
        - package.json
        - package-lock.json
      generates:
        - build/**/*

Here, I'd like to build two "Create React App"-based React apps with some build-time configuration values. I have factored out the general build task in .cra:build which I can reuse to build both apps. To make this work, calling the subtasks app1:build and app2:build requires setting an env map. It would be sufficient to only set the env map for the subtask call without propagating the environment variables further down. If this is needed, it could be done manually by also setting the vars map for the subtask and following the same pattern to set the env map in there.

What do you think @andreynering?

[andreynering]

Hi @sisp,

That probably makes sense. There's a workaround for that, though:

version: '3'

tasks:
  task:
    - task: subtask
      vars:
        FOO_ENV: development

  subtask:
    cmds:
      - npm run build
    env:
      FOO_ENV: "{{.FOO_ENV}}"

[sisp]

This only works when the set of environment variables to be exported by the subtask is fixed. With a generic .cra:build task, the set of needed environment variables depends on the specific app to be built.