Azure-pipelines-agent: Output variables in downstream job don't exist

[damienpontifex]

Agent Version and Platform

Version of your agent? Hosted VS2017 - Agent version 2.140.0

OS of the machine running the agent? Windows

VSTS Type and Version

VSTS

If VisualStudio.com, what is your account name? https://readify.visualstudio.com

What's not working?

In the guide Output Variables under Examples it describes using a tasks output variables in a downstream job such as from Job_2 using $(Job_1.TaskA_1.AuthToken). In another task of the same job, this works (without using the Job_1 prefix) but doesn't in a separate job.

Setup:
1. JobA - Hosted VS2017
- PowerShell Task
- inline script Write-Host "##vso[task.setvariable variable=ChangeFilePath2;isOutput=true]C:\Test"
- Output Variables - Reference Name PS1Output
- PowerShell Task
- inline script Write-Host "This works: $(PS1Output.ChangeFilePath2)"
2. JobB - Hosted VS2017
- PowerShell Task
- inline script Write-Host "This doesn't work $(JobA.PS1Output.ChangeFilePath2)"

Agent and Worker's Diagnostic Logs

JobA.PS1Output.ChangeFilePath2 : The term 'JobA.PS1Output.ChangeFilePath2' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

[bryanmacfarlane]

have you tried with yaml?

https://github.com/Microsoft/azure-pipelines-agent/blob/master/docs/preview/outputvariable.md

[damienpontifex]

No we haven't. I did see in other issues that this was originally just supported in yaml definitions, but the docs page didn't mention that limitation so was using our existing pipeline in the visual editor.

Is this still a restriction for this support (yaml only)?

[chrispat]

Regardless of if we support the in the designer you will still need the explicitly map the outputs you want across jobs. We do not flow them automatically.

[damienpontifex]

Thanks @chrisrpatterson. How do we explicitly map them across jobs?

There isn't a mention in the linked readme of doing anything explicit. Just access them in JobB as $(JobA.PS1Output.ChangeFilePath2). Do we have to register/mark these in some way?

[ericsciple]

@vtbassmatt I can't find information about this in the new docs

@damienpontifex here is an example in our old yaml docs

[vtbassmatt]

It's there, but not very discoverable. All the way at the bottom of this doc.

I have a PR up (internally) which puts a better example in the conditions topic.

[ericsciple]

@vtbassmatt that is a different scenario. The scenario here is consuming the variable in a downstream job (on the agent) - not using the output variable to determine the job condition (orchestrator). Consuming the variable on the agent side requires extra steps to map it into the job message that gets sent down to the agent.

[vtbassmatt]

Ah gotcha. OK, I need to find a place for that.

[vtbassmatt]

Nevermind, it's already there: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=vsts&tabs=yaml%2Cbatch#set-an-output-multi-job-variable

[damienpontifex]

Currently in the visual editor, we can only set pipeline variables...correct me if I'm wrong?

Just using pipeline variables doesn't seem to work:
Setting variable:

• Name: ChangeFilePath2
• Value: $[ dependencies.JobA.outputs['PS1Output.ChangeFilePath2'] ]

Then in the second job from PowerShell running Write-Host This is $(ChangeFilePath2) just echoes $[ dependencies.JobA.outputs['PS1Output.ChangeFilePath2'] ] rather than the variable set in JobA.

Am I correct then that this requires job variables? Which are only currently available in yaml and still coming for the visual editor?

[vtbassmatt]

It works within a job, but not across jobs using the visual editor. If you need to flow variables across jobs, you'll have to use YAML. No timeframe for when it might come to the visual editor.

[damienpontifex]

Thanks for all the assistance on this. In the meantime, we will try it out with yaml.

One question: how can I create a release with a yaml definition? I noticed a few projects on GitHub have a .vsts-release.yml file, but creating this in my repo doesn't create a release definition like a .vsts-ci.yml file does with build. Or do I have to create a release with this file somewhere/somehow from the UI?

[vtbassmatt]

We don't have our Release product running on YAML yet. I see we're using .vsts-release.yml right here in our own repo, but it's really using Build, not Release.

[bryanmacfarlane]

Yep. It's using pipelines (build) yaml to do a release.

[damienpontifex]

So TL;DR summary from these discussions:

• Using output variables in another job is not possible in releases
• Is possible using build and yaml definition with job variables defining the dependency (not build variables)

[vtbassmatt]

I'm not sure what the "(not build variables)" part means. Other than that, you've correctly summarized the situation :)

[damienpontifex]

Meaning you have to define the dependency at the job level.

Not sure if it's the same in yaml when defining variables, but like this from a previous comment when I defined the variable for the entire definition

Just using pipeline variables doesn't seem to work:
Setting variable:

Name: ChangeFilePath2
Value: $[ dependencies.JobA.outputs['PS1Output.ChangeFilePath2'] ]
Then in the second job from PowerShell running Write-Host This is $(ChangeFilePath2) just echoes $[ dependencies.JobA.outputs['PS1Output.ChangeFilePath2'] ] rather than the variable set in JobA.

And thanks for confirming 😄

[mganeshphani]

I am confused... can someone from MSFT declare answer to following question?
Does this support designer pipeline to pass output variables set in one job to a task in another job in same pipeline

[vtbassmatt]

Designer pipelines don't support passing variables from a task in one job to another job. YAML pipelines do support this.

[DeChrist]

Thanks for summary of current situation.
There is still confusion as to what the expected target situation should be: is this by design or will Designer Pipelines (especially for Release) be aligned on the YAML behavior?

[vtbassmatt]

We don't expect to bring this feature to the designer environments. We're working on making YAML fully capable to support release pipelines.

[mganeshphani]

What do you suggest for organizations like mine, who are planning to start
full on Azure devops usage for CICD pipeline, should we start making our
users adapt to YAML definitions or can give a freehand to have them use
designer at will.

Your statement obviously is a red flag to me, saying that "an year or two
down the timeline, we no longer support designer"

On Wed, Jan 2, 2019 at 10:15 AM Matt Cooper [e-mail]
wrote:

We don't expect to bring this feature to the designer environments. We're
working on making YAML fully capable to support release pipelines.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/azure-pipelines-agent/issues/1842#issuecomment-450907009,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AV7GICWg5f2ziZIHaWdKqdMFXzmY4BKGks5u_Ns_gaJpZM4WzVJk
.

--
Ganesh Phani Majeti.

[vtbassmatt]

We don't plan on dropping support for the designer. Definitely not in a year or two - I can't name a feature that this team has removed that quickly in its entire 15+ year history. It's true that new features will land in YAML first. Some, but not all, of them will also come to the designer.

[mganeshphani]

Thanks for your response Matt. Looks like it is safe to stick to YAML to
have full access to broadest of features.

On Thu, Jan 3, 2019 at 6:29 AM Matt Cooper [e-mail] wrote:

We don't plan on dropping support for the designer. Definitely not in a
year or two - I can't name a feature that this team has removed that
quickly in its entire 15+ year history. It's true that new features will
land in YAML first. Some, but not all, of them will also come to the
designer.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/azure-pipelines-agent/issues/1842#issuecomment-451130016,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AV7GIBaCiqA31D1cM_35SXIU0PK7nbTHks5u_fe7gaJpZM4WzVJk
.

--
Ganesh Phani Majeti.

[ajbeaven]

Please bring this to the designer. I would hope that when release pipelines are running on top of YAML that this wouldn't be too difficult to make happen.

[sreekanthsmk]

Any update about this functionality in the classic designer?