Azure-pipelines-agent: YAML - Allow Scripts to Access the OAuth Token

[mccw]

Have you tried trouble shooting?

I can't figure out how to make this work in a YAML build. I've added the following script to my definition:

steps:

• powershell: |
  $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=2.0"
  Write-Host "URL:" $url
  $definition = Invoke-RestMethod -Uri $url -Headers @{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
  Write-Host "Pipeline = $($definition | ConvertTo-Json -Depth 100)"
  env:
  SYSTEM_ACCESSTOKEN: $(system.accesstoken)

but the step that needs the authentication is failing with:

2018-09-13T17:47:15.0019206Z ##[error]OAuth token not found. Make sure to have 'Allow Scripts to Access OAuth Token' enabled in the build definition.

I've attached a copy of the yaml file.

Agent Version and Platform

Version of your agent? 2.140.0

OS of the machine running the agent? Server 2016

VSTS Type and Version

VisualStudio.com

If VisualStudio.com, what is your account name? http://usbc.visualstudio.com

vsts-ci.txt

[TingluoHuang]

@mccw can you share a full build log with me?

[mccw]

@mccw can you share a full build log with me?

I sent logs via email.

[TingluoHuang]

@mccw which email did you send? i didn't get anything. tihuang @ microsoft.com

[mccw]

The logs have been forwarded to the email address provided.

YAML is still very new to me but it looks like env:SYSTEM_ACCESSTOKEN is not being made available to the rest of the build. If I try to write-host from a second powershell script it is blank and I would expect to see the * printed out to host.

Thank you Ting

[mccw]

This passes the variable along but the tag step is still stuck at GIT PUSH ORIGIN which I think means it's still looking for the check mark:

• powershell: |
  $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=2.0"
  Write-Host "URL:" $url
  $definition = Invoke-RestMethod -Uri $url -Headers @{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
  Write-Host "Token: $env:SYSTEM_ACCESSTOKEN"
  Write-Host ("##vso[task.setvariable variable=SYSTEM_ACCESSTOKEN;]$(system.accesstoken)")
  displayName: 'Get OAuth Token'
  enabled: true

[mccw]

I can create the exact same steps in the GUI and they work just fine with "Allow scripts to Access OAuth Token" checked.

[TingluoHuang]

@ericsciple FYI,
The behavior change might related to this:
https://github.com/Microsoft/azure-pipelines-agent/blob/master/docs/preview/yamlgettingstarted-token.md
For YAML build, we always expose SYSTEM_ACCESSTOKEN as a secret variable to task, you don't have to set "Allow scripts to Access OAuth Token" anymore.

however, for the task you are using (GitTag), the author doesn't retrieve variable using task-lib function which cause the inconsistent behavior.

Task code not working today:

if (!($env:SYSTEM_ACCESSTOKEN )) {
    throw ("OAuth token not found. Make sure to have 'Allow Scripts to Access OAuth Token' enabled in the build definition.
            Also, give 'Project Collection Build Service' 'Contribute' and 'Create Tag' permissions - Cog -> Version Control -> {Select Repository/ies}")
}

Task code should work:

if (!(Get-VstsTaskVariable -Name "System.AccessToken")) {
    throw ("OAuth token not found. Make sure to have 'Allow Scripts to Access OAuth Token' enabled in the build definition.
            Also, give 'Project Collection Build Service' 'Contribute' and 'Create Tag' permissions - Cog -> Version Control -> {Select Repository/ies}")
}

[TingluoHuang]

try change your yaml file to following should help you with the git push hang problem:

queue:
  name: Applications

name: $(Build.DefinitionName) - $(year:yy)$(DayOfYear)$(rev:.r)

trigger:
  branches:
    include:
      - master
      - develop

steps:
- checkout: self
  clean: true
  persistCredentials: true

- powershell: |
    $url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/build/definitions/$($env:SYSTEM_DEFINITIONID)?api-version=2.0"
    Write-Host "URL:" $url
    $definition = Invoke-RestMethod -Uri $url -Headers @{ Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN" }
    Write-Host "Pipeline = $($definition | ConvertTo-Json -Depth 100)"
  env:
    SYSTEM_ACCESSTOKEN: $(system.accesstoken)

...
...
...

[mccw]

persistCredenttials works like a charm. I haven't seen that mentioned in any of the documentation.

I've made the change locally and confirmed it does work. I've also passed the recommendation along to the app developer.

Thank you very much for taking a look at this!

[TingluoHuang]

cool, glad you unblock, i will talk with forks after weekends about this.

[geoarchitect]

This solved our git issues as well. In case anyone else has trouble, we are doing an npm version bump on every build and want to commit this back to the branch. The legacy builder works like a charm, but not the yaml.

steps:
- checkout: self
  clean: true
  persistCredentials: true

Adding this to the YAML worked like a charm.

[renaudaste]

Thank you so much @geoarchitect & @TingluoHuang !

For anyone else, documentation can be found here :
https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema#checkout

steps:
- checkout: self  # self represents the repo where the initial Pipelines YAML file was found
  clean: boolean  # if true, execute `execute git clean -ffdx && git reset --hard HEAD` before fetching
  fetchDepth: number  # the depth of commits to ask Git to fetch; defaults to no limit
  lfs: boolean  # whether to download Git-LFS files; defaults to false
  submodules: true | recursive  # set to 'true' for a single level of submodules or 'recursive' to get submodules of submodules; defaults to not checking out submodules
  path: string  # path to check out source code, relative to the agent's build directory (e.g. \_work\1); defaults to a directory called `s`
  persistCredentials: boolean  # if 'true', leave the OAuth token in the Git config after the initial fetch; defaults to false