Azure-pipelines-tasks: Why the variable "SourceBranchName" is overwritten to "merge" in case of PullRequest builds?

[lukas-ais]

Environment

• TFS on-premises: 16.131.27701.1
• Agent - Private: Windows, Agent 2.131

Issue Description

We have a build definition for the master branch with the following "Build number format" configuration:
$(BuildDefinitionName)_$(SourceBranchName)_$(date:yyyyMMdd)$(rev:.r)

When creating a Pull Request to merge into master, this build definition is used to build the sources on feature branch. In this case the variable SourceBranchName is not (anymore) the real branch name, but simply "merge".
Why?

I think this issue is close to #2147 but maybe with slightly different focus. This is why I created a new one.

I would like to see any variable usable in the "Build number format" configuration providing the "feature branch" name in a "Build number format" compatible format. Even more I would like to see the "source branch" in "SourceBranchName"...

The variable SYSTEM_PULLREQUEST_SOURCEBRANCH - mentioned in #2147 - provides value like "refs/heads/FeatureABC", but I would like to see simply "FeatureABC".

Currently I'm using a PS script overwriting the build number while executing (##vso[build.updatebuildnumber]). This is not a final solution because the revision numbering is not correct:

1. The first PR build is named e.g. "MyTool_merge_20181101.1".
2. Due to PS script the build will be renamed to "MyTool_merge_FeatureABC_20181101.1". (Ok so far.)
3. The second PR build is named again "MyTool_merge_20181101.1", because (I assume) the revision numbering is based on original number format pattern.
4. So the second build will be renamed to "MyTool_merge_FeatureABC_20181101.1" and will overwrite the previous build.

[goodies4uall]

https://developercommunity.visualstudio.com/content/problem/529341/buildsourcebranch-variable-in-release-pipeline-is.html

[jeffmaher]

Also experiencing this issue with a GitHub repo that uses Azure Pipelines.

We have a build task that builds and pushes a Docker image for PR branches and tags, using the branch/tag name as the tag for the Docker image. However, Azure Pipelines populates the branch name with merge in PRs.

Here's the step:

  - task: Docker@2
    inputs:
      command: buildAndPush
      repository: cdssnc/hello-esdc-api
      tags: $(Build.SourceBranchName)  #<---- This is always "merge" in PRs
      Dockerfile: src/HelloESDC.API/Dockerfile

[cmeyertons]

+1 - I know changing this behavior would be a breaking change, it’s unfortunate this was not the original behavior as the variable “merge” is pretty useless

[ustun]

Try using $(System.PullRequest.SourceBranch)

However this is not defined in non-PR builds and results in an annoying issue where master builds have numbers of the format literally "$(System.PullRequest.SourceBranch)". I don't know whether there is any syntax for default values like $(System.PullRequest.SourceBranch: 'master')

[lukas-ais]

Try using $(System.PullRequest.SourceBranch)
However this is not defined in non-PR builds and results in an annoying issue where master builds have numbers of the format literally "$(System.PullRequest.SourceBranch)". I don't know whether there is any syntax for default values like $(System.PullRequest.SourceBranch: 'master')

System.PullRequest is not available/useful in non-PR builds. This is why I opened the request.

Further, as initially written: System.PullRequest.SourceBranch alias SYSTEM_PULLREQUEST_SOURCEBRANCH does not provide the data in expected format.

It would be helpful to have syntax for default values. But such a feature is unknown to me.

[ustun]

I wrote the following bash script to overcome this issue for now: https://gist.github.com/ustun/b14dc73d03dca3cf1319a08ee1772120

Hope you find it helpful.

[lukas-ais]

@ustun , did you read my initial statement completely?
I already mentioned that I HAVE a script (PowerShell) to update the build name. But there is the problem, that the automatic numbering will not as expected.

[ustun]

Yes, I read that. Due to that issue, in my version, I opted not to use automatic numbering (depending on git sha and date to make sure there are no conflicts.) Your original issue still remains, but maybe you don't need automatic numbering provided by Azure Devops.

[plompd]

@ustun Thanks for the information on this script. We also have a .ps1 script in our Pipeline where I have a $url as parameter coming from the pipeline. In this variable I try to give the branch name of the PR like this:

-url https://$(System.PullRequest.SourceBranch).test.mydomain.nl

For some reason this is still always 'merge'. I'm not sure if your bash script could be of help here?

[ustun]

@zimplerconsulting sorry, I haven't used azure in a while and my memory is a bit fuzzy. The way I used to debug this would be to output all the vars (env in bash). Maybe that helps in your case?

Also maybe you could trigger the ps1 script not via the pipeline, but from another ps1 script which has access to env vars? I vaguely remember doing funny stuff like that to overcome azure's shortcomings.

[lukas-ais]

Once again, I do not want to use a script to update the build name. This is only a work-around.

I would like to have a build variable providing the brach name, independent whether this a branch build or a merge build.

[davidorbelian]

I would like to have a build variable providing the brach name, independent whether this a branch build or a merge build.

I don't think that is real to expect it to be implemented with the break of the current behavior. The most possible way is to get a System.PullRequest.SourceBranchName and System.PullRequest.TargetBranchName.

Am I right that you have the same format for the Build Name for the master branch build and the pull request to the master branch build?

[lukas-ais]

I don't think that is real to expect it to be implemented with the break of the current behavior.

I do not want breaking behavior. A new variable is fine.

Am I right that you have the same format for the Build Name for the master branch build and the pull request to the master branch build?

Sample build name on master: MyProduct_master_20200608.1
Sample build name on PR: MyProduct_FeatureBranch_20200608.17

[miks]

Maybe scheme versioning it is? docker-compose have it, kubernetes have it, any professionally developed API have it.

[davidorbelian]

I do not want breaking behavior. A new variable is fine.

But you want a single build definition for both trigger builds and PR builds? With that new variable you will need two definitions, one using branch trigger and Build.SourceBranch and other using PR trigger and System.PullRequest.SourceBranchName.

Please also take a look at this comment on the #2147 (sadly the issue is closed and I hope that will be re-opened).

I've also opened a feature request, but it looks like it must get enough user upvotes to be reviewed by a human.

[lukas-ais]

But you want a single build definition for both trigger builds and PR builds?

Yes, I want a single build definition defined for master.
In case of PR build I want to see the branch name in the build name as mentioned above.

With the currently available variables this is not possible exception dynamic renaming the build.
But this causes invalid numbering of builds as mentioned at the beginning.

[davidorbelian]

Yes, I want a single build definition defined for master.
In case of PR build I want to see the branch name in the build name as mentioned above.

Please pay attention that as said in the Docs about System.PullRequest.SourceBranch:

This variable is initialized only if the build ran because of a Git PR affected by a branch policy.

If the new variable System.PullRequest.SourceBranchName will be added, I am sure the same policy will be applied. So if you still have only one build definition, you will have to choose between Build.SourceBranchName and System.PullRequest.SourceBranchName. That means you can not achieve what you want with that kind of new variable. I would recommend to have different build definitions for branch triggers and PR's, so you can use the corresponding variable.

If this is not suitable for you, then an another variable is needed to be added, which takes a value depending on whether the build is activated on branch trigger or PR, which is a less common case, less chance to get upvoted and less likely to be implemented.

[lukas-ais]

Start from the beginning: I asked why the variable SourceBranchName is changed to merge in case PR build.
Then I analyzed other variables like System.PullRequest.*.

Once again:
Yes, I want a single build definition defined for master.
In case of PR build I want to see the branch name in the build name as mentioned above.

In any case the variable SourceBranchName should have a VALID branch name, not a generated like merge.

I never asked explicitly to setup a new variable at System.PullRequest.*. At least I never planned to ask that.
Instead I tried to define a use case and I kindly ask for such a (new) variable.

[davidorbelian]

In this case, this means that you most likely did not quite understand this topic.

When you create a Pull Request, the service that provides this functionality (this is not a Git functionality out of the box) creates a temporary branch and merges history of Source and Target branches. That means when you get a CI trigger on a PR, your actual SourceBranch is not the Source of the PR or the Target of the PR, but the branch created by the service provider (GitHub, Azure Repos etc.). Usually it looks like refs/pull/{pullRequestId}/merge, which means it is a branch created to merge source and target branches for pull request with {pullRequestId}.

If you now take a look at the description of Build.SourceBranchName it says:

The name of the branch in the triggering repo the build was queued for.
Git repo branch or pull request: The last path segment in the ref. For example, in refs/heads/master this value is master. In refs/heads/feature/tools this value is tools.

So answering your question: The value of SourceBranchName is not overwritten. In the case of the refs/pull/{pullRequestId}/merge branch you get merge as the last path segment, and everything looks correct, as expected and as documented.

[miks]

So,

Yes, I want a single build definition defined for master.
In case of PR build I want to see the branch name in the build name as mentioned above.

is not possible because of Azure DevOps CI implementation.

In GitLab there is no need for double pipeline definition at all or some other magic.

[lukas-ais]

So answering your question: The value of SourceBranchName is not overwritten. In the case of the refs/pull/{pullRequestId}/merge branch you get merge as the last path segment, and everything looks correct, as expected and as documented.

Ok, thanks. Understood this detail.

Is it possible to create NEW variable providing the requested information?

[davidorbelian]

In GitLab there is no need for double pipeline definition at all or some other magic.

I do not have much experience working with GitLab. Could you tell me what the variable is called there, which in the case of the branch-triggered build returns the branch name, and in the case of PR-triggered build returns the name of the PR Source branch name?

Thanks in advance!

[davidorbelian]

Is it possible to create NEW variable providing the requested information?

I definitely cannot answer this question, because I have no relation to the development of this functionality. I myself am here to try to push through the idea of adding two new variables, System.PullRequest.SourceBranchName and System.PullRequest.TargetBranchName. The decision-making process begins with the creation of a topic in which users vote. Honestly, I have no idea how and who makes decisions about which features to implement. But I can definitely say that the easier it is to implement a feature and the more users need it, the more likely it will get a green light.

[miks]

I guess it's called CI_COMMIT_REF_NAME

[davidorbelian]

I guess it's called CI_COMMIT_REF_NAME

Looks like they had same behavior and fixed it about a year ago.

You can attach this link as an argument in favor of adding a new variable (or changing the behavior of the old one, but I doubt it is real).

[davidorbelian]

Take a look at Expressions. Here is an example of what you can do with it:

name: $[ variables['branchName'] ]

# Using conditions to set the variable value. 
# I guess if both conditions return true the last will be used.
variables:
  ${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/') }}:
    branchName: $[ replace(variables['Build.SourceBranch'], 'refs/heads/', '') ]
  ${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
    branchName: $[ replace(variables['System.PullRequest.SourceBranch'], 'refs/heads/', '') ]

trigger:
- master
pr:
- master

# your stages/jobs/steps go here

When I run this on a master push I had master build name.
When I run this on a PR to master from release/0.2.0 branch I had release/0.2.0 build name.

Also take a look at Templates. You can describe the above logic once and then reuse that in different repos using templates.

There may be other solutions or potential improvements on this, but I decided to stop on a simple demonstration of the expressions/variables principle.

Hope this helps you.

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days