Azure-devops-docs: More clarification on pipelineId argument

[atg-abhishek]

It isn't immediately obvious on how to retrieve the pipelineId and a search through the documentation pages with "pipelineId" returns a single result with this page. It would be great to have a link detailing what the pipelineId is and how one can obtain it to use within this task.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: b03d67d0-8738-72e6-dbe7-324c1784d4fc
• Version Independent ID: bbb4d774-cd98-3ecb-7ccf-5f49e6f2b425
• Content: Download Pipeline Artifact task - Azure Pipelines
• Content Source: docs/pipelines/tasks/utility/download-pipeline-artifact.md
• Product: devops
• GitHub Login: @erickson-doug
• Microsoft Alias: midenn

[infin8x]

@mitchdenny can you help @elbatk with the information needed to make this improvement? I think pipelineId can be taken from the URL of the build definition e.g. https://dev.azure.com/mseng/AzureDevOps/_build?definitionId=829, but I'm not 100% certain.

[abombss]

pipelineId appears to be BuildId, and not the build definition id. It needs the actual instance id of where the artifact is associated. I was able to make this work by referencing a release variable tied to the artifact alias. My alias is named "artifacts" and using $(RELEASE_ARTIFACTS_ARTIFACTS_BUILDID) did the trick. So the format would be $(RELEASE_ARTIFACTS_<alias>_BUILDID)

If you were trying to consume in a build and not a release pipeline you would need to somehow get the value of $(Build.BuildId)

I hope as this matures there are plans to make pipeline artifacts published from a build automatically in release, just like they are when using the old Build Artifacts. Currently for me that is not happening so I am forced to manually add this step to my release pipeline and associate it with the build pipeline.

The task output is also pretty strange. It reports no data downloaded, but success, and I do actually see the artifacts downloaded and placed in my target path.

2018-12-01T19:14:51.3824191Z ##[section]Starting: Download Pipeline Artifact
2018-12-01T19:14:51.3827063Z ==============================================================================
2018-12-01T19:14:51.3827214Z Task         : Download Pipeline Artifact
2018-12-01T19:14:51.3827258Z Description  : Download Pipeline Artifact
2018-12-01T19:14:51.3827481Z Version      : 0.139.0
2018-12-01T19:14:51.3827569Z Author       : Microsoft Corporation
2018-12-01T19:14:51.3827611Z Help         : Download named artifact from a pipeline to a local path.
2018-12-01T19:14:51.3827652Z ==============================================================================
2018-12-01T19:14:51.8296671Z Download from the specified build: #80000
2018-12-01T19:14:51.8296872Z Download artifact to: /home/vsts/work/r1/a
2018-12-01T19:14:56.2333182Z Information, Could not initialize dataport.
2018-12-01T19:14:56.2369011Z Information, Downloaded 0 out of 32,991 bytes (0%).
2018-12-01T19:14:56.7764748Z Information, Downloaded 0 out of 32,991 bytes (0%).
2018-12-01T19:14:56.7766511Z Information, Download is done!
2018-12-01T19:14:56.7789305Z Information, 
2018-12-01T19:14:56.7790538Z Download statistics:
2018-12-01T19:14:56.7791061Z Total Content: 0 bytes 
2018-12-01T19:14:56.7791556Z Physical Content Downloaded: 0 bytes
2018-12-01T19:14:56.7791627Z Compression Saved: 0 bytes
2018-12-01T19:14:56.7792224Z Local Caching Saved: 0 bytes
2018-12-01T19:14:56.7794636Z Chunks Downloaded: 0
2018-12-01T19:14:56.7794694Z 
2018-12-01T19:14:56.7809709Z Downloading artifact finished.
2018-12-01T19:14:56.7893013Z ##[section]Finishing: Download Pipeline Artifact

[infin8x]

Hmm, I thought we'd already shipped the ability to have Pipeline Artifacts downloaded automatically in release pipelines. @ashokirla or @mitchdenny any updates on that?

In general though, we are working on maturing the story around Pipeline Artifacts. You can see some of the designs here. Thanks for the feedback and for trying them out!

[gregbenner]

@abombss
Is $(Release.Artifacts.drop.BuildId) available in the release pipeline automatically? I get the following error.
2019-01-06T02:30:54.1418454Z /home/vsts/work/_temp/92adcbd9-3fb9-44cb-bac5-70f166f88214.sh: line 2: Release.Artifacts.drop.BuildId: command not found

[mitchdenny]

We have the automatic injection code done but we are waiting on a few updates/deployments before it reaches customers. There was 1 specific scenario where automatic injection works today, in all other cases you need to insert the task manually. It will automatically pick up the build ID for the primary artifact.

Once we have the auto injection bugs fixed it should just work for you (that is to say, we've done the work but its still in the release pipeline).

@gregbenner to answer your question - the ID would be:

Release.Artifacts.{alias}.BuildId

The alias is whatever the artifact is aliases as in the release definition.

[tomjaguarpaw]

I would also like to understand how to make this work effectively. I can download artifacts successfully using a hard coded value for pipelineId but how can I refer to "the last build of the master branch", for example?

[abombss]

If you are in a release pipeline it should be available given the answer in the thread. Reference the artifacts alias buildid. If you are in a build pipeline you will have some additional scripting to locate the buildid of the last build of the master branch. Although I would ask what is the use case to need artifacts in a build pipeline from a prior build? Typically the build is responsible for publishing the artifacts and the release pipeline would consume them directly.

[tomjaguarpaw]

If you are in a build pipeline you will have some additional scripting to locate the buildid of the last build of the master branch

Yes, I'm in a build pipeline. Could you please tell me how to do this?

Although I would ask what is the use case to need artifacts in a build pipeline from a prior build?

I'm not sure what you mean. I have a build that publishes artifacts. I have another build that wants to use those artifacts. Is this somehow an incorrect setup?

[elbatk]

Hey @mitchdenny , bringing you back into the fray to help out here and let me know if we have docs issues that need updating!

[mitchdenny]

@tomjaguarpaw when we designed the Pipeline Artifacts feature our assumption was that most folks would either use Pipeline Artifacts to copy content from job to job within the same pipeline (let's call this scenario 1). The other scenario was using a "build pipeline" as an input to a "release pipeline" (let's call this scenario 2).

What we didn't expect so much was people wanting to use Pipeline Artifacts to download artifacts from a completely separate pipeline … so you've got two independent pipelines, and pipeline A wants to consume a Pipeline Artifact from pipeline B (let's call this scenario 3).

So we cater for scenario 1 & scenario 2 very easily today. If you have a Pipeline Artifact being produced in one job you can simplify specify the name of the pipeline artifact to download it into the next job. Here is an example of the YAML you would use.

steps:
- task: DownloadPipelineArtifact@0
  inputs:
    artifactName: my-artifact-from-earlier-job
    targetPath: $(System.DefaultWorkingDirectory)\my-artifact

For scenario 2, we've also got you covered. You just drag in the Download Pipeline Artifact task in the designer for RM, no need to specify the Pipeline ID because it detects it based on the BUILD_ID associated with the RM job.

In the preview there were some rough edges that we've now addressed and are rolling out. First we didn't provide an easy way to select non-primary builds as the source for artifacts to download in RM. We've to an updated version of the task rolling out soon that addresses that.

Second - in general the download task wasn't very friendly with the pipelineID being an ID that changed all the time based on the builds that had run. So we are adding options to select pipeline definitions and various other selection criteria - this change is also rolling out.

For now - while you are still using the V0 task, the only option is to use the REST APIs to get the latest version. However this is going to be solved as soon as we deploy our updated task.

Alternatively, you could use the Azure CLI to grab the latest build:

az pipelines build list --organization https://dev.azure.com/yourorg --project yourproject --definition-ids 123 --top 1 --query "([].id)[0]"

You would then capture that in a pipeline variable using the ##vso set variable command (docs here).

Once again - this will be much simpler in the near future.

[Stoobs]

@mitchdenny What was the solution for scenario 3?

I've two pipelines, one which is purely for ARM template projects, and another which is where the application code is built and deployed from.

What I was hoping to do was use the build artifacts (ARM template) from the first pipeline in the second pipeline to create a VM, deploy the code to it, then run various tests on the code in the VM, eg coded UI tests, load tests, pen-tests etc as required.

I didn't really want all the ARM template projects included in the same repo as the application solution, so the download pipeline artifacts task seemed like the ideal solution?

[mitchdenny]

We have an updated version of the task rolling out that will make downloading the latest version from another pipeline easier (this will be the V1 task), the underlying implementation is here:

https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Plugins/PipelineArtifact/PipelineArtifactPluginV1.cs

Things have to arrive in a certain order so that we don't break folks, but it is on the way. In addition, for YAML pipeline users we are coming up with a simplified syntax for wiring this all up:

https://github.com/Microsoft/azure-pipelines-yaml/pull/125

[chasewilson]

Hi all! I appreciate the feedback. It looks like the original question has been answered and there will be an update to the task documentation to have readily available info soon. I am closing this issue, however, if you have any more questions or feedback, feel free to open a new issue!