Azure-devops-cli-extension: Commands to get artifacts produced in a build

[jwittner]

Quite possible I'm missing it in all the JSON, but I'm trying to write a script such that I can download the artifacts for a build and then do things with it. From what I can tell, the information about builds that I get doesn't include a download URL for the artifacts.

What am I missing?

[willsmythe]

@jwittner - it's likely the artifact URL for the build is not included in the JSON.

You can take the _links.self URL for a resulting build and append /artifacts to the end to get the list of artifacts for a specific build. See: https://docs.microsoft.com/en-us/rest/api/vsts/build/artifacts/list

[jwittner]

Thanks! That was just the trick I needed! I still like the enhancement of including it explicitly, but feel free to close this if you feel it's unlikely to happen.

[willsmythe]

@jwittner - we will keep the issue open and use it to add better support for querying and retrieving build artifacts.

[geverghe]

Building the commands to download the artifacts from a build run:

Scenarios:

1. List the build artifacts generated in a build run
2. Download build artifacts generated in a build run

Commands:
az pipelines run artifact list
az pipelines run artifact download

[mitchdenny]

@geverghe you can assign this to me. Quick question on terminology - is run what we are using to describe an instance of a pipeline?

So therefore:

Definition --> Pipeline
Build/Release --> Run

We'll also have an upload command as well for integration scenarios.

[geverghe]

As per our discussion with the pipeline team, there is a change in the pipeline commands:

Create pipelines: az pipelines create
List pipelines: az pipelines list - This lists the pipelines by names and provides the last run status for each.
Show pipeline: az pipelines show - This shows one item in the entire list filtered based on pipeline ID/Name
Delete pipeline: az pipelines delete
Update a pipeline: az pipelines update - Update the pipeline associated meta and not the YAML content

Create a pipeline run: az pipelines run
List all pipeline runs: az pipelines runs list - Can be filtered for a pipeline, for a branch (support all filter parameters supported in API)
Show details of a particular pipeline run: az pipelines runs show --run-id

1. Why the change?
   There is a first class notion of run and associated run management in pipelines and therefore the split into az pipelines and az pipelines run command groups. As per the above nomenclature, the updated proposal for artifact list and download would be
   az pipelines runs artifact list --run-id
az pipelines runs artifact download --run-id

Let me know if you need any additional clarification. Open to feedback on the verb as well.

[mitchdenny]

That's fine - I just wanted to know what it was going to be :)

[geverghe]

@mitchdenny is there a timeline on this?

Also, quick update. The subgroup is "runs" rather than "run". The reason is that run is the command and cannot be re-used for sub groups. I have updated the proposal above as well.

[mitchdenny]

I don't have a timeline yet - we've done the plumbing in ArtifactTool but need to punch it through to the AZ CLI extension. Its on the backlog.

[mitchdenny]

@geverghe are we happy with the pluralization of this:

az pipelines runs artifact

Why not pluralize artifact?

[geverghe]

Not elated - open to suggestions on sub group names to hold everything pertaining to pipeline runs.

The problem is that run is a command, i.e., az pipelines run, and therefore run cannot be a sub group name. Therefore, including the plural at the artifact level will not fix the issue.

[mitchdenny]

Ok. That makes sense. You legitimately got a verb called run (start a run) and a noun (a list of runs).

[gauravsaralMs]

@mitchdenny : The ask here is for build artifacts, will pipeline artifact command also support downloading of build artifacts (or are these the same)?

[mitchdenny]

This just does Pipeline Artifacts at this stage. Let me take a look at the FCS API to see what is involved supporting that.

[mitchdenny]

@gauravsaralMs - I should clarify Pipeline Artifacts are the replacement for File Container in the hosted service, so I guess they are the new build artifacts. But we'll need to support both in this command - so that is something that I'll look into.

[atbagga]

I should clarify Pipeline Artifacts are the replacement for File Container in the hosted service, so I guess they are the new build artifacts. But we'll need to support both in this command - so that is something that I'll look into.

@jwittner Pipeline Artifacts commands are available in the latest version of the extension (0.4.0).
You can list, download and upload the artifacts (given the restriction of file containers which will be supported soon and when that functionality lands, you won't need to upgrade the extension for it)

Do give it a try and let us know any feedback.

[atbagga]

@geverghe @mitchdenny Can we close this issue now?

[jwittner]

This is super cool!

One piece of feedback for the api - when I list/show builds they have an id. But when I go to get artifacts I use run-id. These appear to be the same value, but use different terminology, which was originally confusing to me. Probably too late, but perhaps build-id or buildid (matches the url query string for build results) would be more clear. A hint in the help would be nice if it is too late. =)

Browsing around seemed to work pretty well, but downloading failed for me with:

az : ERROR: Encountered an unexpected error.
At line:1 char:1
+ az pipelines runs artifact download --run-id 8968 --artifact-name 'Pr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ERROR: Encountered an unexpected error.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

ArtifactTool.Commands.PipelineArtifacts.PipelineArtifactDownloadException: Artifact with name 'ProjectTemplateUnity CI_master_2019.81.8968.0' was found, but is of type 'Container'.
   at ArtifactTool.Commands.PipelineArtifactDownloadCommand.ExecuteAsync() in D:\a\1\s\src\ArtifactTool\Commands\PipelineArtifacts\PipelineArtifactDownloadCommand.cs:line 41
   at ArtifactTool.Commands.CommandBase.OnExecuteAsync() in D:\a\1\s\src\ArtifactTool\Commands\CommandBase.cs:line 113
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(MethodInfo method, Object instance, Object[] arguments) in C:\projects\commandlineutils\src\CommandLineUtils\Conventions\ExecuteMethodConvention.cs:line 77
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context) in C:\projects\commandlineutils\src\CommandLineUtils\Conventions\ExecuteMethodConvention.cs:line 62
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext() in C:\projects\commandlineutils\src\CommandLineUtils\Conventions\ExecuteMethodConvention.cs:line 25
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass126_0.<OnExecute>b__0() in C:\projects\commandlineutils\src\CommandLineUtils\CommandLineApplication.cs:line 505
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute[TApp](CommandLineContext context) in C:\projects\commandlineutils\src\CommandLineUtils\CommandLineApplication.Execute.cs:line 31
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](CommandLineContext context) in C:\projects\commandlineutils\src\CommandLineUtils\CommandLineApplication.Execute.cs:line 145
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync[TApp](IConsole console, String[] args) in C:\projects\commandlineutils\src\CommandLineUtils\CommandLineApplication.Execute.cs:line 130

I'm assuming this is due to the File Container restriction you mentioned above @mitchdenny ?

[mitchdenny]

Actually - we are snapping to some future terminology we are adopting around Pipelines, and runs (could be either a build or a release conceptually - but it equals a buildId right now).

[atbagga]

@jwittner Lots of functionality planned around pipelines, check out #333. As Mitch mentioned this will all make sense with the other commands coming soon. :)

Keep us posted on the your feedback for CLI and any issues in general.

[jwittner]

@mitchdenny - mind confirming if the error I got downloading is in fact due to the 'File Container' support?

[mitchdenny]

@jwittner thats correct.

[manishsat]

Any plans when Container type artifacts are going to be supported?

[johnterickson]

No plans. You can grab a Container directly from REST. I think it's downloadUrl from https://docs.microsoft.com/en-us/rest/api/azure/devops/build/artifacts/get%20artifact?view=azure-devops-rest-5.1#artifactresource