Azure-pipelines-tasks: Publish Build Artifacts does not work on release definition

[AlexHimself]

The utility task Publish Build Artifacts doesn't appear to work correctly. I've created a simple repro scenario to demonstrate. The cause I can initially think of is that this is running via the on-prem Agent, but I don't think that should matter because it just downloads files locally then uploads/publishes the local artifacts back to the server.

The ultimate error I get is:

2017-01-10T19:32:05.2129999Z ##[error]Unable to process command '##vso[artifact.upload artifacttype=container;artifactname=MyLogsArtifactName;containerfolder=MyLogsArtifactName;localpath=C:\agent\_work\r2\a\UploadLogFiles\MyLogs;]C:\agent\_work\r2\a\UploadLogFiles\MyLogs' successfully. Please reference documentaion (http://go.microsoft.com/fwlink/?LinkId=817296)

2017-01-10T19:32:05.2159990Z ##[error]Value cannot be null.

Steps to reproduce

Initial Setup

Create C:\Temp\VSTS directory and add two dummy files "Test2File.txt" and "TestUploadFile.log".
These represent misc logs that may be collected during a build.

Build Definition

Create a single PowerShell build step with "Type=Inline Script" and use the following code (see screenshot too):

Write-Host "##vso[artifact.upload containerfolder=MyLogs;artifactname=MyLogs;]C:\TEMP\VSTS\Test2File.txt"
Write-Host "##vso[artifact.upload containerfolder=MyLogs;artifactname=MyLogs;]C:\TEMP\VSTS\TestUploadFile.log"

Write-Output "Uploaded 2 files..."

Run this build! And explore/verify the files were uploaded correctly (see screenshot).

Release Definition

Create a new, simple release definition according to the following screenshot, where you select the "Path to Publish" as the directory from the build step. Choose "Server" for artifact type.

Run this release and you'll receive something similar to the error below:

2017-01-10T19:32:04.9330002Z ##[section]Starting: Publish Artifact: MyLogsArtifactName

2017-01-10T19:32:04.9599998Z ==============================================================================

2017-01-10T19:32:04.9599998Z Task         : Publish Build Artifacts

2017-01-10T19:32:04.9599998Z Description  : Publish Build artifacts to the server or a file share

2017-01-10T19:32:04.9599998Z Version      : 1.0.39

2017-01-10T19:32:04.9599998Z Author       : Microsoft Corporation

2017-01-10T19:32:04.9599998Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=708390)

2017-01-10T19:32:04.9599998Z ==============================================================================

2017-01-10T19:32:05.2129999Z ##[error]Unable to process command '##vso[artifact.upload artifacttype=container;artifactname=MyLogsArtifactName;containerfolder=MyLogsArtifactName;localpath=C:\agent\_work\r2\a\UploadLogFiles\MyLogs;]C:\agent\_work\r2\a\UploadLogFiles\MyLogs' successfully. Please reference documentaion (http://go.microsoft.com/fwlink/?LinkId=817296)

2017-01-10T19:32:05.2159990Z ##[error]Value cannot be null.

Parameter name: containerId

2017-01-10T19:32:05.2770001Z ##[section]Finishing: Publish Artifact: MyLogsArtifactName

[ericsciple]

@GitHubSriramB it looks like this is confusion due to RM not considering the visibility filter on the task.

[GitHubSriramB]

Not honoring the visibility filter in RM is as designed. We had added logic in Publish artifacts task to fail when used in Release with appropriate error message. Looks like this logic got regressed. We should fix it.

[GitHubSriramB]

We will take up fixing this issue.

[Rurouni]

I have run into same issue.

We had added logic in Publish artifacts task to fail when used in Release with appropriate error message

Does it mean it is not possible to publish artifacts from Releases by definition?

[GitHubSriramB]

That's right. We currently don't support publishing artifacts from a release.

[GitHubSriramB]

We have improved the error message when artifact.upload command is used from any task when running within Release. This fix will be available in 2.114.0 onwards.

[AlexHimself]

@GitHubSriramB & @ericsciple - I know it's been a year, but I ran into the same scenario again (forgetting that artifact.upload does not work from the release side. The error is not clear though as @GitHubSriramB last message mentioned.

I get this:

2018-05-10T01:40:06.0800268Z ##[error]Unable to process command '##vso[artifact.upload containerfolder=AXLogs;artifactname=AXLogs;]D:\AWK' successfully. Please reference documentation (http://go.microsoft.com/fwlink/?LinkId=817296)
2018-05-10T01:40:06.0814554Z ##[error]Value cannot be null.

So is there anyway to upload files back during a release?

Part of my release to production process is running import commands that produce logs. I'd like to upload/associate the various log files with the releases. I don't want to dump the log file contents back to the release if I can keep them in their files.

[brownbl1]

We also have a need for this ability. We are generating a report (an html performance test) after a deployment step in a release definition and would like to be able to upload that html file to the release for linking to a dashboard eventually but for now just for auditing purposes. We considered shaping the data to look like a test, but we really need the ability to upload the html file, not xml.

[dci-aloughran]

Has anyone got a workaround for this not working on release? I have a file generated that's specific to the first agent phase of a release, and I need to pass it to a second agent phase.

[Boaz101]

Publishing log files from a release would be super useful. The only workaround would be to feed them to the built in logging. However that is much slower when the logs are large.

[bryanmacfarlane]

Reopening to answer question for @dci-aloughran and @Boaz101

@GitHubSriramB - can you confirm whether publishing from release is planned?

[AlexHimself]

@dci-aloughran @Boaz101 @brownbl1 I wrote this PowerShell function as a work-around that I just call from either a Build/Release and it should upload a directory to either.

The main difference being for Releases using "##vso[task.uploadfile]" and for builds "##vso[artifact.upload...", where it seems like builds would allow a directory upload and releases would allow individual file upload.

So for releases, I just loop through a provided directory. Works well enough for my purposes.

function KWVSTSUploadDir()
{
    [CmdletBinding()] 
    param 
    ( 
    [Parameter(Mandatory=$true, Position=0)] 
    [string]$Directory,
    [Parameter(Mandatory=$false, Position=1)] 
    [string]$ContainerFolder = "Misc",
    [Parameter(Mandatory=$false, Position=2)] 
    [string]$ArtifactName = "Misc"
    )

    begin {
        Write-Host "Uploading...$($Directory)"
    }
    process {
        switch ($env:SYSTEM_HOSTTYPE)
        {
            'release' {
                Write-Host "Detected Release...uploading files individually..."

                foreach ($file in (Get-ChildItem -Path $Directory))
                {
                    Write-Host "Release: Uploading $($file.FullName)"
                    Write-Host "##vso[task.uploadfile]$($file.FullName)"
                }
            }
            'build' {
                Write-Host "Detected Build...uploading folder..."
                foreach ($file in (Get-ChildItem -Path $Directory))
                {
                    Write-Host "Build: Uploading $($file.FullName)"
                }
                Write-Host "##vso[artifact.upload containerfolder=$ContainerFolder;artifactname=$ArtifactName;]$($Directory)"
            }
        }
    }
    end {
        Write-Host "Finished uploading for $($env:SYSTEM_HOSTTYPE)"
    }   
}

[brownbl1]

That's helpful. Thanks @AlexHimself

[bergmeister]

@bryanmacfarlane Yes, being able to publish from release is critical to upload error logs (e.g. WiX installation logs) to analyse deployment failures quicker and being able to fix production ASAP

[GitHubSriramB]

vso[task.uploadfile] can be used to upload files to release. We don't have plans to support uploading folder/artifacts from release in near future.

[MichelZ]

Real shame. I think this would be a nice addition. E.g. we built the following pipeline:

• Build definition with Unit Tests, builds code
• Release definition deploying release to Azure
• newman Tests in release definition to run tests against the deployed Infrastructure

Would be nice to upload the tests back to the Artifacts.

[GitHubSriramB]

Thanks for the quick response/feedback.

We will consider adding this support within Release & are will track it in our backlog.

@MichelZ - couple of queries about the scenario you mentioned:

• By "newman tests", are you referring to manual tests?
  
  
  
  • Would you like to upload the tests themselves as artifact within release?
  
  

[MichelZ]

@GitHubSriramB
No, I am referring to tests of the API that are written using Postman (www.postman.com). newman is the "Command Line" execution part which can take a Postman collection and environment, and execute the written API tests.

There is a nice marketplace extension:
https://marketplace.visualstudio.com/items?itemName=carlowahlstedt.NewmanPostman

Ideally, I would like to upload the test results (publish test results) so that they can be displayed somehow like this:

[GitHubSriramB]

@MichelZ - Thanks for the details.

"Publish Test Results" task can be used within release and we support viewing the test results within the release environment from where the tests were executed.

Let us know if you face issues in doing this with newman based test execution.

[daantimmer]

Hello @GitHubSriramB ,

There is a need within our team for this functionality as well. After all automatic builds have succeeded a manual test step is required. This is a manual test report, images, scanned signatures etc. This should all be contained in a zip file and published during an environment.

I've setup a Jenkins job to ask for file input (queue from TFS) which is then fed back in to TFS (Download artifact from Jenkins), but the Publish artifact fails with the message:

##[error]This task can be used to publish to server drop within a Build job only.

[ChrisProlls]

Same need here, any news about it ?

[usamaisb]

I also need the "publish artifacts" ability in release pipelines just like we have available in Build pipeline.
Dont want to use that power-shell command to publish the artifacts because that is not the suitable way to publish in a pipeline.

[kriti218]

Is there any update on this? "uploadFile" uploads with the logs of the release definition. do we have a way to publish artifacts the way we do in build definition? Will "uploadFile" work in build definition also?

[joakimglysing]

I'm using Cypress (end 2 end testing) that generates screenshots, these are attachet to the zip generated from release-pipeline-button "Download all logs" in this way:

Powershell-task (inline):

Compress-Archive -Path "$(System.DefaultWorkingDirectory)/TestFolder/ScreenShots" -DestinationPath "$(System.DefaultWorkingDirectory)/TestFolder/ScreenShots" -Force
Write-host "##vso[task.uploadfile]$(System.DefaultWorkingDirectory)/TestFolder/ScreenShots.zip"

[huserben]

If anyone is interested, I created an extension that allows to _publish_ files to the logs (via the same mechanic as @joakimglysing is mentioning) and also offers a task to download those artifacts again in a later stage.

It can be found on the Marketplace and on github.

[patrickbadley]

@dci-aloughran @Boaz101 @brownbl1 I wrote this PowerShell function as a work-around that I just call from either a Build/Release and it should upload a directory to either.

The main difference being for Releases using "##vso[task.uploadfile]" and for builds "##vso[artifact.upload...", where it seems like builds would allow a directory upload and releases would allow individual file upload.

So for releases, I just loop through a provided directory. Works well enough for my purposes.

function KWVSTSUploadDir()
{
  [CmdletBinding()] 
  param 
  ( 
  [Parameter(Mandatory=$true, Position=0)] 
  [string]$Directory,
  [Parameter(Mandatory=$false, Position=1)] 
  [string]$ContainerFolder = "Misc",
  [Parameter(Mandatory=$false, Position=2)] 
  [string]$ArtifactName = "Misc"
  )

  begin {
      Write-Host "Uploading...$($Directory)"
  }
  process {
      switch ($env:SYSTEM_HOSTTYPE)
      {
          'release' {
              Write-Host "Detected Release...uploading files individually..."

              foreach ($file in (Get-ChildItem -Path $Directory))
              {
                  Write-Host "Release: Uploading $($file.FullName)"
                  Write-Host "##vso[task.uploadfile]$($file.FullName)"
              }
          }
          'build' {
              Write-Host "Detected Build...uploading folder..."
              foreach ($file in (Get-ChildItem -Path $Directory))
              {
                  Write-Host "Build: Uploading $($file.FullName)"
              }
              Write-Host "##vso[artifact.upload containerfolder=$ContainerFolder;artifactname=$ArtifactName;]$($Directory)"
          }
      }
  }
  end {
      Write-Host "Finished uploading for $($env:SYSTEM_HOSTTYPE)"
  }   
}

This is my simple inline powershell task to publish some files. Note that you can get the files by clicking download all logs and navigating down to your task folder.

foreach ($file in (Get-ChildItem -Path "$(System.DefaultWorkingDirectory)/mydirectory/screenshots" -Filter *.PNG -Recurse))
{
    Write-Host "Release: Uploading $($file.FullName)"
    Write-Host "##vso[task.uploadfile]$($file.FullName)"
}