Azure-pipelines-tasks: Download Build Artifacts: : broken file permission mask

[omeshp]

Environment

• Server - VSTS or TFS on-premises?
  VSTS

Issue Description

Permissions on the files are lost when downloaded to the agent. We have some files set to be executable.

[omeshp]

To support retaining of file permissions we need to do changes across the board, first artifact service needs to support retaining metadata for files, then "publish build artifacts" tasks needs to upload the artifacts with permissions. After which "Download build artifacts" task can be updated to honor permissions.

Previous download artifacts in release was writing all files as executable for some agent configurations which isn't correct thing to do.

@bryanmacfarlane Assigning this issue to you since it first requires retaining of file permissions in build artifact.

[schultetwin1]

@bryanmacfarlane have you had anytime to look at this?

We have a CI pipeline that is hitting this. We are using hosted agents to build C++ code and create Linux binaries. These binaries are tests which run against custom hardware. So we upload the binaries as artifacts and download them onto our Self-Hosted agent.

Currently to workaround this issue we manually chmod the binaries before running them but that solution has been fragile.

[afeblot]

We took a more generic approach to work around this:
we just create a tgz of what is supposed to be content of the artifact, and have this tgz in the artifact instead. And expand it on the other side.
But really, we should not have to spend time on such issues...

[mavinm]

Any status on this? It's been over a year since this bug was reported.

[omeshp]

@mitchdenny Are we supporting retaining of permissions in pipeline artifacts? I know it's on the agenda but wondering if it's already implemented.

[mitchdenny]

At this point we don't do it. I think the first priorities for additional metadata to be stored with files in artifacts are:

1. Execute bit
2. Timestamps (may be important for native build caching scenarios)

Both of these are important for both Pipeline Artifacts and Pipeline Caching. Adding @alexmullans for awareness.

[ghost]

I'm also having this issue. Is there a workaround?

[mavinm]

See afeblot@ comment for workaround. Simply make a zip/tgz.

[ghost]

Still not working. Keep getting issues with "File not found" errors even though I list out the files right above the command and it exists.

Submitted an issue at https://developercommunity.visualstudio.com/content/problem/578755/im-unable-to-use-extracted-files-in-a-release-pipe.html

[johncrim]

This is still an issue a year and a half later - seems too significant a problem to have not been addressed, IMO.

[jmalak]

The same irritating problem here. Artifacts contains executable and pipeline fails because executables can not be run. It should be fixed as urgent because it is fatal problem. It is main purpose of Artifact use some executable created before.

[nmiodice]

We have a quick (and dirty) workaround that involves manually setting the permissions we want:

# Setting the scripts to be run as executable
chmod -R $WHATEVER_PERMISSIONS $YOUR_FILE_OR_DIRECTORY

Can we get an update on the status?

[drdamour]

this is an issue with pipeline artifacts too. AND it causes issues with terraform because best practice of terraform is to artifact and restore the whole .terraform directory and the x bits on the providers are lost. had to workaround with

chmod -R a+x ./.terraform/plugins/linux_amd64/*

[jahead]

@mitchdenny Are we able to get an update on this? It's been over a year, and no feedback?

[mitchdenny]

Adding @johnterickson

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[drdamour]

Stale but important

[jmalak]

It was already resolved by adding note in artifact documentation that permission are not preserved due to Windows filesystem.
Don't expect any solution. It works as designed. Nice way how to fix problem 🥇

[jahead]

@jmalak Maybe I've missed something in the history of this issue and the issues referencing this one. But no where was it communicated as resolved AFAIK or working as designed.

We have a workaround

@mitchdenny Are we supporting retaining of permissions in pipeline artifacts? I know it's on the agenda but wondering if it's already implemented.

Would suggest that there was the intent of at least the possibility supporting this natively.

Hopefully we get some feedback soon

[jmalak]

It should be resolved proper way, what I commented is Microsoft quick solution.
I am using my own solution that I encapsulate artifact into archive with permission and if I need it I restore it from archive (7z format on Windows and tar.gz on NIX).