upload-artifact double-zips a zip

[kcgen]

When upload-artifact is provided an existing zip file in the path::

        with:
          name: asset-package-${{ env.VERSION }}-linux-x86_64
          path: asset-package-${{ env.VERSION }}-linux-x86_64.zip

It double-zips the zip, resulting a zip within a zip.

_"You're using it wrong, don't provide a zip file!"_, might be a counter argument. However, there are existing workflows and build chains that automatically build zip files on the fly, and it would make logical sense for the resulting asset zip-file to be used "straight away", instead of re-zipped (which needlessly burns CPU, always results in a larger file, and wastes user and down-stream script time double-unzipping).

Yes, a work-around would be to unzip our zipfile to a temporary directory, and then provide that to the asset zipper - however we're now wasting three rounds of zipping. (initial zip, unzip, re-zip).

_"But we can't trust your zipfile; what if it's actually some other file in disguise!"_, could be another argument. Ok.. but the asset zipper already trusts any file anyway; so security already isn't a concern (people could already be providing viruses and trojans to the asset zipper for all we know.)

If this is a concern, then a "content/type" check could be applied to confirm it's a bonafide zip-file; or the zipfile could be decompressed to /dev/null (and confirmed good), as opposed to double-zipping the zip.

Overall, GitHub provides developers full control over the content that's provided in the asset zipfile.
Using a provided zipfile straight away is just another means to provide the same content without a second layer of packaging.

[konradpabjan]

This is an issue on the download-artifact side with how the server downloads/streams the artifacts.

This will be fixed with the v2 versions of the actions that are currently in-development. The behavior right now for this is really annoying 😕

[kcgen]

@konradpabjan , thanks!

This is an issue on the download-artifact side with how the server downloads/streams the artifacts.

Very interesting knowing that the assets aren't being zipped by the upload action (potentially wasting space?), and instead the zipping is done by the downloader service.

Well, I'm not picky about what happens behind the curtain, but happy you're on this and that we won't have zipped-zips come v2!

[filips123]

@konradpabjan When will v2 be released?

[eine]

Ref https://github.com/actions/upload-artifact/issues/3#issuecomment-526673109

[konradpabjan]

We're planning on having a v2-beta sometime around mid-January (possibly a little earlier than that but I don't want to over-promise 😊)

[mk-pmb]

I'd really appreciate if we could get uploading of verbatim single files going asap, so I can avoid setting up an interim FTP upload.

[pjoe]

Ran into this as well. Using 7zip as part of the build is order of magnitude faster than having upload-artifact handle the zipping.

For our specific repo timings are (3719 files, 150MB uncompressed, 44MB compressed):

| Operation | Time |
|---|---|
| Use 7zip to compress | 12.3s |
| Upload (and zip) .zip | 15s |
| Use upload-artifact to zip and upload | 2m 36s |

[kcgen]

We're planning on having a v2-beta sometime around mid-January

Thanks @konradpabjan -- just checking in on how this is going; any news?

[konradpabjan]

The v2-preview is out now! You can see the pinned issue for more information.

Initially I thought we would get this done for v2 but unfortunately this is not the case 😞

Basically, this double-zip isn't caused by how the artifact get uploaded, but rather it's a limitation of the GitHub UI in terms of how you can later download the artifact. The long term fix is to offer a more enhanced experience in terms of how to download artifacts (browse and download individual components from the UI). For v2 we were considering some back-end changes/hack to prevent extra zipping if a zip file is already detected, but we steered away from that.

See my comment here for more information: https://github.com/actions/upload-artifact/issues/3#issuecomment-598820814 (same things applies to an individual file that gets uploaded, a zip is the only supported way that you can currently download an artifact from the UI)

There is GZip compression that is used during upload (which I initially though was responsible for this, apologies for some of my conflicting comments earlier).

[schultzern]

+1

[konradpabjan]

@afwn90cj93201nixr2e1re The artifact double zip is currently a UI limitation and it's independent of how the artifact gets uploaded. There is no zip that gets created during artifact upload.

We don't have anything yet to allow users to individually download files from the UI so a zip gets dynamically created when you click to download the artifact (this is done so that all files are downloaded at once). Long term we have plans to enhance this significantly so you individually download files (and browse them) without any extra zipping. At the moment this is an unfortunate limitation that we have 😕

[afwn90cj93201nixr2e1re]

get u, just removed .zip from filename @konradpabjan

r u working on github? Can u try to explain why i should have write and read access to code (contents) to able send (access) dispatch to repo?
Coz /repo/dispatches not described at any point's.
https://developer.github.com/v3/apps/permissions/#permission-on-contents

Why i can call dispatch via read only accesses?
Why dispatch not described as another one option?

[carrotIndustries]

Being able to download files without the additional ZIP wrapper would be highly appreciated as I use actions to build ready-to-extract zip files and having to extract that zip from another zip just confusion among users.

[jimis]

As an update, the issue still exists with v2. I use upload-artifact@v2 to upload a single .tar.gz file of about 300MB size. When trying to download it on the UI, it's only possible to download a ZIP archive of the already compressed tarball.

There is also about 30s lag between clicking and actually starting the download; I assume zipping happens on demand when trying to download it, as @konradpabjan mentioned.