Upload-artifact: [feature request] Allow conditional uploads

[Niceplace]

Hi folks ! I would like to request a feature.

Here is an example from an issue we are currently having, where we want to upload artifacts on a certain condition. We want to upload screenshots that are generated with cypress but only when the test run has failed (cypress manages the folder creation if it needs to create screenshots).

Currently, we have to add another step to verify that the folder exists and based on the output of that step, perform the upload. This makes the workflow a bit difficult to manage because on a run of that workflow where everything is successful, the step that checks the existence of the screenshot folder will be in a failure state and that can be confusing.

      - name: Check if screenshots folder exists
        if: always()
        run: test -d cypress/screenshots

      - name: When present, upload screenshots of test failures
        uses: actions/upload-artifact@v1
        if: success()
        with:
          name: cypress-screenshots
          path: cypress/screenshots

Could it be possible to have a implicit check, not as a default behaviour but with an option to skip the upload on a given condition ? Something like:

      - name: When present, upload screenshots of test failures
        uses: actions/upload-artifact@v1
        with:
          name: cypress-screenshots
          path: cypress/screenshots
          condition: folder-exists

folder-exists could map to an internal check that equals to test -d or any other equivalent. Different conditions could be implemented.

Maybe what I am asking it out of scope for this action but I thought I would take my chances.

Cheers !

[konradpabjan]

Thanks for the suggestion @Niceplace!

We recently released the v2-preview, more information can be found here: https://github.com/actions/upload-artifact/issues/62

The behavior has changed significantly between v1 and I think your specific problem can be solved. With v2-preview, you can use wildcards so you try something like this:

        uses: actions/upload-artifact@v2-preview
        with:
          name: cypress-screenshots
          path: cypress/screenshots/* # or maybe cypress/**/*

Behind the scenes, we use glob patterns to evaluate the provided path (instead of checking for the directory) so if no files are found to upload, no artifact will be uploaded and a warning will show up. Most importantly, the upload step also won't fail.

Would this new behavior work?

[Niceplace]

Yes ! The most important for me is that the step does not fail if the folder is empty or does not exist, that's awesome @konradpabjan thanks ! I'll try it out.

[ax4]

Thanks for this awesome [feature request] ! I have benefit from this v2-preview feature to conditional upload my artifacts! Thank you so much!

But I would like to add one more question, is there any environmental variable where I could set the warning-level lower, so I don't need to see the warning shows up? Since some "no files" behavior is known to our team's workflow, maybe it's not too important to see those missing uploads. But for those important files expected, a warning of missing is still wanted? - that means a customizable warning-level setting would be nice! Thank you!

[ax4]

@ax4 Thanks for this awesome [feature request]! I have benefit from this v2-preview feature to conditional upload my artifacts! Thank you so much!

But I would like to add one more question, is there any environmental variable where I could set the warning-level lower, so I don't need to see the warning shows up? Since some "no files" behavior is known to our team's workflow, maybe it's not too important to see those missing uploads. But for those important files expected, a warning of missing is still wanted? - that means a customizable warning-level setting would be nice! Thank you!

Hi, I would like to share my fix to my own problem! So people coming to this issue could read different fixes to their own specific needs.

It looks clear from @konradpabjan 's answer that after the release of v2-preview on Mar 20, it is said:

Behind the scenes, we use glob patterns to evaluate the provided path (instead of checking for the directory) so if no files are found to upload, no artifact will be uploaded and a warning will show up. Most importantly, the upload step also won't fail.

And from the issue tracker #62 we could read @konradpabjan 's another comment there https://github.com/actions/upload-artifact/issues/62#issuecomment-620682080 on April 28, says:

v2 upload-artifact has been merged into master! Thanks everyone for the feedback and testing!
https://github.blog/changelog/2020-04-28-github-actions-v2-artifact-actions/

You can now use it by doing actions/upload-artifact@v2. I'll be deleting the v2-preview branch soon (probably in a week or so), so please switch over so your builds don't break.

Moving forward it should be a lot easier now to accept contributions and add features since this effectively decouples artifact upload from the runner

Therefore, so far, it's very clear that upload-artifact@v2 is ready-to-use!

• BTW, in the official document of https://help.github.com/en/actions/configuring-and-managing-workflows/persisting-workflow-data-using-artifacts the examples are still using artifact!@v1 as examples. 😉

Then back to the question of whether to use and how to use conditional uploads, with and without given the warning. I would suggest the following three steps (as a better practice):

1. use upload-artifact@v2, which will give you a better experience of the file (artifact) upload:
   i. it uses glob patterns (or called wildcard pattern to search file: wiki-glob(programming)
   ii. it gives a warning if no file is found, like ⚠No files were found for the provided path. No artifacts will be uploaded.
   iii. the job won't fail, so it will not bring down the workflow (at the most time)
2. carefully read the warnings! and don't over-use the upload-artifact@v2 and Annotations (where gh actions will display the warnings at). After reading the warnings, please think twice about the following:
   i. what reason and what behavior caused the missing artifact file.
   ii. is this behavior known as normal (expected success)? or is it considered as a real error (expected error message)?
   iii. do you wish to skip this empty upload by if property?
3. write your conditions in the if part to correct a better and reduced warning logs:
   i. in the README part, it only says if: failure() link
   ii. but actually it does support more context and expression defined by two official documents
   iii. Context and expression syntax for GitHub Actions
   iv. workflow syntax: jobs..steps.if

I did the above three steps, to fix my own workflow, as (Permalink here):

    - name: Upload .dmg file (if any exists)
      if: ${{ runner.os == 'macOS' }}
      uses: actions/upload-artifact@v2
      with:
        name: ${{ matrix.os }}.dmg
        path: build/*.dmg

So hope this long comment will help you! Enjoy!

[Niceplace]

Closing this issue as the behavior is now implemented. Thanks folks !