Upload-artifact: Exclude files/dirs either by name or regexp

[tysonite]

It would be nice to have a way to exclude some files/directories from uploading. I would think about something like this:

steps:
- uses: actions/checkout@v1

- run: mkdir -p path/to/artifact

- run: echo hello > path/to/artifact/hello.txt
- run: echo hello > path/to/artifact/world.txt

- uses: actions/upload-artifact@v1
  with:
    name: my-artifact
    path: path/to/artifact
    exclude: .*rld.*

So only hello.txt will be archived/uploaded.

Might be ANT syntax is good.

[olegburov]

Or filter like:

steps:
- uses: actions/checkout@v1

- run: mkdir -p path/to/artifact

- run: echo hello > path/to/artifact/A/hello.txt
- run: echo hello > path/to/artifact/A/B/world.txt
- run: echo hello > path/to/artifact/C/hello-world.txt

- uses: actions/upload-artifact@v1
  with:
    name: my-artifact
    path: path/to/artifact/A/**/*

So only /A/hello.txt and /A/B/world.txt will be archived/uploaded.

[solodon4]

Why not just accept your regular wildcards for paths: one should be able to list multiple wildcards, all files matching them will be included. When the wildcard is negative (starting from !), files matching it will be excluded.

[eine]

@solodon4, I included your proposal to use !, in https://github.com/actions/upload-artifact/issues/3#issuecomment-524442814

[solodon4]

Thank you @eine ! My point was also to have it as unified as possible with other places you guys use path patterns, e.g. path triggers where you can have an array of path patterns, both positive and negative. This will ensure the cognitive burden of learning where what is accepted will be minimized.

[JesseTG]

I agree, this would be nice.

[eng-dibo]

+1

[EwoutH]

Being able to include and exclude certain files using and * (wildcards) and ** (recursive wildcards) would be extremely useful. AppVeyor supports this.

It's also very useful to search for files like with a certain name, especially if you don't know exactly in which directory they ended up in.

Here's an example of wildcard use in Netflix: https://github.com/Netflix/vmaf/blob/master/appveyor.yml#L59

[konradpabjan]

With the v2-preview, we have added support for wildcards so you can do basic filtering using **/*. You can check that out here: https://github.com/actions/upload-artifact/issues/62

The @actions/glob package that is being used internally for v2-preview does support exclude patterns out of the box: https://github.com/actions/toolkit/tree/master/packages/glob#exclude-patterns

To fully take advantage of this though, we will have to wait until support for multiple paths gets added since even with the v2-preview we only accept a single path: https://github.com/actions/upload-artifact/issues/55

Afterwards it should be possible to do something like this (not 100% sure about the final YAML syntax for multiple paths) :

- uses: actions/upload-artifact@v2
  with:
    name: my-artifact
    paths: 'path/to/artifact/A/**/*', '!path/to/Artifact/A/temp/**/*', ''!path/to/Artifact/A/**/*.rdl'

[ax4]

With the v2-preview, we have added support for wildcards so you can do basic filtering using **/*. You can check that out here: #62

The @actions/glob package that is being used internally for v2-preview does support exclude patterns out of the box: https://github.com/actions/toolkit/tree/master/packages/glob#exclude-patterns

To fully take advantage of this though, we will have to wait until support for multiple paths gets added since even with the v2-preview we only accept a single path: #55

Afterwards it should be possible to do something like this (not 100% sure about the final YAML syntax for multiple paths) :

- uses: actions/upload-artifact@v2
  with:
    name: my-artifact
    paths: 'path/to/artifact/A/**/*', '!path/to/Artifact/A/temp/**/*', ''!path/to/Artifact/A/**/*.rdl'

Hi @konradpabjan ! I'm not sure how the final YAML syntax has been so far for multiple glob patterns in upload-artifact v2-preview and upload-artifact-v2, but I managed to do the following with the current glob pattern calling:

- uses: actions/upload-artifact@v2
  with:
    name: my-artifact
    paths: "path/to/artifact/A/**/*\n!path/to/Artifact/A/temp/**/*\n!path/to/Artifact/A/**/*.rdl"

Note that, the glob module does support multiple glob patterns by join-ing patterns with \n in their source and README. And in the YAML file, the way to pass line escaper \n is to use double-quote to warp the text (Reference to strings-in-yaml: to quote or not to quote).

Also, some links to my test if anyone needs an idea: my testing workflow file, testing commit, and runner result.

Comment updated on July 10th:

Thank @konradpabjan again for creating the PR#94 13 hours ago, and he has already merged it 5 hours ago. Congrats! Now as he suggests in a new version of README. Use | in yml/yaml file to pass the exclude pattern. See example at: https://github.com/actions/upload-artifact/blame/master/README.md#L59-L69

Thank you so much for the fix, PR, and documenting in README!