Salt: file.recurse exclude_pat improvements (accept a list)

[josmardias]

I tried to use exclude_pat but it seems not working for folders, only files.
I think it will be very useful if we can ignore a subtree by matching a folder.
And i believe that exclude_pat can be an array, so we can match more than one file by glob.

So...

/home/josmar/tree:
  file.recurse:
    - source: salt://test/tree
    - include_empty: True
    - clean: True
    # not sure if this regex is right, but my intention here is 
    # to match all files inside these folders.
    - exclude_pat: 'E@(^excluded/*)|(^holder/excluded/*)'
    - user: josmar
    - group: josmar
    - dir_mode: 550
    - file_mode: 440


/home/josmar/tree/holder/excluded/managed:
  file.managed:
    - user: josmar
    - group: josmar
    - require:
      - file: /home/josmar/tree

Can be improved like:

/home/josmar/tree:
  file.recurse:
    - source: salt://test/tree
    - include_empty: True
    - clean: True
    - exclude_pat:
      - excluded
      - holder/excluded
    - user: josmar
    - group: josmar
    - dir_mode: 550
    - file_mode: 440

And the whole "/home/josmar/tree/excluded" folder will be ignored by recurse. No copying and no cleaning.

[thatch45]

Thanks for the heads up, this should exclude directories

[Mrten]

@thatch45, can this milestone be moved to something in the more immediate future? Currently

- exclude_pat: ".svn"

does not work and all my recursively copied config directories have .svn directories littered in them (with potentially unnecessary sensitive information in there).

[basepi]

This seems like a reasonable request. I have no idea about the complexity of its implementation, but I'm going to move it to the current milestone, see if we can get someone to take it up.

[basepi]

@Mrten it's using globbing on the entire relative path. So you need to do - exclude_pat: "*.svn*" and it works.

However, I did discover that include_empty is completely ignoring include and exclude patterns, so I will still be submitting a fix.

[Mrten]

I was writing a rebuttal for the _.svn_ thing and then discovered I used include_empty ;) Thanks for catching and fixing!

Please don't close the bug yet though as it asks for a list of patterns for include_pat/exclude_pat

[Mrten]

Oh, and @basepi, please make a note in the docs for include/exclude_pat for the globbing, with the .svn example, I'm sure lots of folks will be happier for it :)

[Mrten]

Should we reopen a bug then for a list of exclude_pats?

[basepi]

Oh, right, I missed that part of the feature request. =)

[tkwilliams]

An additional "nice to have" (read: it bites me in the @ss all the time) would be to be able to match on the directory path, or portions thereof. E.g.:

${splunkdir}/splunk/etc:
  file.recurse:
    - source: salt://splunk/files/etc
    - template: mako
    - exclude_pat: E@system/local/(server|inputs)\.conf
    - clean: true

Simply excluding on E@(server|inputs).conf doesn't work for this, because there are a number of files named server.conf and/or inputs.conf at different locations in the tree (Splunk is super-annoying that way) and most of them SHOULD be managed.

[markuskramerIgitt]

@basepi as you gave the advise to use the exclude_pat: "*.svn*" in 2013, should it not work in Salt 2014.1.13 (Windows) and Salt 2014.7.0 (Debian)?

Unfortunately, this expression copies all .svn folders:

/localdir:
  file.recurse:
    - clean: True
    - source: salt://remotedir
    - exclude_pat: "*.svn*"

[basepi]

@markuskramerIgitt Sounds like a bug. Open a new issue, please?

[basepi]

@markuskramerIgitt Please include as much information as you can, including version information.

[corywright]

Any idea on when we might see file.recurse be able to accept a list?

I really want to be able to do this:

/home/build:
  file.recurse:
    - source: salt://buildbox/home/build
    - exclude_pat:
      - .ssh/authorized_keys
      - .gnupg/pubring.gpg
      - .gnupg/secring.gpg

Thanks.

[pgporada]

Being able to exclude files like this would be helpful here too. File.directory could also use this improvement.

[mchugh19]

I think this can be closed with the merged list support for include_pat/exclude_pat.