Cache: hashFiles() does not work for valid patterns

[poiru]

The expected behavior is that all three of the following steps pass on all platforms:

- name: Cache **/README.md
  uses: actions/cache@preview
  with:
    path: .
    key: test-${{ runner.os }}-${{ hashFiles('**/README.md') }}
  continue-on-error: true

- name: Cache README.md
  uses: actions/cache@preview
  with:
    path: .
    key: test-${{ runner.os }}-${{ hashFiles('README.md') }}
  continue-on-error: true

- name: Cache *README.md
  uses: actions/cache@preview
  with:
    path: .
    key: test-${{ runner.os }}-${{ hashFiles('*README.md') }}
  continue-on-error: true

But right now hashFiles(README.md) and hashFiles(*README.md) fails macOS:

2019-11-02T18:08:19.8057240Z ##[error]The template is not valid. 'hashFiles(README.md)' failed. Search pattern 'README.md' doesn't match any file under '/Users/runner/runners/2.160.0/work/github-actions-hashfiles-test/github-actions-hashfiles-test'
2019-11-02T18:08:19.8346580Z ##[error]The template is not valid. 'hashFiles(*README.md)' failed. Search pattern '*README.md' doesn't match any file under '/Users/runner/runners/2.160.0/work/github-actions-hashfiles-test/github-actions-hashfiles-test'

And hashFiles(README.md) and hashFiles(**/README.md) fails Windows:

2019-11-02T18:09:01.2005576Z ##[error]The template is not valid. 'hashFiles(**/README.md)' failed. Search pattern '**/README.md' doesn't match any file under 'd:\a\github-actions-hashfiles-test\github-actions-hashfiles-test'
2019-11-02T18:09:01.2212553Z ##[error]The template is not valid. 'hashFiles(README.md)' failed. Search pattern 'README.md' doesn't match any file under 'd:\a\github-actions-hashfiles-test\github-actions-hashfiles-test'

See example repo: https://github.com/poiru/github-actions-hashfiles-test
And run: https://github.com/poiru/github-actions-hashfiles-test/commit/23dcdc7705d1660ba845291f744dd9b4a9157458/checks?check_suite_id=293065410

[willnorris]

I've discovered that they need OS-specific path separators. For Windows, try hashFiles(**\README.md).

[willnorris]

I ended up setting up a completely separate job for Windows in my repo for other reasons (https://github.com/willnorris/imageproxy/pull/202/files).

But having separate caching steps that checked the OS worked as well. Of course, this is almost certainly a bug that should be fixed upstream, but this will work in the meantime:

steps:

 - uses: actions/cache@preview
   if: runner.os != 'Windows'
   with:
      key: ${{ runner.os }}-${{ hashFiles('**/README.md') }}

 - uses: actions/cache@preview
   if: runner.os == 'Windows'
   with:
      key: ${{ runner.os }}-${{ hashFiles('**\README.md') }}

[shivammathur]

If path separator is exposed in the runner context, then it will be easy to setup on all OS with the same interface.

For example

- name: Cache *README.md
  uses: actions/cache@preview
  with:
    path: .
    key: test-${{ runner.os }}-${{ hashFiles(format('{0}{1}{2}', github.workspace, runner.separator, 'README.md')) }}

[joshmgross]

hashFiles will be updated soon so that relative paths are rooted at github.workspace, which will allow hashFiles('README.md').

/ not working on Windows is definitely a bug, ideally / would work on all platforms so that you wouldn't need a runner.separator variable.

[filipesilva]

Also seeing this on https://github.com/filipesilva/create-cljs-app. I have to setup a dual cache setup as well:

    - name: Cache yarn on unix
      if: matrix.os != 'windows-latest'
      uses: actions/cache@v1
      with:
        path: ~/.yarn-cache
        key: unix-yarn-${{ hashFiles('**/yarn.lock') }}
        restore-keys: |
          unix-yarn-
    - name: Cache maven on unix
      uses: actions/cache@v1
      if: matrix.os != 'windows-latest'
      with:
        path: ~/.m2
        key: unix-m2-${{ hashFiles('**/shadow-cljs.edn') }}
        restore-keys: |
          unix-m2-
    - name: Cache yarn on windows
      if: matrix.os == 'windows-latest'
      uses: actions/cache@v1
      with:
        path: ~/.yarn-cache
        key: windows-yarn-${{ hashFiles('**\yarn.lock') }}
        restore-keys: |
          windows-yarn-
    - name: Cache maven on windows
      uses: actions/cache@v1
      if: matrix.os == 'windows-latest'
      with:
        path: ~/.m2
        key: windows-m2-${{ hashFiles('**\shadow-cljs.edn') }}
        restore-keys: |
          windows-m2-

[stomar]

@joshmgross The current hashFiles documentation states: _"Returns a single hash for the set of files that matches the path pattern. The path is relative to the GITHUB_WORKSPACE directory [...]"_ (as of 2019-11-09).

However, hashFiles('Gemfile.lock') does _not_ work at the moment. Documentation error or do I misunderstand something?

[joshmgross]

@stomar Documentation is currently not correct. There's a few bugs with hashFiles right now, one of which is that it's not relative to the GITHUB_WORKSPACE. This issue is tracking that, and the fix should be rolling out soon.

[joshmgross]

hashFiles should now work with / on Windows and the path is now correctly set to be relative to GITHUB_WORKSPACE

[filipesilva]

Can confirm the two fixes @joshmgross mentioned work for me in https://github.com/filipesilva/create-cljs-app/pull/22. But I also noticed that hashfiles on windows is giving me a different hash than on linux/osx:

• windows https://github.com/filipesilva/create-cljs-app/pull/22/checks?check_run_id=299887723
• linux https://github.com/filipesilva/create-cljs-app/pull/22/checks?check_run_id=299887687

My global git config has autocrlf = input andeol = lf. Not sure if that would affect the remote though, since I don't have any local git config or anything.

[joshmgross]

@filipesilva I believe on windows the line endings will be replaced by CRLF in checkouts, the autocrlf config will just change how the line-endings will be committed to the repository.
See https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration and https://help.github.com/en/github/using-git/configuring-git-to-handle-line-endings#global-settings-for-line-endings

Do you need these hashes to be the same? It's recommended to use a different cache per OS depending on your ecosystem (node_modules can sometimes change or contain compiled dependencies for example)

[filipesilva]

I'm mostly fine if they are not the same. It's true that node_modules can have different artifacts via platform deps or postinstalls, but generally an install after cache restore will still do the right thing.

I just found it surprising that with the same settings, on GH Actions the caches were different.

[TivCh]

@joshmgross Does hashFiles have any limitation on binary files? I'm trying to hash a sqlite db and I'm getting an empty key.

[dhadka]

@TivCh hashFiles should work on binary files. I think the more likely reason for an empty hash is the pattern isn't matching any files. The logs should contain more information on what files are being processed by hashFiles if you enable step debug logging.

If you do see an issue with hashFiles, please file an issue with https://github.com/actions/runner/, which owns that function.

Edit: I suppose there could also be file locking or related issue going on. Would also be good to check for any such errors in the logs.