Php-cs-fixer: GithubAction: use php-cs-fixer cache

[staabm]

hey guys,

when using the latest php-cs-fixer on a github-action build, I want to utilize caching.

that should be similar to composer dependencies caching which can be achieved with

    - name: Get composer cache directory
      id: composer-cache
      run: echo "::set-output name=dir::$(composer config cache-files-dir)"

    - name: Cache dependencies
      uses: actions/cache@v1
      with:
        path: ${{ steps.composer-cache.outputs.dir }}
        key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
        restore-keys: ${{ runner.os }}-composer-

see https://github.com/actions/cache/blob/master/examples.md

will it work, if I add github-action caching for the .php_cs.cache file, or might the cs-fixer get confused by the file beeing present? which variables should be used for the cache-key?

thanks for your ideas/input in advance

[staabm]

we might have to work arround some limitations, because it is documented that you cannot cache a single file

https://help.github.com/en/actions/configuring-and-managing-workflows/caching-dependencies-to-speed-up-workflows#input-parameters-for-the-cache-action

[julienfalque]

Using PHP CS Fixer's cache feature requires accurate files modification times which is rarely the case on CI systems because they create a new copy of the source for each job, having modification times always greater than what was stored in .php_cs.cache. Not sure about Github Action though as I never played with it yet.

An alternative solution is to explicitly pass the files that changed as arguments to PHP CS Fixer. Also maybe we can make the cache system smarter by e.g. comparing file checksums in addition to modification times?

[staabm]

Using PHP CS Fixer's cache feature requires accurate files modification times which is rarely the case on CI systems because they create a new copy of the source for each job, having modification times always greater than what was stored in .php_cs.cache.

ahh this might be the reason why phpstan uses a md5 based hashing instead of file-modification times for its cache feature. thanks for the insights.

phpstan seems to use 1 cache file per 1 source file.. that way it can re-use the cache across different builds (even the builds of different branches can re-use stuff from e.g. builds running on the master branch). it doesnt use a single cache file for everything and re-builds it entirely when only a few files change

[staabm]

maybe we can make the cache system smarter by e.g. comparing file checksums in addition to modification times?

I did some investigations and as far as I can tell, the cache is already based on checksums, not on modification times:

https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/master/src/Cache/FileCacheManager.php

[staabm]

I did some tests and came to the following examples (which works \o/):

https://github.com/clxmstaab/PHP-CS-Fixer/blob/ed202482e4c0e5691bf8453c5cfc20a13f84349a/.github/workflows/caching.yml#L30-L60

important points:

• the actions/cache@1 GitHubAction only works with directories
• we need to define a separate directory, e.g. within /tmp/ create it and put the cache file in there.
• it does not work to put the cache file directly into /tmp and cache/restore the /tmp folder

example action run, which shows it works:
https://github.com/clxmstaab/PHP-CS-Fixer/runs/653574779?check_suite_focus=true

compare this run to the following, which takes 20+ secs on the very same codebase, just without caching:
https://github.com/clxmstaab/PHP-CS-Fixer/runs/653574764?check_suite_focus=true

at this point, I would consider the remaing steps are only documentation how to do it.

[bendavies]

we are doing this.
you can either key on the commit.sha so you get an additive cache

$config->setCacheFile(__DIR__.'/.build/php-cs-fixer/.php_cs.cache');

      - name: "Cache cache directory for friendsofphp/php-cs-fixer"
        uses: actions/cache@v1
        with:
          path: .build/php-cs-fixer
          key: php-${{ matrix.php-version }}-php-cs-fixer-${{ github.sha }}
          restore-keys: php-${{ matrix.php-version }}-php-cs-fixer-

      - name: "Run friendsofphp/php-cs-fixer"
        run: vendor/bin/php-cs-fixer fix --diff --diff-format=udiff --dry-run --verbose

or if you don't like that, you could create your own key based on a manually calculated cache key with something like

find src tests -type f -name "*.php" -exec md5sum {} + > php-cs-fixer.checksum

      - name: "Cache cache directory for friendsofphp/php-cs-fixer"
        uses: actions/cache@v1
        with:
          path: .build/php-cs-fixer
          key: php-${{ matrix.php-version }}-php-cs-fixer-${{ hashFiles('php-cs-fixer.checksum') }}
          restore-keys: php-${{ matrix.php-version }}-php-cs-fixer-

[staabm]

@bendavies thx for your suggestions.
Both do not make sense to me though.

you can either key on the commit.sha so you get an additive cache

This would only work when the same commit would be built over and over again. I dont think this is a common use-case in CI

or if you don't like that, you could create your own key based on a manually calculated cache key with something like

This does not make sense to me either. The builtin caching mechanics of php-cs-fixer already keep track of this conditions, see https://github.com/FriendsOfPHP/PHP-CS-Fixer/blob/f385e1b93988e72445a7585a1acceacea2b39d2e/src/Cache/FileCacheManager.php#L20-L25

Maybe I am missing something?

[bendavies]

hi @staabm .

First, you need to understand that there is a peculiarity with actions/cache: it will not update a cache item if the key does not change. This can be clearly seen in your workflow run here:
https://github.com/clxmstaab/PHP-CS-Fixer/runs/653574779?check_suite_focus=true

Hopefully you now see why your workflow does not work. your cache will never update, even if php files are changed, since your cache key is hard coded to ${{ runner.os }}-fixer-2. So even if you update php files, you'll be left with a stale cache, which will get more and more stale over time.

Now:

Maybe I am missing something?

Yes, you are missing the usage of the fall back to restore-keys.

Taking this example,

      - name: "Cache cache directory for friendsofphp/php-cs-fixer"
        uses: actions/cache@v1
        with:
          path: .build/php-cs-fixer
          key: php-${{ matrix.php-version }}-php-cs-fixer-${{ github.sha }}
          restore-keys: php-${{ matrix.php-version }}-php-cs-fixer-

The following will happen.

1. dev commits commit aaaaaaaa
2. Cache misses for key php-7.4-php-cs-fixer-aaaaaaaa
3. Cache misses for keyphp-7.4-php-cs-fixer
4. php-cs-fixer runs.
5. Cache is saved with key php-7.4-php-cs-fixer-aaaaaaaa
6. Workflow Ends.
7. dev commits commit bbbbbbbb
8. Workflow runs:
9. Cache misses for key php-7.4-php-cs-fixer-bbbbbbbb
10. Cache HITS for keyphp-7.4-php-cs-fixer- (which is actually php-7.4-php-cs-fixer-aaaaaaaa saved in the previous run)
11. php-cs-fixer runs with a fully primed cache, very fast, only fixing updating changed php files in commit bbbbbbbb
12. Cache is saved with key php-7.4-php-cs-fixer-bbbbbbbb
13. Workflow Ends.
14. dev commits commit ccccccc
15. Workflow runs:
16. etc...

There are other semantics of the restore-key fallbacks related to branches which you can read here.

The upshot that if you want a cache to be updated, you need to use a changing cache key. ${{ github.sha }} achieves that nicely.
However, you only need to do this when you don't have some asset to hash on, e.g. we use ${{ hashFiles('composer.lock') }} for composer dep caching.

I hope that helps.

[julienfalque]

the cache is already based on checksums

Damn, not sure why I thought it was based on timestamps only 😆️

[localheinz]

@staabm

This works well for me:

• https://github.com/ergebnis/php-library-template/blob/d97e45ff409fb7e591dda96cb2cd771e901552d2/.php_cs#L41

  $config->setCacheFile(__DIR__ . '/.build/php-cs-fixer/.php_cs.cache');
  

• https://github.com/ergebnis/php-library-template/blob/d97e45ff409fb7e591dda96cb2cd771e901552d2/.github/workflows/integrate.yaml#L77-L88

        - name: "Create cache directory for friendsofphp/php-cs-fixer"
          run: "mkdir -p .build/php-cs-fixer"
  
        - name: "Cache cache directory for friendsofphp/php-cs-fixer"
          uses: "actions/[email protected]"
          with:
            path: ".build/php-cs-fixer"
            key: "php-${{ matrix.php-version }}-php-cs-fixer-${{ hashFiles('composer.lock') }}"
            restore-keys: "php-${{ matrix.php-version }}-php-cs-fixer-"
  
        - name: "Run friendsofphp/php-cs-fixer"
          run: "vendor/bin/php-cs-fixer fix --config=.php_cs --diff --diff-format=udiff --dry-run --verbose"
  

[SpacePossum]

Hi!
Thanks for the question @staabm and everyone here for sharing all this information on this topic.
I'm going to close the issue now as I think the question got answered, let me know if not and we can always reopen.