Cache: feature request: local cache store for self-hosted runners

[jonathanong]

because i, as well as many others, are having issues with the caching service, a nice feature for self-hosted runners would be to have a cache local to the self-hosted runner, perhaps configurable by the self-hosted runner.

currently for me, it's faster to not cache and just run npm ci than to load/save cache on self-hosted runners.

[simophin]

This will be very useful! We don't have the best connection to the backend store, it is usually faster to just download all the things again.

[joshmgross]

👋 Hey @jonathanong, could you describe your feature request a bit more?

Most tool ecosystems already have a local cache, this action's purpose is to save that local cache in a central location to share between runners. For a self-hosted runner, running workflows will naturally populate those local caches.

[patrickhulce]

I am also interested in this feature and can explain my use case @joshmgross.

I use the cache action not just for node_modules or toolchain installed deps but for expensive compilation tasks internal to our app as well. It'd be nice to not have to build out a bespoke cache system to run on the self-hosted runner and instead leverage the existing caching logic of this action to store it locally instead of having to redownload these large binaries over and over.

example action steps we use

     - name: Get Image CLI version
        id: image-cli-version
        run: node ./log-image-cli-version.js
      - name: Cache Image CLI
        id: cache-image-cli
        uses: actions/[email protected]
        with:
          path: built-dependencies/image-cli
          key: ${{ runner.os }}-image-cli-${{ steps.image-cli-version.outputs.version }}
      - run: bash ./compile-expensive-image-cli.sh
         if: steps.cache-image-cli.outputs.cache-hit != 'true'

[jonathanong]

It'd be nice to not have to build out a bespoke cache system to run on the self-hosted runner

I think that's the main point. I could set up my own caching system, but then it would be a custom action. Ideally, it's the same actions API, but with a different "backend".

Another feature I'd like is to avoid slow cache uploads/downloads, which are much slower if your self-hosted runner doesn't have great internet speeds, especially upload speeds.

[jonathanong]

this action's purpose is to save that local cache in a central location to share between runners.

This local cache can be on the local self-hosted runner and could be shared by other self-hosted runners on the same machine.

[joshmgross]

Thank you all for your feedback. At this time there are no plans to make a local caching service, as that functionality exists in some form with each tool's cache. The logic around matching cache keys and branch scopes is all handled by the internal cache service and would have to be duplicated into the action to support any local cache functionality.

[joshuapinter]

I would love to see this as well, primarily for node_modules via yarn. We're using a farm of self-hosted runners and using actions/cache is the same, if not slower, than just installing all the dependencies again from a blank slate, depending on the utilization of the network, etc.

We have to use our own tasks/steps to essentially do what actions/cache does but with a local directory. Just would be nice to not have to re-invent the wheel here and, instead, specify a cache "location" or "path" to use instead of using the remote cache on GitHub/Azure.

Here's a quick benchmark to show you the performance hit self-hosted runners have with using the default remote cache:

Remote Cache Using github/actions.

2m48s

And that's WITH a cache hit.

Local Cache.

Cache Miss

1m42s

Cache Hit

~0m01s

Anyway, just would be nice to not have to do this manually and get all the nice features of actions/cache for self-hosted runners.