Cache: Post actions/cache@v1 takes 4+ hours to complete

[jagregory]

I was directed here by support. I suspect this is related to #95.

I have a Node repository using Yarn, and when the Post cache action runs it takes around 5 hours to complete, and when it does complete it doesn't seem to actually successfully cache anything (so subsequent builds continue to take 5 hours).

Sat, 29 Feb 2020 07:01:21 GMT
##[debug]..Evaluating hashFiles:
Sat, 29 Feb 2020 07:01:21 GMT
##[debug]....Evaluating String:
Sat, 29 Feb 2020 07:01:21 GMT
##[debug]....=> 'yarn.lock'
Sat, 29 Feb 2020 07:01:21 GMT
##[debug]Search root directory: '/home/runner/work/proj/proj'
Sat, 29 Feb 2020 07:26:52 GMT
##[debug]Search pattern: '/home/runner/work/proj/proj/yarn.lock'
Sat, 29 Feb 2020 11:46:48 GMT
##[debug]Found 4942714 files
Sat, 29 Feb 2020 11:46:48 GMT
##[debug]1 matches to hash

In the debug logs I can see it's likely related to the hashFiles call. I don't know why it's searching though when my hashFiles is set to just a single file: key: ${{ runner.os }}-yarn-${{ hashFiles('yarn.lock') }}

[dhadka]

Hi @jagregory , thanks for reporting this and sorry for the inconvenience it's causing. Right now, it's enumerating all the files and checking for pattern matches. We already have a fix to switch to globbing, which should only scan the directories/files that would match the pattern.

We are starting to roll out this fix, but it may take a few days so we can validate the fix and make sure there are no regressions. I'll update this issue with the status.

[dhadka]

@jagregory I just received word that the new globbing feature is rolled out to all runners. Please let us know if this resolves the issue for you. I'll close this issue in a few days if we don't hear back.

[thegnuu]

@dhadka I just found this issue because we have the same problem. We are building an electron app, on linux-latest and windows-latest it works perfectly. Unfortunately on macos-latest it is not working which is causing us an unnecessary usage of a lot of minutes because the macos minutes are multiplied by 10!

For now I just cancel the job as soon as it reaches the Post Yarn cache step, but I think this should not be the idea of an action ;)

[dhadka]

@thegnuu Just to confirm, in the Post Yarn cache step, you see the logs indicating it's scanning millions of files? Has this improved in all recently (starting around 3/25) due to the rollout of globbing?

Please try enabling debug logging and include the logs for the Post Yarn cache step here. Thank you!

[thegnuu]

@dhadka would love to provide you additional information, unfortunately I am currently not able to run yarn on my macos environment because of network errors while installing rxjs. As soon as it works again I will let you know.

I am just switching from Azure to GitHub and just created the action yesterday, so I am not able to tell you how this was before.

[thegnuu]

@dhadka unfortunately I am still not able to run my action on macos.

error An unexpected error occurred: "https://registry.yarnpkg.com/rxjs/-/rxjs-6.5.3.tgz: ESOCKETTIMEDOUT".

I don't know why this happens, the issue just occurs on macos. I will test it again by the end of this week, maybe the issue will be resolved since it looks like some sort of network issue.

Will let you know as soon as I am able to fetch some logs.

[dhadka]

@thegnuu Any luck getting unblocked and testing the cache fix? If I don't hear back in a few days, I'll close this issue, but please feel free to re-open or create a new one if you're still seeing this issue.

[jagregory]

OP here, just realised I didn't provide an update on this: everything is working as expected now. My pipelines have been running for a week or so since the fix was put in place with no issues. I'm personally happy to consider this one fixed.