Azure-pipelines-tasks: CacheBeta@1 caching post-job fails for Carthage with: Error, System.IO.IOException: Too many levels of symbolic links

[DimMad]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: CacheBeta@1

Environment

• Server - Azure Pipelines

  • account name: lunaluna-mti
  • team project name: LunaLuna
  • build definition name/build number: 5073

• Agent - Hosted:

  • macOS-10.14

Issue Description

The following task fails as described in the title of the issue:

- task: CacheBeta@1
  displayName: 'Cache Check/Restore for Carthage'
  inputs:
    key: 'carthage | $(Agent.OS) | Cartfile.resolved'
    path: 'Carthage'
    cacheHitVar: 'CARTHAGE_CACHED'

Task logs

Starting: Cache Check/Restore for Carthage
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 1.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key: carthage|Darwin|Cartfile.resolved
 - carthage          [string]
 - Darwin            [string]
 - Cartfile.resolved [file] --> A699F644FB9F370EC4A51F968F5634D38DFDB196E361385AA962EEA9EB762F31
Resolved to: carthage|Darwin|VOMCLbzc4Ip1wD1uHIdOFknBmFtovmJbAaYg5eDX7NM=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 0e630b80-9d36-4252-9694-5e96d74272b8
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `carthage|Darwin|VOMCLbzc4Ip1wD1uHIdOFknBmFtovmJbAaYg5eDX7NM=`
Information, There is a cache miss.
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session 0e630b80-9d36-4252-9694-5e96d74272b8
Information, 5002 files to be processed in 51 groups.
Information, 2 out of 5002 files processed (Group: 51/51)
Error, Failed to hash a page of files. Exception listed below...
Error, System.IO.IOException: Too many levels of symbolic links
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at Microsoft.VisualStudio.Services.Content.Common.FileStreamUtils.OpenFileStreamForAsync(String filePath, FileMode mode, FileAccess fileAccess, FileShare fileShare, FileOptions extraOptions)
   at Microsoft.VisualStudio.Services.BlobStore.Common.ChunkerHelper.CreateFromFileAsync(IFileSystem fileSystem, String path, CancellationToken cancellationToken, Boolean configureAwait)
   at Microsoft.VisualStudio.Services.BlobStore.Common.FileBlobDescriptor.CalculateAsync(IFileSystem fileSystem, String rootDirectory, Boolean chunkDedup, String relativePath, FileBlobType fileBlobType, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.GetFileBlobDescriptorAsync(String rootDirectory, Boolean chunkDedup, PageItem pageItem, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.<>c__DisplayClass25_0.<<PaginateAndProcessFilesHelperAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.PaginateAndProcessFilesHelperAsync(String sourceDirectory, Boolean chunkDedup, IEnumerable`1 pathsInDir, Action`1 hashCompleteCallback, Int32 currentPage, Int32 totalPages, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.<>c__DisplayClass22_0.<<PaginateAndProcessFiles>b__1>d.MoveNext()
Information, 102 out of 5002 files processed (Group: 48/51)
Information, 202 out of 5002 files processed (Group: 6/51)
Information, 302 out of 5002 files processed (Group: 2/51)
Information, 402 out of 5002 files processed (Group: 7/51)
Information, 502 out of 5002 files processed (Group: 4/51)
Information, 602 out of 5002 files processed (Group: 10/51)
Information, 702 out of 5002 files processed (Group: 3/51)
Information, 802 out of 5002 files processed (Group: 13/51)
Information, 902 out of 5002 files processed (Group: 9/51)
Information, 1002 out of 5002 files processed (Group: 5/51)
Information, 1102 out of 5002 files processed (Group: 14/51)
Information, 1202 out of 5002 files processed (Group: 17/51)
Information, 1302 out of 5002 files processed (Group: 8/51)
Information, 1402 out of 5002 files processed (Group: 11/51)
Information, 1502 out of 5002 files processed (Group: 18/51)
Information, 1602 out of 5002 files processed (Group: 1/51)
Information, 1702 out of 5002 files processed (Group: 21/51)
Information, 1802 out of 5002 files processed (Group: 12/51)
Information, 1902 out of 5002 files processed (Group: 15/51)
Information, 2002 out of 5002 files processed (Group: 22/51)
Information, 2102 out of 5002 files processed (Group: 20/51)
Information, 2202 out of 5002 files processed (Group: 25/51)
Information, 2302 out of 5002 files processed (Group: 23/51)
Information, 2402 out of 5002 files processed (Group: 16/51)
Information, 2502 out of 5002 files processed (Group: 19/51)
Information, 2602 out of 5002 files processed (Group: 27/51)
Information, 2702 out of 5002 files processed (Group: 29/51)
Information, 2802 out of 5002 files processed (Group: 30/51)
Information, 2902 out of 5002 files processed (Group: 26/51)
Information, 3002 out of 5002 files processed (Group: 32/51)
Information, 3102 out of 5002 files processed (Group: 33/51)
Information, 3202 out of 5002 files processed (Group: 34/51)
Information, 3302 out of 5002 files processed (Group: 24/51)
Information, 3402 out of 5002 files processed (Group: 37/51)
Information, 3502 out of 5002 files processed (Group: 35/51)
Information, 3602 out of 5002 files processed (Group: 39/51)
Information, 3702 out of 5002 files processed (Group: 38/51)
Information, 3802 out of 5002 files processed (Group: 28/51)
Information, 3902 out of 5002 files processed (Group: 43/51)
Information, 4002 out of 5002 files processed (Group: 41/51)
Information, 4102 out of 5002 files processed (Group: 42/51)
Information, 4202 out of 5002 files processed (Group: 31/51)
Information, 4302 out of 5002 files processed (Group: 45/51)
Information, 4402 out of 5002 files processed (Group: 46/51)
Information, 4502 out of 5002 files processed (Group: 49/51)
Information, 4602 out of 5002 files processed (Group: 44/51)
Information, 4702 out of 5002 files processed (Group: 47/51)
Information, 4802 out of 5002 files processed (Group: 40/51)
Information, 4902 out of 5002 files processed (Group: 36/51)
Information, ApplicationInsightsTelemetrySender correlated 3 events with X-TFS-Session 0e630b80-9d36-4252-9694-5e96d74272b8
##[error]Too many levels of symbolic links
Finishing: Cache Check/Restore for Carthage

[johnterickson]

@fadnavistanmay - Can you share the opt-in env var for TARing?
@DimMad - Once @fadnavistanmay shares, can you try it out?

[fadnavistanmay]

@DimMad - can you try AZP_CACHING_TAR : true and see if it helps.

[DimMad]

Thank you both. Will do and edit here.
Just to make sure. What I am going to try is to set that as a variable under the inputs: tag. Is that correct?

Edit 1: In case my usage assumption above was correct then it did not work. However, as I am getting Unexpected property AZP_CACHING_TAR I am probably mistaken. I am very new in the pipelines, so could you please let me know how to set an "opt-in env var for TARing"? Tried looking for it but couldn't find anything related.

Edit 2: Combining the information I found in:

• YAML Schema -> Task
• Variables -> Secret

My task was formatted as follows for 3 tests (just in case syntax matters):

- task: CacheBeta@1
  env: { AZP_CACHING_TAR: 'true' } 
  # env: { AZP_CACHING_TAR: true } the second time
  # env: # the third time
  #   AZP_CACHING_TAR: true 
  displayName: 'Cache Check/Restore for Carthage'
  inputs:
    key: 'carthage | $(Agent.OS) | Cartfile.resolved'
    path: 'Carthage'
    cacheHitVar: 'CARTHAGE_CACHED'

All of the above resulted in the same failure with the same log.
In case that this was wrong too, once again, please point me in the right direction.

[johnterickson]

Set these two variables at the job-level. This is just to opt-in to the new functionality before it is made the default.

job:
  - variables:
     AZP_CACHING_TAR: 'true'
     AZP_CACHING_CONTENT_FORMAT: 'SingleTar'
  - steps:
     - task: CacheBeta@1
...

[DimMad]

@johnterickson
Thank you very much for the pointer.

Got the chance to run the cache related steps only today and had a success.

Probably not related but I will run the whole pipeline tomorrow to make sure restoration is also possible, and edit here so we can close the issue.

Edit: Restoration was also successful. Thank you very much for the support. I think this can be closed.

[johnterickson]

Glad to hear @DimMad . Do note that we're rolling out TARing as default with agent 160. If this is what you want, you can just remove this env var. If you want to specifically not use TARing, then set AZP_CACHING_CONTENT_FORMAT to Files. We'll be documenting this new environment variable.