Azure-pipelines-tasks: System.IO.FileLoadException: Unable to load symbolic/hard linked file

[ivailop]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: CacheBeta

Environment

• Server - Azure Pipelines

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number: https://s2technologies.visualstudio.com/testspace/_build/results?buildId=1104

• Agent - Hosted or Private:

  • If using Hosted agent, provide agent queue name: Ubuntu 16.04

Issue Description

Task defined as:

- task: CacheBeta@0
  inputs:
    key: bundle | $(Agent.OS) | Gemfile.lock
    path: $(BUNDLE_PATH)
  displayName: Cache bundle gems

Fails during "Post-job: Cache bundle gems" with System.IO.FileLoadException: Unable to load symbolic/hard linked file

Task logs

##[section]Starting: Cache bundle gems
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 0.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key `bundle | Linux | Gemfile.lock`...
File hashes summarized as `hTIqJ3fX2+6SXJd6UpzXFnYQqQATflNFows4Jkh5BZc=` from BASE64(SHA256(`
SHA256(s/Gemfile.lock)=[15008]0DBAA318817D73F39C93650B7E988CD82747B05B005D252DC1B3B5BD74200106`))
Resolved to `bundle|Linux|hTIqJ3fX2+6SXJd6UpzXFnYQqQATflNFows4Jkh5BZc=`.
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `bundle|Linux|hTIqJ3fX2+6SXJd6UpzXFnYQqQATflNFows4Jkh5BZc=`
Information, There is a cache miss.
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session 38e62829-bc8b-4752-83db-c700fa61638a
Information, Processing .artifactignore file surfaced 58 files. Total files under source directory: 18911
Information, 18853 files to be processed in 190 groups.
Error, Failed to calculate hash for the file's reparse point: /home/vsts/work/1/s/vendor/bundle/ruby/2.6.0/gems/public_activity-1.6.3/test/migrations/001_create_activities.rb
Error, Failed to hash a page of files. Exception listed below...
Error, System.IO.FileLoadException: Unable to load symbolic/hard linked file.  Check 'fsutil behavior query SymlinkEvaluation' to ensure proper behavior. ---> System.IO.FileNotFoundException: Could not find file '/home/vsts/work/1/s/vendor/bundle/ruby/2.6.0/gems/public_activity-1.6.3/test/migrations/001_create_activities.rb'.
File name: '/home/vsts/work/1/s/vendor/bundle/ruby/2.6.0/gems/public_activity-1.6.3/test/migrations/001_create_activities.rb'
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at Microsoft.VisualStudio.Services.Content.Common.FileStreamUtils.OpenFileStreamForAsync(String filePath, FileMode mode, FileAccess fileAccess, FileShare fileShare, FileOptions extraOptions)
   at Microsoft.VisualStudio.Services.BlobStore.Common.ChunkerHelper.CreateFromFileAsync(IFileSystem fileSystem, String path, CancellationToken cancellationToken, Boolean configureAwait)
   at Microsoft.VisualStudio.Services.BlobStore.Common.FileBlobDescriptor.CalculateAsync(IFileSystem fileSystem, String rootDirectory, Boolean chunkDedup, String relativePath, FileBlobType fileBlobType, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.GetFileBlobDescriptorAsync(String rootDirectory, Boolean chunkDedup, PageItem pageItem, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.GetFileBlobDescriptorAsync(String rootDirectory, Boolean chunkDedup, PageItem pageItem, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.<>c__DisplayClass25_0.<<PaginateAndProcessFilesHelperAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.Content.Common.NonSwallowingActionBlock.<>c__DisplayClass5_0`1.<<CreateNonSwallowingFunc>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.PaginateAndProcessFilesHelperAsync(String sourceDirectory, Boolean chunkDedup, IEnumerable`1 pathsInDir, Action`1 hashCompleteCallback, Int32 currentPage, Int32 totalPages, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.ArtifactServices.App.Shared.PrecomputedHashesGenerator.<>c__DisplayClass22_0.<<PaginateAndProcessFiles>b__1>d.MoveNext()
Information, 100 out of 18853 files processed (Group: 128/190)
Information, 200 out of 18853 files processed (Group: 126/190)
...

[johnterickson]

@ivailop Thanks for the report! What is at the path /home/vsts/work/1/s/vendor/bundle/ruby/2.6.0/gems/public_activity-1.6.3/test/migrations/001_create_activities.rb that it fails on? Is it a symlink to something that doesn't exist?

[ivailop]

It is a symbolic link - see source at https://github.com/chaps-io/public_activity/tree/1-6-stable/test/migrations

[johnterickson]

Ok agent 160 is rolling out with TARing and that should better handle symbolic links - keep an eye out for it! We'll keep this open until it's resolved.

[fadnavistanmay]

Hi @ivailop - this should be resolved in the agent version 2.160.0 . Could you please try that and let us know how it goes.

Thanks.

[ivailop]

It appears to be working for my use case.

[fadnavistanmay]

Glad to hear it's working. :)

[rmetzger]

Is the TAR support limited to the cache? it would be nice if PublishPipelineArtifact would support that as well.