Msbuild: MSBuild disables hard linking when building in Visual Studio

[jaredpar]

Consider these two property groups inside of MS.Common.CurrentVersion.targets:

• https://github.com/Microsoft/msbuild/blob/90506c8f6c5e4b82a09bafa6ae62a41e9e7367bb/src/Tasks/Microsoft.Common.CurrentVersion.targets#L4171-L4175
• https://github.com/Microsoft/msbuild/blob/90506c8f6c5e4b82a09bafa6ae62a41e9e7367bb/src/Tasks/Microsoft.Common.CurrentVersion.targets#L4154

These will unconditionally override some of the hard link settings when building inside of Visual Studio. This means that hard links are pretty much useless in VS which is typically the primary build scenario for developers.

Here is data from the Roslyn repository on how much benefit we get from this on the command line at least:

| | Build Time | Binaries Size Explorer | Binaries Size DU |
|-|-|-|-|
| No hard link | 4:00 | 13.9GB | 14.9GB |
| Hard links | 3:28 | 13.8GB | 1.5GB |

[rainersigwald]

From original ancient commit notes:

Changeset 1114670: Fix: 665545 PERF: Change copy-local to use Hardlinks, where possible.

NOTE: This is off and is only available to command-line builds even when enabled through set CreateHardLinksForCopyFilesToOutputDirectoryIfPossible=true or setting the property in a project / targets import. i.e. It will always be off when inside VS no matter what you set CreateHardLinksForCopyFilesToOutputDirectoryIfPossible to because VS loves to arbitrarily lock assemblies in the output directory.

and a comment on that internal bug

This will be disabled by default, and forcibly disabled when in Visual Studio (VS loves to lock assemblies for some bonkers reason).

So presumably it actively caused VS build problems in 2009. That is at the very least ripe for reevaluation.

[jaredpar]

I'm going to "fix" this locally and experiment a bit. No errors won't mean too much but if I get an immediate issue I'll report back.

[Neme12]

Thanks!! I modified this file in my MSBuild directory and it's been working perfectly so far. The only question I have is: will this file need to be updated after every VS update?

[jaredpar]

I modified this file in my MSBuild directory and it's been working perfectly so far.

Man I've had a bit of trouble getting this to work for me. What changes did you make? Just commenting out the two property groups around hard linking?

[Neme12]

I removed the '$(BuildingInsideVisualStudio)' == 'true' or part in all places in the file.

[jaredpar]

Okay. Sounds like its' definitely a PEBKAC issue on my part then.

[voltcode]

Hi guys, any updates on when VS would stop preventing CreateHardLinksForCopyFilesToOutputDirectoryIfPossible from working correctly? Speeding up the build would be awesome for many developers.

[BrunoJuchli]

Regarding performance:

Back a few years ago we've set the build to use hard links if possible to increase performance.
I've now found that this wasn't active anymore in our console build (use fsutil hardlink list MyFile.exe to see list of hardlinks [note that 1 hard link is the normal case, so for hard linking to work there should be at least 2 entries being listed]).

I've adapted MS.Common.CurrentVersion.targets to use hard links by default.
Again, i've verified that this works using fsutil hardlink list MyFile.exe. All nuget dependencies were copied by hard links.

The performance improvement measured approximates to zero. Really, I've double checked.

Note:
Build contains 100+ projects and a total artifact size of over 8GiBis. Of those I expect more than 80% to be hard linked.
The disk used is a Samsung 960 EVO 500GB (m.2).

Measurements;

• run three builds without hard-linking, they took 1:20, 1:14 and 1:19.
  
  
  
  • three builds with hard-linking: 1:19, 1:17, 1:16.
  
  

---

I still consider hard-linking beneficial because it can reduce Total Bytes Written significantly and thus prolong the life of an SSD.

---

Here's a patch with the changes I made. I use MSBuild 15 (VS2017). Base-path on my machine is: C:Program Files (x86)Microsoft Visual Studio2017EnterpriseMSBuild15.0Bin

hardlink.patch.zip

It would be interesting to see if others can confirm my findings.

@jaredpar
What was the hardware (disk) you ran your tests on?

[archimed7592]

NOTE: This is _off_ and is only available to command-line builds even when enabled through set CreateHardLinksForCopyFilesToOutputDirectoryIfPossible=true or setting the property in a project / targets import. i.e. It will always be _off_ when inside VS no matter what you set CreateHardLinksForCopyFilesToOutputDirectoryIfPossible to because VS loves to arbitrarily lock assemblies in the output directory.

Is it possible to give us an option to override this behavior?
For example slightly change the condition
'$(BuildingInsideVisualStudio)' == 'true'
to
('$(BuildingInsideVisualStudio)' == 'true' and '$(BuildingInsideVisualStudio_AllowHardLinks)' != 'true')

It will allow us to define these properties at solution level in Directory.Build.props and we won't have to patch Microsoft.Common.CurrentVersion.targets every time Visual Studio is updated.