Msbuild: Copy task support for symlinks

[rainersigwald]

This is a feature request from an internal Microsoft team. Copy currently supports hardlinks via UseHardlinksIfPossible. It could also support symlinks.

On Windows, this should probably have a nice error message when the build is run unelevated and the user doesn't have SECreateSymbolicLinkPrivilege.

[martinscholz83]

Hello @rainersigwald. Is it ok if i take this issue?

[Sarabeth-Jaffe-Microsoft]

Hi @maddin2016,

Thanks for your interest! Before beginning work, can you outline your design and testing strategy?

Thanks,
MSBuild Team

[AndyGerlicher]

Also we thought it was worth noting on this issue (and others like it) we need to be compatible cross-platform. At minimum it would have to not break anything in xplat, but we'd be much happier if the solution had parity on mac/linux. If you do go forward with this, you might want to develop it in the xplat branch to begin with.

Thanks again for the interest!

[martinscholz83]

Hi @Sarabeth-Jaffe-Microsoft, i think the important part here is to decide to create a hardlink or a symlink. Or neither one. Or if both values are true throw an error that user have to decide between.
Then i would create unit and integration tests to test these beahvoirs.
And of course create native Method for CreateSymlink
https://msdn.microsoft.com/de-de/library/windows/desktop/aa363866%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

[rainersigwald]

@maddin2016 Sounds good. What changes to the task parameters are you envisioning? Just adding a bool CreateSymlink, or something else?

And yes, it'd be great if you could start this work in xplat and add the POSIX syscall to create a symlink as well as the Windows one.

[martinscholz83]

Ok. Then I will start on xplat. Yes a bool. Or maybe an enum to bind UseHardlinkOrSymlink. I will see.

[martinscholz83]

@rainersigwald. How do you code in xpath. With VSCode? If open with code i got errors like Unable to resole project reference C.\...\Microsoft.Build.csproj. Or do you simply edit files with an editor.

[rainersigwald]

@maddin2016 If you've already run one command-line build, xplat should open and build just fine in Visual Studio 2015. If you're not seeing that something's wrong--can you be more precise about the errors.

[martinscholz83]

build tells me

c:\Users\martin\Documents\GitHub\msbuild\src\dir.targets(53,3): error MSB4019: Das importierte Projekt "c:\Users\martin\Documents\GitHub\msbuild\packages\Nerdbank.GitVersioning\1.4.19\build\dotnet\Nerdbank.GitVersioning.target
s" wurde nicht gefunden. Vergewissern Sie sich, dass der Pfad in der <Import>-Deklaration korrekt und die Datei auf dem
 Datenträger vorhanden ist. [c:\Users\martin\Documents\GitHub\msbuild\src\dirs.proj]

for all the projects.

[martinscholz83]

In msbuild\packages i found version Nerdbank.GitVersioning.1.4.30

[rainersigwald]

Ahh. And how are you invoking build? It should work if you run RebuildWithLocalMSBuild.cmd (current name is confusing and will be fixed soonish). I thought that problem only affected some rarer scenarios. It's #694 and will be fixed by #693.

[martinscholz83]

RebuildWithLocalMSBuild fails with

C:\Users\scholzmartin\Documents\GitHub\ms
build\dir.targets(87,5): error MS
B3073: The command "C:\Users\scholzmartin\Documents\GitHub\msbuild\bin\Debug-NetCore\Windows_NT
_Deployment_Test\co
rerun C:\Users\scholzmartin\Documents\GitHub\msbuild\bin\Debug-NetCore\Windows_NT_Deployment_Test\xunit.console.netcore.exe C:\Users\scholzmartin\Documents\GitHub\m
sbuild\bin\Debug-NetCo
re\Windows_NT_Deploym
ent_Test\\Microsoft.Build.Engine.UnitTests.dll 
 -notrait category=nonwindowstests -noshadow -parallel none -notrait "CrashesOnNetCore=true" -xml C:\Users\scholzm
artin\Documents\GitHub\msbuild\bin\Debug-NetCore\Windows_NT_De
ployment_Test\Microsoft.Build.Engine.UnitTests.dll_core_TestResults.xml  -notrait category=nonwindow
stests > C:\Users\scholzmar
tin\Documents\GitHub\msbuild\bin\Debug-NetCore\Windows_NT_Deployment_Test\Microsoft.Build.Engine.UnitTests.dll_core_stdout.txt" exited with code 1. [C:\Users\scholzmart
in\Documents\GitHub\msb
uild\src\XMakeBuildEngine\UnitTests\Microsoft.Build.Engine.UnitTests.csproj]

** Build completed. Exit code: 1

Local build failed with error level 1

I'm on a Win10 Machine with .Net Core 1.0.0 RC2 installed

[martinscholz83]

So here step by step what i've done

1. clone project
2. checkout to xplat branch
3. run RebuildWithLocalMSBuild.cmd

Is this correct?

[rainersigwald]

Yes, that's correct! Does the solution now open in VS? Getting a unit test failure isn't good but at least the build worked . . .

[martinscholz83]

Unfortunately not :confused:

[martinscholz83]

I will Test it on another machine

[martinscholz83]

Same on my second machine :confused:

[rainersigwald]

@maddin2016 Ok, no need to waste more time on this. You can work on the master branch, but please keep cross-platform compatibility and multiple runtimes in mind--add comments and so on to ease the future xplat development.

We'll try to track down what's broken in xplat currently.

[martinscholz83]

Hi @rainersigwald. I have created an early preview where i use an enum to decide between SymLink and HardLink.
Can you tell me if this goes in the right direction. Or maybe a second bool makes more sense. Because with enum type i have no idea how to map that attribute.<xs:attribute name="UseHardOrSymlinksIfPossible" type="msb:LinkType" /> It convert this into an string, an not LinkType.
Here is the commit
https://github.com/maddin2016/msbuild/commit/a6adb621150e1f5de390ca8a53c3fcadf4869188

[rainersigwald]

Thanks for checking in! Looking at the changes required, I think I'd prefer a second bool, for a few reasons:

• Backward compatibility: existing projects use UseHardlinksIfPossible, so we shouldn't remove it (even if an enum is the better choice now).
• Enum problems (as you've described)

[martinscholz83]

Ok. Then I continue with second boolean.

[martinscholz83]

I'm ready with version with a second bool. One thing i would discuss is where i can place an error if UseHardlinksIfPossible and UseSymbolicLinksIfPossible are booh selected. For now i have placed it here https://github.com/maddin2016/msbuild/blob/copy-task-support-for-symlinks/src/XMakeTasks/Copy.cs#L313. But with that we get an error for every file. Is there any higher level where we can check if both values are selected. And second question is what kind of exception we should raise https://github.com/maddin2016/msbuild/blob/copy-task-support-for-symlinks/src/XMakeTasks/Copy.cs#L315

[rainersigwald]

Can you put the error in ValidateInputs? That also suggests that you should use Log.LogErrorWithCodeFromResources with a new resource named something like Copy.OnlyOneLinkType.

[martinscholz83]

:see_no_evil: only a few scrolls down :sweat_smile:

[martinscholz83]

I've created a new resource. But the message bucket is overflowed for Copy. Is the text ok?

[rainersigwald]

I don't actually know what to do in the case of bucket overflow! We'll have to figure that out. Let's just allocate a new block (looks like MSB3891 - MSB3900 is the next free block) as Task: Copy Overflow and use MSB3891. We'll think about this harder in the pull request review, but I can't think of a better option at the moment.

For text, Copy.ExactlyOneTypeOfDestination is pretty similar to this situation (a spec error because two independent arguments are specified incompatibly). Let's riff off of that. Maybe something like

Both "{0}" and "{1}" were specified in the project file. Please choose one or the other.

?

Might also be good to rename the new resource to be consistent with that one: Copy.ExactlyOneTypeOfLink.

[martinscholz83]

Ok. That sounds good

[martinscholz83]

I think i'm ready to pull. @rainersigwald do you want to look over before pull request? https://github.com/maddin2016/msbuild/tree/copy-task-support-for-symlinks

[rainersigwald]

Go ahead and submit the PR! That's usually a cleaner way to have discussion about details.

[hickford]

Hi. Has UseSymboliclinksIfPossible been implemented? Which version of msbuild has the feature?

[rainersigwald]

@hickford: Yes, thanks to @maddin2016! It's available in VS "15" Preview 5's copy of MSBuild, which I just created a release for (so now you can see that from the commit view). This is of course prerelease. It should be in the final version of VS "15" . . . when that is released.