Paket 4: Paket.Restore.targets mess up dependencies for `dotnet pack`

Created on 16 Mar 2017 · 28Comments · Source: fsprojects/Paket

Description

potnet pack incorrectly calculate NuGet package dependencies after adding Paket.Restore.targets into fsproj.

Repro steps

Here https://github.com/fsprojects/FsUnit/tree/master everything works as expected.

build.fsx uses paket.template to create FsUnit.nuget for net45
then it uses dotnet pack to create package for netstandard
note that both project files in the same folder
then script uses dotnet-mergenupkg to merge two packages into one

Resulting package has correct dependencies:
screen shot 2017-03-16 at 6 51 38 pm 1

Execute following steps to reproduce the bug (or use branch from https://github.com/fsprojects/FsUnit/pull/104)

call paket install .paket\paket.exe install
prev step update paket to version 4 and add Paket.Restore.targets to all netstandard and netcore projects (see https://github.com/fsprojects/FsUnit/pull/104)
after that execute build.cmd BuildPackage
resulting nuget package bin\FsUnit.3.0.0.nupkg will have very strange dependencies

screen shot 2017-03-16 at 6 52 56 pm

Expected behavior

Only 3 dependencies fro .NET Standard 1.6 framework:

.NET Framework 1.6.*
FSharp.Core 4.1.*
NUnit 3.6.*

Actual behavior

Direct dependencies to lots of netframework packages

Known workarounds

bug

Source

sergey-tihon

Most helpful comment

Ok paket 4.1.3 works with dotnet cli 2.0.0-preview1-005683

yay!

forki on 30 Mar 2017

❤3

All 28 comments

Dotnet pack is not really supported since we can't specify to use direct
dependencies for it. @davkean @enricosada is there a way to know which
command was executed when we are in the paket.restore.targets?

Am 16.03.2017 18:36 schrieb "Sergey Tihon" notifications@github.com:

Description

potnet pack incorrectly calculate NuGet package dependencies after adding
Paket.Restore.targets into fsproj.
Repro steps

Here https://github.com/fsprojects/FsUnit/tree/master everything works as
expected.

build.fsx uses paket.template to create FsUnit.nuget for net45

then it uses dotnet pack to create package for netstandard

note that both project files in the same folder

then script uses dotnet-mergenupkg to merge two packages into one

Resulting package has correct dependencies:
[image: screen shot 2017-03-16 at 6 51 38 pm 1]
https://cloud.githubusercontent.com/assets/1197905/24009800/1d0d2c7a-0a87-11e7-939f-fb4453eb9a17.png

Execute following steps to reproduce the bug (or use branch from
fsprojects/FsUnit#104 https://github.com/fsprojects/FsUnit/pull/104)

call paket install .paket\paket.exe install

prev step update paket to version 4 and add Paket.Restore.targets
to all netstandard and netcore projects (see fsprojects/FsUnit#104
https://github.com/fsprojects/FsUnit/pull/104)

after that execute build.cmd BuildPackage

resulting nuget package bin\FsUnit.3.0.0.nupkg will have very
strange dependencies

[image: screen shot 2017-03-16 at 6 52 56 pm]
https://cloud.githubusercontent.com/assets/1197905/24010046/ca7a2db8-0a87-11e7-97a2-0b9595afcd19.png
Expected behavior

Only 3 dependencies fro .NET Standard 1.6 framework:

.NET Framework 1.6.*

FSharp.Core 4.1.*

NUnit 3.6.*

Actual behavior

Direct dependencies to lots of netframework packages
Known workarounds

No

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fsprojects/Paket/issues/2191, or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNPI5EtFjJwzSeE6i1s3GDG2ICPrMks5rmXMvgaJpZM4MfpDb
.

forki on 16 Mar 2017

All command now are just targets in a dependency list.

dotnet pack => dotnet msbuild /t:Pack. so you can use verbose to see flow.

dotnet msbuild /t:Pack -v diag , or maybe is ok dotnet pack -v diag too

so msbuild debugging 😄

enricosada on 16 Mar 2017

@forki Do we have workarounds? (build everything using paket only)

sergey-tihon on 16 Mar 2017

yes paket pack should work, but let me check if I can get this working before my next vacation tomorrow ;-) or at least a plan

forki on 16 Mar 2017

@sergey-tihon shit it doesn't build on my machine.

forki on 16 Mar 2017

it says Rebuild can't be found

forki on 16 Mar 2017

can you attach a zip that is just before pack?
so that I can jsut call dotnet pack?

forki on 16 Mar 2017

@forki did not know about https://tweetdeck.twitter.com ;)

Here it is https://1drv.ms/u/s!ApsoUQG-fyF-gdJfCNRphEAuIqn75Q

sergey-tihon on 16 Mar 2017

Mhm I think we would need to change the input of the packages for the pack task in NuGet.Build.Tasks.Pack.targets and filter for direct dependencies. Paket already knows which they are, but I can't see how I can tell PackTask which dependencies to add.

<!-- Call Pack -->
<PackTask PackItem="$(PackProjectInputFile)"
          PackageFiles="@(_PackageFiles)"
          PackageFilesToExclude="@(_PackageFilesToExclude)"
          PackageVersion="$(PackageVersion)"
          PackageId="$(PackageId)"
          Authors="$(Authors)"
          Description="$(Description)"
          Copyright="$(Copyright)"
          RequireLicenseAcceptance="$(PackageRequireLicenseAcceptance)"
          LicenseUrl="$(PackageLicenseUrl)"
          ProjectUrl="$(PackageProjectUrl)"
          IconUrl="$(PackageIconUrl)"
          ReleaseNotes="$(PackageReleaseNotes)"
          Tags="$(PackageTags)"
          TargetPathsToAssemblies="@(_TargetPathsToAssemblies->'%(FinalOutputPath)')"
          TargetPathsToSymbols="@(_TargetPathsToSymbols)"
          TargetFrameworks="@(_TargetFrameworks)"
          AssemblyName="$(AssemblyName)"
          PackageOutputPath="$(PackageOutputPath)"
          IncludeSymbols="$(IncludeSymbols)"
          IncludeSource="$(IncludeSource)"
          PackageTypes="$(PackageType)"
          IsTool="$(IsTool)"
          RepositoryUrl="$(RepositoryUrl)"
          RepositoryType="$(RepositoryType)"
          SourceFiles="@(_SourceFiles)"
          NoPackageAnalysis="$(NoPackageAnalysis)"
          MinClientVersion="$(MinClientVersion)"
          Serviceable="$(Serviceable)"
          VersionSuffix= "$(VersionSuffix)"
          AssemblyReferences="@(_References)"
          ContinuePackingAfterGeneratingNuspec="$(ContinuePackingAfterGeneratingNuspec)"
          NuspecOutputPath="$(BaseIntermediateOutputPath)"
          IncludeBuildOutput="$(IncludeBuildOutput)"
          BuildOutputFolder="$(BuildOutputTargetFolder)"
          ContentTargetFolders="$(ContentTargetFolders)"
          RestoreOutputPath="$(RestoreOutputAbsolutePath)"
          NuspecFile="$(NuspecFileAbsolutePath)"/>

forki on 17 Mar 2017

Worst case, the workaround could be for paket to generate and write a nuspec and then input that path as the NuspecFile param here.

baronfel on 20 Mar 2017

Tbh calling paket pack and reusing the nuspec is not that bad.

forki on 20 Mar 2017

@forki any sample build.fsx(project) that already use paket pack for netcore fsproj? that I can read\try\reuse?

sergey-tihon on 20 Mar 2017

Nope sorry. That's something we need to build. But I'm on vacation for another week.

forki on 20 Mar 2017

@sergey-tihon btw you can use git to make smaller repros

git archive -o "<archiveName>.zip" HEAD

so it only archives what's under src control so you're not archiving your packages too and you can drag&drop archives straight into comments

empty.zip

cloudRoutine on 24 Mar 2017

@cloudRoutine Thank you, in this case is easier to post a link the repo https://github.com/fsprojects/FsUnit
But @forki asked for folder ready for dotnet pack with all compilation artifacts.

sergey-tihon on 24 Mar 2017

whoops i totally overlooked that 😆

I got a very different error when I ran build.cmd BuildPackage

Finished Target: All
Starting Target: SourceLink (==> All)
source indexing C:\Users\jared\Downloads\FsUnit\FsUnit\src\FsUnit.MbUnit\bin\Release\FsUnit.MbUnit.pdb
source indexing C:\Users\jared\Downloads\FsUnit\FsUnit\src\FsUnit.MsTestUnit\bin\Release\FsUnit.MsTest.pdb
source indexing C:\Users\jared\Downloads\FsUnit\FsUnit\src\FsUnit.NUnit\bin\Release\FsUnit.NUnit.pdb
Running build failed.
Error:
System.Exception: unable to load proj file `C:\Users\jared\Downloads\FsUnit\FsUnit\src\FsUnit.NUnit\FsUnit.NUnit.netstandard.fsproj` with properties: [("Configuration", "Release")], error The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Users\jared\Downloads\FsUnit\FsUnit\src\FsUnit.NUnit\FsUnit.NUnit.netstandard.fsproj
   at [email protected](String message) in C:\projects\sourcelink\SourceLink\VsProj.fs:line 22
   at SourceLink.VsBuild.Project.Load.Static(String proj, IEnumerable`1 globalProps) in C:\projects\sourcelink\SourceLink\VsProj.fs:line 22
   at SourceLink.VsBuild.Project.LoadRelease.Static(String proj) in C:\projects\sourcelink\SourceLink\VsProj.fs:line 23
   at [email protected](String projFile) in C:\Users\jared\Downloads\FsUnit\FsUnit\build.fsx:line 172
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source)
   at [email protected](Unit _arg11) in C:\Users\jared\Downloads\FsUnit\FsUnit\build.fsx:line 170
   at Fake.TargetHelper.runSingleTarget(TargetTemplate`1 target) in C:\code\fake\src\app\FakeLib\TargetHelper.fs:line 511

cloudRoutine on 24 Mar 2017

I removed SourceLink from the build in paket4 branch (I do not run it anyway, because I compile on mono)

sergey-tihon on 24 Mar 2017

here's an initial simple project
initial.zip

and the project after pack
pakkit.zip

the obj dir holds all the fancy generated stuff used by msbuild15

so i'm pretty sure this is the nuspec being used by the pack task

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>pakkit</id>
    <version>1.0.0</version>
    <authors>pakkit</authors>
    <owners>pakkit</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Package Description</description>
    <dependencies>
      <group targetFramework=".NETCoreApp1.0">
        <dependency id="Microsoft.NETCore.App" version="1.0.4" exclude="Build,Analyzers" />
        <dependency id="FSharp.Core" version="4.1.0.2" exclude="Build,Analyzers" />
        <dependency id="System.Reflection.Metadata" version="1.0.0" exclude="Build,Analyzers" />
        <dependency id="System.Reflection" version="1.0.0" exclude="Build,Analyzers" />
        <dependency id="System.Collections.Immutable" version="1.0.0" exclude="Build,Analyzers" />
        <dependency id="System.Collections.Generic" version="1.0.0" exclude="Build,Analyzers" />
        <dependency id="System.Collections.Concurrent" version="1.0.0" exclude="Build,Analyzers" />
      </group>
    </dependencies>
  </metadata>
  <files>
    <file src="C:\Users\jared\Desktop\fsexp\pakkit\bin\Release\netcoreapp1.0\pakkit.runtimeconfig.json" target="lib\netcoreapp1.0\pakkit.runtimeconfig.json" />
    <file src="C:\Users\jared\Desktop\fsexp\pakkit\bin\Release\netcoreapp1.0\pakkit.dll" target="lib\netcoreapp1.0\pakkit.dll" />
  </files>
</package>

cloudRoutine on 28 Mar 2017

@forki As a note, i think a brutal way to do it (until paket integrate cleanly) is:

Override the nuspec just after is generated.

that can be done running a target After the GenerateNuspec target file.

So something like

<Target Name="PaketOverrideNuspec" AfterTargets="GenerateNuspec">
   <Execute Command="paket fixnuspec -o $(NuspecFilePath)" />
</Target>

So it doesnt matter what are the refernces in the nuspec generated by default by nuget

so dotnet pack will work ootb

BEST Is also possibile to just fix the deps in the nuspec, because is a simple xml file, so all the Authors, Description propertis will continue to work.
And also the Content will continue to work as usual.

enricosada on 29 Mar 2017

ok with paket 4.1 it should work. You need to run paket install once to get the new targets file, but after that we should integrate normally into dotnet pack

forki on 30 Mar 2017

ok it's not working yet

forki on 30 Mar 2017

@enricosada our task modifies the nuspec in obj but dotnet pack still puts the original nuspec into the nupkg. I assume our target runs too late

forki on 30 Mar 2017

dotnet pack .\pakkit.fsproj /p:NuspecFile=.\obj\pakkit.1.0.0.nuspec
will work in the meantime

cloudRoutine on 30 Mar 2017

Ok paket 4.1.3 works with dotnet cli 2.0.0-preview1-005683

yay!

forki on 30 Mar 2017

❤3

Thank you @forki for fix!

First run (after *.targets update) build failed (see image below) looks like I have to clean obj folder from build script somewhere before dotnet pack. Just want to verify that it is expected behaviour
paket4

FsUnit.NUnit.netstandard -> /Users/sergey/github/FsUnit/src/FsUnit.NUnit/bin/Release/netstandard1.6/FsUnit.NUnit.dll
  Paket version 4.1.6
  Paket failed with:
        Specified file 'obj/FsUnit.2.4.0-alpha.nuspec;obj/FsUnit.3.0.0-alpha-1.nuspec;obj/FsUnit.3.0.0-beta1.nuspec;obj/FsUnit.3.0.0.nuspec;obj/FsUnit.3.0.1.nuspec
NUnit.2.4.0-alpha.nuspec;obj/FsUnit.NUnit.netstandard.3.0.0-alpha-1.nuspec' does not exist.
/Users/sergey/github/FsUnit/.paket/Paket.Restore.targets(59,5): error MSB3073: The command "/Library/Frameworks/Mono.framework/Commands/mono --runtime=v4.0.30319 "
y/github/FsUnit/.paket/paket.exe" fix-nuspec file "obj/FsUnit.2.4.0-alpha.nuspec;obj/FsUnit.3.0.0-alpha-1.nuspec;obj/FsUnit.3.0.0-beta1.nuspec;obj/FsUnit.3.0.0.nus
it.3.0.1.nuspec;obj/FsUnit.NUnit.2.4.0-alpha.nuspec;obj/FsUnit.NUnit.netstandard.3.0.0-alpha-1.nuspec" references-file "/Users/sergey/github/FsUnit/src/FsUnit.NUni
.NUnit.netstandard.fsproj.references" " exited with code 1. [/Users/sergey/github/FsUnit/src/FsUnit.NUnit/FsUnit.NUnit.netstandard.fsproj]
Running build failed.
Error:
System.Exception: dotnet command failed on pack "FsUnit.NUnit.netstandard.fsproj" /p:Version=3.0.1 --configuration Release
  at [email protected] (System.String message) [0x00001] in <58b6a6e7ccf1c534a7450383e7a6b658>:0
  at Microsoft.FSharp.Core.PrintfImpl+StringPrintfEnv`1[TResult].Finalize () [0x00012] in <5584aad2904cf4daa7450383d2aa8455>:0
  at Microsoft.FSharp.Core.PrintfImpl+Final1@224[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x00038] in <5584aad29
383d2aa8455>:0
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3253[T2,TResult,T1].Invoke (T2 u) [0x00001] in <5584aad2904cf4daa7450383d2aa8455>:0
  at Fake.DotNetCli.RunCommand (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] setCommandParams, System.String args) [0x00055] in <58b6a6e7ccf1c534a7450383e7a6b658>
  at [email protected] (Microsoft.FSharp.Core.Unit _arg5) [0x00051] in <56e49d341f454df2b321248bc68aa6d8>:0
  at Fake.TargetHelper+targetFromTemplate@195[a].Invoke (Microsoft.FSharp.Core.Unit unitVar0) [0x00001] in <58b6a6e7ccf1c534a7450383e7a6b658>:0
  at Fake.TargetHelper.runSingleTarget (Fake.TargetHelper+TargetTemplate`1[a] target) [0x00040] in <58b6a6e7ccf1c534a7450383e7a6b658>:0

sergey-tihon on 3 Apr 2017

no idea? does it work now?

forki on 3 Apr 2017

Seems like yes (almost fine)...
after deletion of obj folder with old nuspec files.

During the build I got an error from Restore.targets(17,5). Seems like paket assume that each project has corresponding paket.references file. But build did not failed.

  Paket version 4.1.6
  Paket failed with:
        Could not find file "/Users/sergey/github/FsUnit/tools/paket.references".
/Users/sergey/github/FsUnit/.paket/Paket.Restore.targets(17,5): error MSB3073: The command "/Library/Frameworks/Mono.framework/Commands/mono --runtime=v4.0.30319 "
/Users/sergey/github/FsUnit/.paket/paket.exe" restore --project "/Users/sergey/github/FsUnit/tools/NuGetMerge.fsproj" " exited with code 1. [/Users/sergey/github/F
sUnit/tools/NuGetMerge.fsproj]
  Restoring packages for /Users/sergey/github/FsUnit/tools/NuGetMerge.fsproj...
  Restoring packages for /Users/sergey/github/FsUnit/tools/NuGetMerge.fsproj...

paket4-nugetmerge