Fsharp: Compiler doesn't build using dotnet

[auduchinok]

$ dotnet build FSharp.Compiler.Service.sln
Microsoft (R) Build Engine version 16.7.0+7fb82e5b2 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\FSComp.fs and C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\FSComp.resx from ..\FSComp.txt since up-to-date
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\DependencyManager.fs and C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\DependencyManager.resx from ..\Microsoft.DotNet.DependencyManager\DependencyManager.txt since up-to-date
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\FSIstrings.fs and C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\FSIstrings.resx from ..\fsi\FSIstrings.txt since up-to-date
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\UtilsStrings.fs and C:\Developer\fcs-sln\artifacts\obj\FSharp.Compiler.Service\Debug\netstandard2.0\UtilsStrings.resx from ..\..\utils\UtilsStrings.txt since up-to-date
C:\Users\Eugene.Auduchinok\.nuget\packages\xlifftasks\1.0.0-beta.20420.1\build\XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named FSComp.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Service\FSharp.Compiler.Service.fsproj]

Build FAILED.

C:\Users\Eugene.Auduchinok\.nuget\packages\xlifftasks\1.0.0-beta.20420.1\build\XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named FSComp.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Service\FSharp.Compiler.Service.fsproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:01.75

[auduchinok]

It's interesting that FSharp.sln can't be built either.

$ dotnet build FSharp.sln
Microsoft (R) Build Engine version 16.7.0+7fb82e5b2 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Server.Shared\FSharp.Compiler.Server.Shared.fsproj (in 341 ms).
  Restored C:\Developer\fcs-sln\tests\fsharpqa\testenv\src\PEVerify\PEVerify.csproj (in 341 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Service\FSharp.Compiler.Service.fsproj (in 360 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Build\FSharp.Build.fsproj (in 360 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj (in 389 ms).
  Restored C:\Developer\fcs-sln\tests\service\data\CSharp_Analysis\CSharp_Analysis.csproj (in 11 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Interactive.Settings\FSharp.Compiler.Interactive.Settings.fsproj (in 139 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Private.Scripting\FSharp.Compiler.Private.Scripting.fsproj (in 210 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.Private\FSharp.Compiler.Private.fsproj (in 256 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Compiler.nuget\Microsoft.FSharp.Compiler.csproj (in 270 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\fsc\fsc.fsproj (in 636 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\fsiAnyCpu\fsiAnyCpu.fsproj (in 78 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Compiler.Private.Scripting.UnitTests\FSharp.Compiler.Private.Scripting.UnitTests.fsproj (in 371 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.DependencyManager.Nuget\FSharp.DependencyManager.Nuget.fsproj (in 130 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\FSharp.Core\FSharp.Core.fsproj (in 94 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Compiler.Service.Tests\FSharp.Compiler.Service.Tests.fsproj (in 281 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Compiler.UnitTests\FSharp.Compiler.UnitTests.fsproj (in 368 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\fsi\fsi.fsproj (in 443 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Compiler.ComponentTests\FSharp.Compiler.ComponentTests.fsproj (in 548 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Build.UnitTests\FSharp.Build.UnitTests.fsproj (in 205 ms).
  Restored C:\Developer\fcs-sln\src\fsharp\Microsoft.DotNet.DependencyManager\Microsoft.DotNet.DependencyManager.fsproj (in 135 ms).
  Restored C:\Developer\fcs-sln\tests\FSharp.Test.Utilities\FSharp.Test.Utilities.fsproj (in 1.36 sec).
  Restored C:\Developer\fcs-sln\tests\fsharp\FSharpSuite.Tests.fsproj (in 1.68 sec).
C:\Developer\fcs-sln\artifacts\Bootstrap\fsc\Microsoft.FSharp.Targets(281,9): error MSB6006: "dotnet.exe" exited with code -2147450749. [C:\Developer\fcs-sln\src\fsharp\FSharp.Core\FSharp.Core.fsproj]
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\Microsoft.DotNet.DependencyManager\Debug\net472\DependencyManager.fs and C:\Developer\fcs-sln\artifacts\obj\Microsoft.DotNet.DependencyManager\Debug\net472\DependencyManager.resx from DependencyManager.txt since up-to-date
FSharpEmbedResourceText: Skipping generation of C:\Developer\fcs-sln\artifacts\obj\Microsoft.DotNet.DependencyManager\Debug\net472\UtilsStrings.fs and C:\Developer\fcs-sln\artifacts\obj\Microsoft.DotNet.DependencyManager\Debug\net472\UtilsStrings.resx from ..\..\utils\UtilsStrings.txt since up-to-date
  PEVerify -> C:\Developer\fcs-sln\artifacts\bin\PEVerify\Debug\net472\PEVerify.exe
  PEVerify -> C:\Developer\fcs-sln\artifacts\bin\PEVerify\Debug\netcoreapp3.1\PEVerify.dll

Build FAILED.

C:\Developer\fcs-sln\artifacts\Bootstrap\fsc\Microsoft.FSharp.Targets(281,9): error MSB6006: "dotnet.exe" exited with code -2147450749. [C:\Developer\fcs-sln\src\fsharp\FSharp.Core\FSharp.Core.fsproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:05.96

Is there any workaround to make dotnet build work?

[abelbraaksma]

Is there any workaround to make dotnet build work?

Use build.cmd?

I've played with the dotnet build commands, and you can make them work, but it requires setting the right arguments. To find them, I typically check the build server on a given PR and scroll to the appropriate command, which is output in the log. Then take that command and paste it on the commandline.

[cartermp]

@KevinRansom any thoughts on this?

[cartermp]

Speaking with @auduchinok this isn't about building the repo for the first time, it's for successive rebuilds as a part of actual development and smoke testing. This prevents building/running changes from any environment that calls dotnet build. It's fairly serious because contributors who use Rider cannot use their tool to build and test changes, even though Rider is simply calling build on the project. VS uses the _desktop_ compiler to build things, which may be why it's not an issue from VS.

[baronfel]

Just adding in that I can reproduce this issue on my macos environment as well. I updated main yesterday and running build.sh now gives three classes of errors:

• duplicate assembly attributes in PEVerify.csproj
• XliffTasks duplicate resource issues in the DependencyManager infrastructure
• Language/parser changes to numeric literals breaking the FSharp.Core.UnitTests project

/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/.NETCoreApp,Version=v3.1.AssemblyAttributes.cs(4,12): error CS0579: Duplicate 'global::System.Runtime.Versioning.TargetFrameworkAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(14,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(15,12): error CS0579: Duplicate 'System.Reflection.AssemblyConfigurationAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(17,12): error CS0579: Duplicate 'System.Reflection.AssemblyFileVersionAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(18,12): error CS0579: Duplicate 'System.Reflection.AssemblyInformationalVersionAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(19,12): error CS0579: Duplicate 'System.Reflection.AssemblyProductAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(20,12): error CS0579: Duplicate 'System.Reflection.AssemblyTitleAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/oss/fsharp/artifacts/obj/PEVerify/Debug/netcoreapp3.1/PEVerify.AssemblyInfo.cs(21,12): error CS0579: Duplicate 'System.Reflection.AssemblyVersionAttribute' attribute [/Users/chethusk/oss/fsharp/tests/fsharpqa/testenv/src/PEVerify/PEVerify.csproj]
/Users/chethusk/.nuget/packages/xlifftasks/1.0.0-beta.20502.2/build/XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named FSInteractiveSettings.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [/Users/chethusk/oss/fsharp/src/fsharp/FSharp.Compiler.Interactive.Settings/FSharp.Compiler.Interactive.Settings.fsproj]
/Users/chethusk/.nuget/packages/xlifftasks/1.0.0-beta.20502.2/build/XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named FSComp.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [/Users/chethusk/oss/fsharp/src/fsharp/FSharp.Compiler.Service/FSharp.Compiler.Service.fsproj]
/Users/chethusk/.nuget/packages/xlifftasks/1.0.0-beta.20502.2/build/XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named DependencyManager.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [/Users/chethusk/oss/fsharp/src/fsharp/Microsoft.DotNet.DependencyManager/Microsoft.DotNet.DependencyManager.fsproj]
/Users/chethusk/.nuget/packages/xlifftasks/1.0.0-beta.20502.2/build/XliffTasks.targets(123,5): error : Two or more source files to be translated in the same project are named FSDependencyManager.txt. Give them unique names or set unique XlfTranslatedFilename metadata. [/Users/chethusk/oss/fsharp/src/fsharp/FSharp.DependencyManager.Nuget/FSharp.DependencyManager.Nuget.fsproj]
/Users/chethusk/oss/fsharp/tests/FSharp.Core.UnitTests/FSharp.Core/OperatorsModule2.fs(380,42): error FS1151: This number is outside the allowable range for signed native integers [/Users/chethusk/oss/fsharp/tests/FSharp.Core.UnitTests/FSharp.Core.UnitTests.fsproj]
/Users/chethusk/oss/fsharp/tests/FSharp.Core.UnitTests/FSharp.Core/OperatorsModule2.fs(383,61): error FS1151: This number is outside the allowable range for signed native integers [/Users/chethusk/oss/fsharp/tests/FSharp.Core.UnitTests/FSharp.Core.UnitTests.fsproj]

Here's a zipped binlog (thanks Github!) that should provide all the details:
Build.binlog.zip

[baronfel]

The duplicate resources seem to follow a pattern:
The resources are being added both from the intermediate output directories on the per-project level, as well from the shared artifacts intermediate directories.

    obj/Debug/netstandard2.0/DependencyManager.resx
    obj/Debug/netstandard2.0/FSComp.resx
    obj/Debug/netstandard2.0/FSIstrings.resx
    obj/Debug/netstandard2.0/UtilsStrings.resx
    ../FSStrings.resx
    /Users/chethusk/oss/fsharp/artifacts/obj/FSharp.Compiler.Service/Debug/netstandard2.0/FSComp.resx
    /Users/chethusk/oss/fsharp/artifacts/obj/FSharp.Compiler.Service/Debug/netstandard2.0/DependencyManager.resx
    /Users/chethusk/oss/fsharp/artifacts/obj/FSharp.Compiler.Service/Debug/netstandard2.0/FSIstrings.resx
    /Users/chethusk/oss/fsharp/artifacts/obj/FSharp.Compiler.Service/Debug/netstandard2.0/UtilsStrings.resx

Here's the file listing from doing build.sh, and the error in this case is from building FSharp.Core.fsproj specifically.

[baronfel]

I got past this for my local builds by commenting out the import of the CoordinateXlif.targets on the top of Directory.Build.targets at the solution root.

[cartermp]

@auduchinok quick question, when Rider does a build of either FSharp.sln or FSharp.Compiler.Service.sln, does it do:

dotnet build

or

dotnet build <the-sln>.sln

or something else?

Either case is busted but the answer might influence how it gets busted

[brettfo]

This repo uses the Arcade SDK which is a common dotnet-wide build infrastructure. Among other things, it redirects $(IntermediateOutputPath) from the default of obj/Debug to $(RepoRoot)/artifacts/obj/$(ProjectName)/Debug/.... Looking at @baronfel's output, both values are getting selected which makes me think that dotnet build or dotnet restore was run at some point when the repo was clean (meaning Arcade hasn't been restored and didn't have an opportunity to redirect $(IntermediateOutputPath)) OR Rider was opened in that state and it ran the equivalent of build/restore. Have you noticed a pattern of when this occurs vs. when you open Rider? The only officially supported way of using any dotnet repo, Roslyn included, is to run restore.sh (or build.sh which also calls restore) any time you pull from upstream, because Arcade is updated frequently, like in #10254.

[baronfel]

This requirement is very difficult to reconcile with FSAC, which uses design-time builds via dotnet-proj-info to crack project dependencies and track things like the intermediate output directory in order to handle invalidation. It would be ideal if a non-MS-contributor-friendly workflow based on the dotnet cli (and common .net CLI concepts like the Directory.Build.props/targets) alone was workable. The dedicated FCS repo built successfully with nothing more than dotnet build which made it ideal for community engagement.

I was also trying just now to do the arcade-friendly build workflow of

• clone down fresh copy of this repo while having the correct SDK installed
• ./build.sh
• and I still get a slew of errors (different ones than reported) about conflicting references to several framework references, so at this point I just sorta shrug helplessly.

[auduchinok]

@cartermp IIRC it executes MSBuild tasks directly but it effectively works the same way as using dotnet build on needed projects.

[auduchinok]

OR Rider was opened in that state and it ran the equivalent of build/restore. Have you noticed a pattern of when this occurs vs. when you open Rider?

@brettfo Rider uses design-time build (which may do a restore internally), and it worked flawlessly prior to the recent FCS solution changes (after calling build.cmd on pulling changes). I expect Rider to work if the following works, since it's what we used to do with the solution in fcs folder.

build.cmd
dotnet build FSharp.Compiler.Service.sln

[cartermp]

Re-tagging @KevinRansom since he's back from vacation. We spoke about this breifly today, it appears that @brettfo can reproduce this too, but...with different errors. This is looking to be a big, lovely cuddle puddle of msbuild annoyances.

[KevinRansom]

Okay:

dotnet build FSharp.sln never worked. The reason it didn't work is simply that the proto compiler is built for the desktop framework and dotnet build tries to run fsc.exe on the coreclr, and that doesn't work, because there is no deps.json or runtime.config. The code controlling this has been in place for nearly ever, at least since we started shipping in the sdk.

dotnet build FSharp.CompilerService.sln never worked, because it's only just been added and it won't work for the same reason as FSharp.sln, VisualFSharp.sln. Although I imagine the old one in fcs may have because it barely used any of our build stuff.

Now ... there is a possibility that it may have appeared to work if a proto compiler hadn't been built. However, that would be pretty pointless, since running the tests would fail for any new feature not yet supported by the RTM F# compiler that ships with the dotnet sdk specified in the global.json. Which would make working on the compiler would be pretty tough ...

For example I get this error when building FSharp.sln
c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core\OperatorsModule2.fs(380,42): error FS1151: This number is outside the allowable range for signed native integers [c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj] c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core\OperatorsModule2.fs(383,61): error FS1151: This number is outside the allowable range for signed native integers [c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj] c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core\OperatorsModule2.fs(380,42): error FS1151: This number is outside the allowable range for signed native integers [c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj] c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core\OperatorsModule2.fs(383,61): error FS1151: This number is outside the allowable range for signed native integers [c:\kevinransom\fsharp\tests\FSharp.Core.UnitTests\FSharp.Core.UnitTests.fsproj]

Anyway, in order to make it work I guess we will have to build the proto compiler to be a coreclr app. I will start work on this.