Or-tools: Google.OrTools netstandard packages failing to install correctly in net472

[mwpowellhtx]

I have some .NET Framework targeted unit tests which I am updating from net45x to net472.

Others of my projects making the adjustment to netstandard2.0 work fine, but the Google.OrTools package is failing to add as a reference to my project following install.

Additionally, all of the run-times are being added as package references, when I think I should only be seeing the Windows package.

Consequently, any references to OrTools bits in the test assemblies are broken. Google.Protobuf package reference seems to be working okay, but I half expected that it would be fine.

For reference, kindly see my OrTools wrappers. Currently published versions are pre-netstandard2.0 updates. I've got those updates in the works.

[mwpowellhtx]

Part of me wonders if in fact it is a packaging, filtering issue. Another part of me wonders if for something like this it is even a tooling issue. I'm just concerned from an OrTools perspective what can be done to work over, under, through, around, etc.

[mwpowellhtx]

I can manually add the DLL reference, but I'm pretty sure with NuGet packaging involved, I do not want to be doing that, ever.

<Reference Include="Google.OrTools">
  <HintPath>..\packages\Google.OrTools.6.9.5841\ref\netstandard2.0\Google.OrTools.dll</HintPath>
</Reference>

[Mizux]

Did you try using PackageReference like I did in example/dotnet/*.csproj ?
Google.OrTools should contains the reference -> need to check the nuget package...

Currently Google.OrTools "umbrella" depends on the three runtimes.
If i remove the dependencies and only keep the runtime.json it seems that dotnet publish will retrieve the runtime (good) but dotnet build/run not (bad) IIRC.

I see a guy using a prebuild target or something similar to dynamically retrieve the runtime according to the RID when executing dotnet build. Need to retrieve how is done and if I can integrate it.

[mwpowellhtx]

@Mizux I'm not working with netcoreapp here. I am targeting net472 and working through the NuGet Package Manager.

[mwpowellhtx]

@Mizux I'm also not clear yet if/whether bits behind the scaffolding need to be or even can be better filtered depending on platform. The first thing for me is whether a NuGet Package Manager install happens cleanly end-to-end.

[mwpowellhtx]

Fingers crossed; replacing packages.config with net472 targeted <PackageReference ... /> so far so good.

[mwpowellhtx]

Hmm, I am receiving FileNotFoundException and BadImageFormatException exceptions.

I am building for AnyCPU, but I wonder should I build for x64?

System.IO.FileLoadException
Could not load file or assembly 'Google.OrTools, Version=6.9.5841.0, Culture=neutral, PublicKeyToken=600e580726a33e50' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Kingdom.OrTools.ConstraintSolver.Samples.AspectBasedSudokuSolverTests.Verify_that_aspect_yields_solution()

[stuarthillary]

I have successfully been using the latest NuGet package within a .NET 4.7.2 project. The "trick" is to follow the advice by Scott Hanselmann here https://www.hanselman.com/blog/ReferencingNETStandardAssembliesFromBothNETCoreAndNETFramework.aspx

You need to add the node <RestoreProjectStyle>PackageReference</RestoreProjectStyle> within the first <PropertyGroup> of your csproj file.

[stuarthillary]

Also, you need to build for x64 as the NuGet package does not provide a 32-bit build.

[Mizux]

@mwpowellhtx it was not a problem of netcoreapp TFM it's just the .csproj format that has been updated to be more "nuget friendly".

note: Since I usually write .csproj by hand, I never was faced to this VS IDE issue.
TIPS: using

<RestoreSources>../../packages;$(RestoreSources);https://api.nuget.org/v3/index.json</RestoreSources>

You can force Nuget/MS Build to first look in relative path to .csproj ../../packages to look for a nuget package before searching on the nuget.org registry. (look at example/dotnet :smile: )
note: this is also why I now put the packages directory in zip/tar.gz archives so it works even if we still have not uploaded it on nuget.org
warning: Still if nuget.org has a newer version that the local one it will use it.

I'll see if can make a note on it in ortools/dotnet/REDME.md

Reference:

• A mapping between project.json and csproj properties
• Additions to the csproj format for .NET Core: PackageReference
• Example of MSBuild 15.1 documentation using it

[mwpowellhtx]

I'm not targeting netcoreapp for this at the moment. I am targeting net472.

I did have to do a bit of tinkering in the csproj to insert the PackageReference items.

It's not a NuGet issue I don't think; I've got over half a dozen different sources ranging from local to staged builds, etc. That's not the issue.

@stuarthillary I'll pursue that advise, thanks!

[mwpowellhtx]

Still does not work. I think I'm past the initial error(s), but still, from Test Explorer:

[10/25/2018 11:01:57 AM Warning] [xUnit.net 00:00:00.74] Exception discovering tests from Kingdom.OrTools.ConstraintSolver.Samples.Tests: System.BadImageFormatException: Could not load file or assembly 'Kingdom.OrTools.ConstraintSolver.Samples.Tests, Version=3.0.1.19744, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An attempt was made to load a program with an incorrect format.
File name: 'Kingdom.OrTools.ConstraintSolver.Samples.Tests, Version=3.0.1.19744, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Xunit.Sdk.ReflectionAssemblyInfo..ctor(String assemblyFileName) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Reflection\ReflectionAssemblyInfo.cs:line 33   at Xunit.AppDomainManager_AppDomain.CreateObject[TObject](AssemblyName assemblyName, String typeName, Object[] args) in C:\Dev\xunit\xunit\src\xunit.runner.utility\AppDomain\AppDomainManager_AppDomain.cs:line 84
   at Xunit.Xunit2Discoverer..ctor(AppDomainSupport appDomainSupport, ISourceInformationProvider sourceInformationProvider, IAssemblyInfo assemblyInfo, String assemblyFileName, String xunitExecutionAssemblyPath, String configFileName, Boolean shadowCopy, String shadowCopyFolder, IMessageSink diagnosticMessageSink, Boolean verifyAssembliesOnDisk) in C:\Dev\xunit\xunit\src\xunit.runner.utility\Frameworks\v2\Xunit2Discoverer.cs:line 101
   at Xunit.XunitFrontController.CreateInnerController() in C:\Dev\xunit\xunit\src\xunit.runner.utility\Frameworks\XunitFrontController.cs:line 136
   at Xunit.XunitFrontController.EnsureInitialized() in C:\Dev\xunit\xunit\src\xunit.runner.utility\Frameworks\XunitFrontController.cs:line 163
   at Xunit.Runner.VisualStudio.VsTestRunner.DiscoverTestsInSource[TVisitor](XunitFrontController framework, LoggerHelper logger, TestPlatformContext testPlatformContext, RunSettings runSettings, Func`4 visitorFactory, Action`4 visitComplete, String assemblyFileName, Boolean shadowCopy, TestAssemblyConfiguration configuration) in C:\Dev\xunit\xunit\src\xunit.runner.visualstudio\VsTestRunner.cs:line 280

Or from ReSharper runner:

System.IO.FileLoadException
Could not load file or assembly 'Google.OrTools, Version=6.9.5841.0, Culture=neutral, PublicKeyToken=600e580726a33e50' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

Edit: Followed up through xunit channels for the Test Explorer/xUnit.net issue. Will focus my attention through ReSharper for the moment since it at least seems to be able to load things, identify test cases, and tries to run (unsuccessfully).

[mwpowellhtx]

@Mizux At present this is the error throughout:

System.IO.FileLoadException
Could not load file or assembly 'Google.OrTools, Version=6.9.5841.0, Culture=neutral, PublicKeyToken=600e580726a33e50' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Kingdom.OrTools.ConstraintSolver.Samples.BasicTests.VerifyThatFlattenWorksCorrectly(Int32 x, Int32 y)

Along these lines, i.e. System.IO.FileLoadException.

[mwpowellhtx]

I am standing up a netstandard and netcoreapp variant of the desktop net472 test assemblies and starting to receive better results. Tests are at least running to begin with. There is probably some tooling issue or another involved with the desktop net472 x64 targets, I'm not sure. Kind of disappointing where that's concerned, but the message is clear: desktop is not the future for dotnet. Ostensibly, perhaps netcoreapp is the way to go?

Thoughts?

[mwpowellhtx]

My tests now pass having adopted netcoreapp at the head assembly. I'll close this for now. I'll post my Github link here in a bit; I wouldn't mind having some eyes take a look at the desktop net472 issue.

[mwpowellhtx]

Oh, and BTW @lperron @Mizux it's FAST. Pretty incredibly so. This is using the legacy ConstraintSolver, for instance. I do not have metrics per se, but at least in terms of visually what I remember, very quick. Perhaps a combination of ReSharper improvements as well as overall netstandard netcore, et al. When I wrap this conversion up, I'm turning my attention to the SAT solver as a potential migration path for the ConstraintSolver wrappers.

[Mizux]

@mwpowellhtx Don't know if it help but we have this
https://github.com/google/or-tools/blob/master/ortools/dotnet/Google.OrTools.Tests/Google.OrTools.Tests.csproj.in using xunit and Google.OrTools...

[mwpowellhtx]

@Mizux Thanks. I've figured out the netcoreapp bits and they are working. It's the net472 x64 targets that are a mystery. ReSharper partially works but does not load the Google resource; it is at least discovering unit tests, i.e. for TheoryAttribute tests. The Visual Studio Test Runner (i.e. via xunit visual studio runner) does not seem to work at all, nothing seems to be discovered and they do not run at all. I have not yet tried running with xunit console runner. I'm guessing, not vetted at the moment, this is to do with an x86 based tooling, but I could be wrong.

[GuyBenhaim]

Hello,
From your message: "Oh, and BTW @lperron @Mizux it's FAST. Pretty incredibly so. This is using the legacy ConstraintSolver, for instance. I do not have metrics per se, but at least in terms of visually what I remember, very quick. Perhaps a combination of ReSharper improvements as well as overall netstandard netcore, et al. "

Have you found a way to have Solver noticeably faster? If so, can you please explain how or direct to an example?

[mwinamu]

What worked for me was first instaling Install-Package Google.Protobuf -Version 3.7.0 which is a dependency and later did Install-Package Google.OrTools -Version 6.7.4973. In other words , dependencies first then the package . Weirdly a later version still had the same issue so i had to stick to this version.Hope that helps