Monogame: Upgrade SharpDX to the latest version.

[Chaosus]

I found MonoGame uses old version of SharpDX(2.5.1). Should we start using 2.6.3 or wait for 3.0 ?

[KonajuGames]

The Windows 10 support uses the latest version, and the plan is to migrate the rest over to that version soon.

[Chaosus]

Ah, good to know. I close this issue when its happened.

[tomspilman]

So we use SharpDX 3.0 but there are even newer versions. We need someone to fix the issues around supporting the latest SharpDX releases and test it on all the SharpDX platforms.

[SimonDarksideJ]

As discovered while doing the NuGet SharpDX dependancies, I did find that versions later than Sharp DX 3.0 do not work at all.

Nothing ends up being drawn, so I'm guessing something has fundamentally changed.

If we are retargeting, we should aim for the latest 4.x release (my two pence worth)

Any ideas what could be causing this drawing breakdown? (can easily test with the new NuGet packages and simply update the SharpDX NuGet dependency to a later level.

[mrhelmut]

Both @Jjagg & @vpenades have done some work in that regard, and seem to have the runtime working on SharpDX 4.0.

[Jjagg]

I upgraded to 3.1 back then, just a few adjustments needed to get the runtime working. I ran into issues with the HLSL compiler though, which is why I didn't set up a PR. I lost track of it afterwards.

[afrodynamics]

Upgrading SharpDX is a great idea, and I think it would go a long way to resolving some issues we've had with alt-tab and "device lost/reset" events in our game.

If you take a look at PlatformPresent() you'll notice that the Present() call on the swap chain is void, despite MSDN saying it returns an HRESULT code. We are currently not handling any of the potential error codes here, and probably should be. The graphics device events (DeviceLost, DeviceReset, etc) are still unimplemented, and I believe upgrading sharp DX would help here, and probably in other places too.

This all might be just a natural part of the upgrade, but I figure I should mention it here.

[vpenades]

I've just did a proof of concept:

• Checked out latest Monogame Dev. - working on Windows Desktop Platform
• Removed SharpDx 2.6.1 references and replaced them with SharpDx 4.0.1 Nuget Packages
• Amended code:
  Bool replaced with SharpDX.Mathematics.Interop.RawBool
var reverb = new SharpDX.XAudio2.Fx.Reverb(Device);
  and

var desktopWidth = monitor.Description.DesktopBounds.Right - monitor.Description.DesktopBounds.Left;
var desktopHeight = monitor.Description.DesktopBounds.Bottom - monitor.Description.DesktopBounds.Top;

Then I created a simple Load and Render 3D cube, and the cube rendered correctly. It's true I was loading "BlenderDefaultCube.xnb" from the MonoGame Assets, without any content processing. But at first sight, upgrading the game framework to 4.01 doesn't seem too complicated.

Now I'm going to check the content pipeline SharpDx.Compiler issue previously reported.

[Jjagg]

Other stuff I did when upgrading:

• Audio was broken, had to make a small change when creating XAudio2 master voice: https://github.com/Jjagg/MonoGame/commit/0aa2f25e8a04cf244b1617dcb7a156245fc66cc8#diff-3de68f33bc3ba00f98961648ab695c79
• Lots of #ifs cleaned up because of changing SharpDX.Mathematics to SharpDX.Mathematics.Interop, and removed the SharpDX.Mathematics dll: https://github.com/Jjagg/MonoGame/commit/e58b285355c261a39d006649f2d1703f9ef06610

[mrhelmut]

Important note while considering the upgrade:

SharpDx 3.* and 4.* have dropped Direct3D 10.* (as well as XAct) support and suggest to fall back to 2.6.* for that. Direct3D 9.* is supported by all versions, though.

Direct3D 10.* represents about 10% of the Steam user base.

[SimonDarksideJ]

Would that be beneficial to then move to a D3D 9 / 11 style of support, effectively dropping/skipping 10.
Or more of a D3D 9/12 model?

[nkast]

We don't use the DX10 API as far as I know. We support older cards through DX11 and feature levels.
DX9 is still required for shared surfaces, working with XAML which is based on DX9 and things like that. That's probably the reason SharpDX still supports it.

[tomspilman]

We don't use the DX10 API as far as I know.

Correct... we don't. We only use the DX11 API which internally supports DX9 and DX10 level hardware.

[vpenades]

I've gave a try to the content processor + SharpDx 4.1

I've ran MonoGame.Tests and some asserts failed when testing TestDefines() on EffectProcessorTests.cs

After some research, I've found that on EffectObject.hlsl.cs / TwoMGFX.EffectObject.CompileHLSL method there's a SharpDx call that return an incorrect result:

// Compile the shader into bytecode.                
                var result = SharpDX.D3DCompiler.ShaderBytecode.Compile(
                    shaderInfo.FileContent,
                    shaderFunction,
                    shaderProfile,
                    shaderFlags,
                    0,
                    null,
                    null,
                    shaderInfo.FilePath);

                // Store all the errors and warnings to log out later.
                errorsAndWarnings += result.Message;

                if (result.HasErrors)
                    throw new ShaderCompilerException();

When passing an incorrectly defined shader (the one that includes the token Bar;) the SharpDX.Compile method should return a result structure where HasErrors == true, which triggers the expected ShaderCompilerException.

Instead, it returns a Result structure where result.HasErrors == false, but result.Message contains the error message pointing to the bar; token.

So this seems to be a bug in SharpDX side.

In order to sort it out temporarily, I did these changes:

bool hasErrors = result.Message != null && result.Message.ToLower().Contains("error");

// Store all the errors and warnings to log out later.
errorsAndWarnings += result.Message;

if (result.HasErrors || hasErrors)
         throw new ShaderCompilerException();

So even if the SharpDX method incorrectly reports the compilation succeeded, we can still detect a failed compilation and throw the expected exception.

@Jjagg are these the issues you were having with D3DCompiler ?

[Jjagg]

I don't really remember :3

[SimonDarksideJ]

@vpenades do you have a fork for your POC so we can pitch in?
Could really see if we can get this SharpDX upgrade in 3.7 if possible.

[vpenades]

Created the fork, but struggling with the third party dependencies, I will probably need assistence on that.

To begin with, As I understand, Monogame still supports Windows Phone 8.1, and the last SharpDx version that supports it is 3.11

After SharpDX 4.0 they dropped Windows Phone 8.1 and they only support Windows Desktop and Windows UAP with bait&switch net standard packages.

Also, it seems SharpDX is now available only as NuGet packages... I have to recover the DLLs from the NuGet Packages.

My suggestion would be to also drop Windows Phone 8.1 on MonoGame 3.7 and go for SharpDx 4.x.

The reason to go for SharpDx 4.x and skip 3.11 is that, given SharpDX did an architectural change between 3.11 and 4.x:-

• If we go for 3.11 just to keep Phone 8.1 support, we will get stuck again with 3.11, because future updates would mean to handle the non trivial SharpDx 4 architectural change.
• If we go for 4.x, we loose Phone 8.1, but it will be much easier to stay up to date with SharpDx updates.

thoughts?

[SimonDarksideJ]

@tomspilman @KonajuGames
I've no issue dropping WP8.1 for 3.7 (or Win8.1) for that matter. Like other cast off platforms, support will be rooted in the older releases.
Focus should be on Desktop, UWP, iOS / Android / Consoles. Thoughts?

[Jjagg]

I'm in favor of jumping to SharpDX4.0 too. I don't care much for Win8.1 either, does anyone even target that? Just Windows DX and UWP are fine IMO.

@vpenades What do you mean by architectural change, got any links for that? I don't see anything significant in the 4.0 release notes.

[vpenades]

@Jjagg If you check versions 2,3 and 4 of SharpDx Packages, you'll see the difference:

2.6.3 supports:

• Net.20
• Net.40
• WP8-ARM
• WP8-x86
• WinRT

3.1.1 supports:

• net45
• PCL net45+netcore45+wpa81+monoAndroid1+MonoTouch1

4.0.1 supports:

• net.40
• net.45
• netcoreapp1.0
• netstandard1.1
• uap10.0

As you can see, the frameworks support from version 2, to 3, to 4 is completely different in each version.

Version 2 does not have any cross platform support at all... it just drops assemblies for different platforms, so you can target one platform at a time, but not all platforms at the same time, leading to cross platform assembly management hell (this is also what monogame is suffering right now).

Version 3 added cross platform compatibility with a shared PCL library, which is great, but it's compatibility with UWP is limited. Compatibility is granted indirectly by supporting the right PCL profile, but that profile also implies support for android and iOS, which is completely wrong. Also, PCL has been superseded by net.standard as the preferred mechanism for cross platform development.

Version 4 finally has everything done right: there's a Net.Standard1.1 assembly that can be used as a bridge (bait) for true cross platform development, and true support for Desktop and UWP via net.4x and uap10.0 assemblies. Also, In this version, WinRT/WP8 support is dropped. It also drops PCL support in favour of Net.Standard1.1

With SharpDx 4.0.1, you can write a 3d engine as a single net.standard library, and reuse it, out of the box, in both Desktop and UWP, no if / endif blocks nor assembly referencing tricks required.

[SimonDarksideJ]

By the sounds of it @vpenades , that would mean we would need NetStandard PCL support to move to V4.
Not sure if Protobuild supports this yet or if it simply means changing the PCL profile we use currently. Would need to look in to it.

[Jjagg]

Thanks for elaborating :)
I'm still fully in favor of switching to 4.0.

we would need NetStandard PCL support to move to V4.

@DDReaper what do you mean? We can use the .Net Standard lib fine from my understanding. Or just include the uwp and desktop libs directly for their respective platform
It doesn't change how the MG PCL works, right? I thought that was just a skeleton with the MG API.

[vpenades]

@Jjagg Ideally, we should follow the same path followed by SharpDx, that is, to have an "abstract" net.standard version of Monogame.Framework. But this also needs to be done for OpenGl, not only for DirectX, which makes things a lot more complicated.

@DDReaper In order to support SharpDx 4 you don't neccesarily need to make monogame as net.standard (althought it would be the right thing to do), you can still support 4.0.1 per platform, and do the assembly switch with if /endif and assembly reference hacks as now. Also, since we have to keep supporting non-nuget references, it's almost the only choice.

[SimonDarksideJ]

@DDReaper what do you mean?

We would need to ensure our "bait and switch" PCL to support NetStandard @Jjagg , nothing more than that.
Not proposing we make MG NetStandard capable :S

[Jjagg]

If we drop WP8.1 what would we actually lose? Can WP devices that are up to date run UWP apps? If not we may want to keep it.

If everyone is fine with dropping Win8.1 and WP8.1 I'll set up a PR.

cc @tomspilman @nkast

[KakCAT]

I think this can be closed
https://github.com/MonoGame/MonoGame/pull/5949

[Jjagg]

Resolved in #5949.