Cefsharp: DirectX rendering

[alphamax]

Hello,

It is not a real Issue but a way to propose modifications.
I have made changes on CefSharp in a multiple way :

• On windows OSR, make a Direct X rendering instead of Bitmap rendering
  
  
  
  • Allow region partial rendering
  
  
  • Merge combobox inside rendering (and manage rotation)
  
  
• On windows OSR, take touch natively
  
  
  
  • with this PR : https://bitbucket.org/chromiumembedded/cef/pull-requests/104/touch-processing-for-osr-different-api/diff
  
  
• On windows OSR, enhanced rendering
  
  
  
  • with this PR : https://bitbucket.org/chromiumembedded/cef/pull-requests/96/osr-fix-gpu-cpu-readback-performance/diff
  
  

I think the two last points are now temporary work and Cef55 binaries are available on this repository : https://github.com/intuilab/CefSharp/tree/55-Touch-DirectX in Nuget folders.
I want to know how to propose the first point. Do you prefer a new "Cef.WPF.DirectX" project and people can chose it ? do you prefer it in the already existing Cef.WPF project ?
Another question, do you want an adaptative rendering system (if DirectX is to lame, switch on bitmap rendering) ?

Thanks for you ideas !

[GrabzIt]

This sounds like a great improvement.

Only my opinion but I would add it to the current wpf project, as CEFSharp doesn't have different projects for different technologies and default to bitmap rendering. Then you could have a enum that allows you to select directx.

[stanislav-suslov]

Hello guys! I'm using CefSharp in my project (big thanks for your work, it's great project!) for playing video from YouTube, but perfomance is so poor. I'm beginner in WPF and my question: is this trouble by bitmap rendering? Thank you for answer.

[bjarteskogoy]

@alphamax, does this behave properly when using DPI scaling?

[Ratcha9]

@alphamax @GrabzIt
Hello,
These changes sound awesome! Do you have any plans to create a PR and merge this?

[alphamax]

@bjarteskogoy I have deactivated the "DPI" problem, i mean, in my company we do not need defferent rendering in case of DPI change.
@Ratcha9 I will propose this PR when i have "free" time. I'll do my best...

[Ratcha9]

@alphamax Thank you. I would like to test the improvements. Could you provide some nuget package or something?

[alphamax]

I have an ALTERNATE repo located here :
https://github.com/intuilab/CefSharp/tree/55-Touch-DirectX
You can download the "final" version of my dev before integrating it in my company solution.

Hope it helps you.

[mortalV]

I'm using CEFSharp for showing webGL inside my WPF project.
For performance reason I can use only WinForms version but I have many problems with controls over browser and must use 2 windows.
I've tried this alternative repo with DirectX rendering.
Maybe I making something wrong or don't understand idea but with this repo I see the same performance as for WPF version.
I as understand cef must render to texture in video memory and D3DImage just show it without memory copy but I see the same performance and CPU load as WPF.

[perlun]

@alphamax Thanks for an interesting contribution. Would you care to outline some of the major improvements in the DirectX version? (like, 3-5 bulletpoints) Just so we know what was the main reason for you doing it; I'm sure there were good reasons but let's just be explicit here.

Especially since people seems to have given it a try and not really understood the main point of it, it could make sense.

[amaitland]

On windows OSR, enhanced rendering
with this PR : https://bitbucket.org/chromiumembedded/cef/pull-requests/96/osr-fix-gpu-cpu-readback-performance/diff

The CEF issue was merged a little while ago and is available in the 63.0.0-pre01 (and all future versions).

With the changes I just committed as part of #2237 it's now possible to access the raw buffer exposed by the CefRenderHandler::OnPaint call.

I have an ALTERNATE repo located here :
https://github.com/intuilab/CefSharp/tree/55-Touch-DirectX

With the new changes anyone should be able to subclass ChromiumWebBrowser and implement their own rendering method, or their own IBitmapFactory and add support for DirectX. An unnecessary memory copy can be eliminated. I'm not really expecting that you'll see a huge improvement in performance if you do implement DirectX as the memory copies are likely CPU bound.

When CEF adds support for a shared texture/surface then it's likely performance can be improved, the upstream issue is https://bitbucket.org/chromiumembedded/cef/issues/1006/cef3-support-rendering-to-a-hardware-gl

[alphamax]

@perlun , sorry for my late response :)
We are using Cef/CefSharp as a html widget in our product. In some cases, we may have many web browsers and we could also have 4k web browsers. Those web browsers may display kind of 3D animations (100% time refreshing). We have experienced that writable bitmap copy are fairly slowing down the rendering process on a big resolution. On the other hand, we have tried to optimize it and we have found that a DirectX texture can receive frames and can "give it to WPF". Using this process and helped with the CEF pull request #96, we reach 60fps rendering on recent computers. Don't hesitate to reach me it you have further questions :)

@amaitland This is a great news, this will improve the rendering process and i think it will be easier to inject data in textures with this changes. I need to update our code and propose a fully packaged pull request now.

[alphamax]

@perlun do i try to propose a pull request ?

[alphamax]

Anyone interested ? :)

[bjarteskogoy]

Yes

[GrabzIt]

Yes

[amaitland]

There is a new CEF PR that adds actual directx support at https://bitbucket.org/chromiumembedded/cef/pull-requests/158/support-external-textures-in-osr-mode/diff

If/when that is merged we can look at adding support to CefSharp main repository, as no extra libs should be required, just expose a couple of extra methods.

Short term if you are committed to support and maintenance then we can create a CefSharp.Wpf.DirectX repository within the overarching CefSharp project here on GitHib. I think the added dependencies and the specialized knowledge required make it ideal to host as a subproject and be published as a separate Nuget package that depends on the current set of Nuget packages.

[GrabzIt]

Sounds like a good plan. Do you think the new nuget package also add be able to add directx to offscreen?

[amaitland]

Do you think the new nuget package also add be able to add directx to offscreen?

@GrabzIt Not sure how the two are related? With the OffScreen package, you can render the image provided any way you like. What scenario are you thinking of? The changes I've made so far in https://github.com/cefsharp/CefSharp/issues/2237 will allow you access to the raw IntPtr, so you can effectively do whatever you like with the rendered bitmap.

There is a new CEF PR that adds actual directx support at https://bitbucket.org/chromiumembedded/cef/pull-requests/158/support-external-textures-in-osr-mode/diff

This PR will provide the ability to render directly to a shared texture/surface. Exposing that in the OffScreen version will be trivial.

https://github.com/intuilab/CefSharp/tree/55-Touch-DirectX

My comment above relates this branch, where the bitmap is coped into memory, then copied into a texture/surface. I haven't done any performance analysis, I'm somewhat skeptical that it will actually yield amazing performance improvements, it's still limited by the same OSR rendering implementation that CEF provides currently. WPF already renders using hardware acceleration where possible. We are bound by memory copy functions, some improvements can be made on that front, e.g. only copy the dirty rect changes into the backbuffer.

I've also seen one report that WPF doesn't render as efficiently as it could, see #2275 for details.

[alphamax]

Yesterday i had a look on the separated rendering system. It is a great.

I have created a DirectXBitmapRenderHandler & tested it. Seems to work great on my side.
To be 100% sure about perf gain, you must set WindowlessFrameRate to 60. In this case i was able to gain 5 fps to a threejs demo on my computer.

Once again, It is a great improvement and the PR 158 is very interesting !

[cristiancarli]

Hi, @alphamax
I've tried to downloads and compile the brach https://github.com/intuilab/CefSharp/tree/55-Touch-DirectX, but compilation of CefSharp.Core fails with the following message:

found incorrect match for 'RuntimeLibrary': value 'MTd_StaticDebug' does not match 'MDd_DynamicDebug' in AssemblyInfo.obj CefSharp.Core in ...\libcef_dll_wrapper. lib

Any suggestions?
Thanks in advance

[alphamax]

Hello,

First point, you may try on master branch. You should build without problems (reply if not). On master, i move everything on v63:

• I rebuild cef 63 with Touch pull request.
• I rebuild the wrapper to expose the touch Api
• I adapt the C++ & Chromium cef browser to handle it on Wpf component. (i do not handle it on Wpf test app, i was too lazy)

Second point : your error come from a wrong version of the cef wrapper. It is due to a wrong flag on the command line for this specific utilisation. If you checkout and, without any changes, there is this error: blame me. I should fail replacing all wrappers properly in the solution. If you dont want to move to 63, i will fix this issue. What platform are you trying to build ? Debug or Release ?

Thanks

(ps on master, cef v63 have gpu rendering optimisation pull request & cefsharp may be rendered on directX texture)

[cristiancarli]

Hello @alphamax ,
i've tried to compile the master branch but doesn't work.
I think that the cef pakages are missing. Or I have to compile the relative cef branch too and repacking for cefsharp ?

Thanks

[alphamax]

Hello @cristiancarli

No you do not need anything special. If it does not compile immediately, i need to fix it.
Let me have a look (i was on holiday, sorry for the long delay).
I'll call you back when it is ok/fixed.

Regards,

[alphamax]

Hello @cristiancarli

It was totally my fault, i did not deactivate Nuget for the CEF packages. I also did not commit the entire workload of CEF (due to gitignore).

You can now test again. I tryied on a fresh new cloned repo and it succeed.

Feel free to reply if you have any problems.

[cristiancarli]

Hi @alphamax ,
Thank you so much, now it works.

Regards.

[cristiancarli]

Hi @alphamax , I see that there is the new version of cefsharp (65), do you think it is difficult to bring these features into the new version?

[alphamax]

Hi @cristiancarli ,

At this time we do not choose vif we stay in 63 or move in v65 (we might prefer 65).
We will achieve this after holidays with @Intuilab-Daures (early September).

Regards,

[amaitland]

CEF has added support for D3D11 shared textures, closing this, you can track progress of exposing the CEF API at #2527

The changes were only merged into CEF's master build, it will be roughly December/January before they are available in a stable release build.

[webcris77]

HI, we found and issue (crash) of v65 with DirextX render and touch fix 👍
From inside a CefSharp instance, please navigate to:
https://www.google.com/doodles

and click onto the language dropdown. You’ll get this error:
System.AccessViolationException
CopyMemoryGentle
DirectXRenderHandler

[alphamax]

I will have a look as soon as I can :)

[cristiancarli]

Hi @alphamax I found a problem in a dropdown menu, if i click on the button to open it the menu opens correctly but then im not able to select one option in the menu (using touch input, instead with mouse works correctly)

Best regards!

[alphamax]

Well thanks for your feedback. I already gave a way to fix it to another person. I just detect when popup is open and handle a touch as a mouse click (through Cef primitives).
When I have time, I will fix it properly, but I am 200% busy :/

[rukiman]

Is directx released yet in any version of CefSharp or I need to use alphamax's fork? I have a webgl page and it is chewing up a lot of cpu from all that bitmap copying even though the actual rendering is in GPU.

[amaitland]

Is directx released yet in any version of CefSharp

@rukiman No. For best performance I still recommend using the WinForms version in a WindowsFormsHost. I understand this doesn't suite everyone's requirements, regardless it is currently the best option for performance.

Now #2527 has been resolved someone with experience in DirectX can investigate how the DirectX 11 implementation provided by CEF can be used with WPF (which supports DirectX 9).

[cristiancarli]

Does anyone have news about select box issue ?

Regards

[rukiman]

Is directx released yet in any version of CefSharp or I need to use alphamax's fork? I have a webgl page and it is chewing up a lot of cpu from all that bitmap copying even though the actual rendering is in GPU.

I ended up setting the windowless frame rate to 20 and also reducing the dimensions of the webview as much as possible as I am using a transparent background webgl page. This brought down my cpu to acceptable levels. In case anyone wants a work around. I'm using the standard WPF CefSharp version.

[alphamax]

Thanks for the tips !

2019-01-17 0:56 UTC+01:00, rukiman [e-mail]:

Is directx released yet in any version of CefSharp or I need to use
alphamax's fork? I have a webgl page and it is chewing up a lot of cpu
from all that bitmap copying even though the actual rendering is in GPU.

I ended up setting the windowless frame rate to 20 and also reducing the
dimensions of the webview as much as possible as I am using a transparent
background webgl page. This brought down my cpu to acceptable levels. In
case anyone wants a work around.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/cefsharp/CefSharp/issues/2108#issuecomment-454989538

[cristiancarli]

Well thanks for your feedback. I already gave a way to fix it to another person. I just detect when popup is open and handle a touch as a mouse click (through Cef primitives).
When I have time, I will fix it properly, but I am 200% busy :/

Hi sorry to bother you but can you link or quote the solution you gave to this bug so we can try to resolve it ourself ?
Best regards.

[alphamax]

I am sorry, the code I speak about belongs to my client, I can't link it because I don't have the right to give it.
I can guide you through the resolution: In the ChromiumWebBrowser class, you can override the Touch events and then, if popup is open, You should be able to send mouse click or move to the browser instead...

Best regards,

Benjamin

[cristiancarli]

I am sorry, the code I speak about belongs to my client, I can't link it because I don't have the right to give it.
I can guide you through the resolution: In the ChromiumWebBrowser class, you can override the Touch events and then, if popup is open, You should be able to send mouse click or move to the browser instead...

Best regards,

Benjamin

Hi, thanks for the tip, we manage to resolve the bug.
Another question I have seen that in v65 there is no directx render, is that for technical reasons?
Do you think it is possible to adapt the v63 directx stuff in v65 ?

Best regards
Cristian

[alphamax]

Yes you should be able to, but you may wait V71 to be more integrated (thanks to new integration of texture).

[AliFlux]

Hi. Is there any update on DirectX rendering support? My webgl app is a bit sluggish on CefSharp.

[amaitland]

I have created #2770 to track adding support for DirectX rendering, I am looking for volunteers to help out. Please only comment if you are willing and able to help.