Microsoft-ui-xaml: Proposal: Look again at XAML rendering for WinUI 3.X and beyond

[mdtauk]

Proposal: XAML Rendering improvements

Reconsider the changes made from Windows 8 and Windows 10 onwards, compared the the rendering seen in WPF and Direct2D

Summary

WPF introduced the idea of rendering vector based UI and controls through a DirectX surface, which at the time mimicked the 2D GDI Visual Styles of Windows XP, Vista, 7 and 8.

When Windows Phone and Windows 8 came around, changes were made with text anti-aliasing, and the pixel perfect precision was lost - no doubt done for reasons of performance on limited power devices.

As we move forward, and mobile form factors are no longer a focus for Microsoft - Perhaps it is time to rethink these changes, and bring the crispness to WinUI 3.0 that WPF and GDI enjoys. Bringing these platforms closer in look as one Windows Presentation Platform.

Rationale

Comparing WPF and WinRT XAML side by side, WPF is noticeably crisper, the Cleartype coloured anti-aliasing is in use, and elements align to the pixel grid better.

But there are still some devices where performance will require looking at, so I propose these changes be implemented in a way that can be turned on or off by OEMs.

Scope

| Capability | Priority |
| :---------- | :------- |
| Cleartype will make text much more legible and readable | Must |
| Single pixel thicknesses for borders will always be crisp | Must |
| Optional flags that OEMs could activate to tweak performance at a device level | Could |
| System XAML brushes optimised for screen tech (pure black for OLED but dark grey otherwise | Could |

Open Questions

• How many options should be exposed as flags to the WinUI 3.X Renderer?
• Will there need to be Windows APIs which set these flags that screen drivers control, or OEMs set when building their system base images?
• If cross platform WinUI XAML renderers become a thing, will there need to be extensible flags to tweak how rendering occurs on those platforms?
• Are there Xbox, Surface Hub and Hololens flags that should be added to "guide" the XAML renderer on those devices?

[eklipse2k8]

Pretty sure Xaml in Win10 is rendered with DirectWrite using Direct2D/DX11 in a DirectComposition surface. Was the switch to gray-scale anti-aliasing really done for phone support? Documentation suggests it's just one of the many anti-aliasing modes you can switch DirectWrite to. Win10 in general uses gray-scale anti-aliasing instead of ClearType, and I find ClearType's RGB dithering to be a bit jarring.

Personally, I look forward to all displays supporting 200%+ DPI and then anti-aliasing can just be disabled entirely.

[mdtauk]

Pretty sure Xaml in Win10 is rendered with DirectWrite using Direct2D/DX11 in a DirectComposition surface. Was the switch to gray-scale anti-aliasing really done for phone support? Documentation suggests it's just one of the many anti-aliasing modes you can switch DirectWrite to. Win10 in general uses gray-scale anti-aliasing instead of ClearType, and I find ClearType's RGB dithering to be a bit jarring.

Personally, I look forward to all displays supporting 200%+ DPI and then anti-aliasing can just be disabled entirely.

The Windows UI as in GDI and DWM text is rendered with the coloured anti-aliasing. As does the WPF rendering. It is only the WinRT UWP apps which default to the greyscale rendering.

But that is only text, shapes and controls also do not render with pixel precision, which I believe was done for performance reasons, but end up looking muddy and blurry.

[jevansaks]

Was the switch to gray-scale anti-aliasing really done for phone support?

It was definitely deliberate in the phone days, we had a number of very interesting limitations for displays. Sub-pixel RGB cleartype has only two modes (RGB and BGR) and it makes very specific assumptions about the subpixel layout of the display. It works when the display is LCD and horizontal, but when the display is vertical then you don't have a regular "R/G/B" subpixel pattern. If you've ever tried to rotate a display to portrait in windows, you'll notice the subpixel cleartype works very poorly.

In phone we had almost no phones whose portrait orientation had a subpixel pattern that was RGB. Almost all LCD displays were actually natively landscape so you couldn't do subpixel cleartype in portrait.

Phone also had PenTile OLED displays which had a subpixel pattern which didn't have a cleartype algorithm designed for it. (That may have since changed).

Furthermore, subpixel RGB cleartype works very poorly for animations (have you ever noticed animated text in WPF "snap" at the end of the animation?) and for taking screenshots and scaling.

Personally I agree with @eklipse2k8 that we should focus our energy on high-res displays and try to get away from smoothing text, just get smaller pixels.

There is likely some concrete work to do to make sure that text is vertically aligned well to avoid the smearing problem that is shown in the original issue's screenshots. I know that one of the rendering devs discovered a bug not long ago that was causing XAML to not snap to vertical baselines; there may be more to do there.

[mdtauk]

@jevansaks That is very interesting, that the screen technologies were so directly tied to ClearType and so the decision to go greyscale was made.

For larger text it is not as much of an issue of course, And high dpi displays can scale. But for 100% scaling, on a desktop LCD monitor, the smaller text does suffer from a smudging.

How that may be solved is obviously more difficult than it may seem.

Pixel precise borders and other thin elements are still potential problems, but not sure how much of a problem it would be, to warrant any work to be done. Maybe once the code is on github, the community may notice things that could be improved.

It's just a pity that WPF can look crisper, as does CSS and HTML - compared to UWP XAML.

[charlesroddie]

@jevansaks I agree with @eklipse2k8 that we should focus our energy on high-res displays and try to get away from smoothing text, just get smaller pixels.

| Upgrades | Cost |
| --- | --- |
| 1 billion laptops to 4k | $500 billion |
| 1 billion monitors to 4k | $400 billion |
| GPU upgrades to match increased res. | $300 billion |
| Total | $1.2 trillion |

Switching cleartype back on for WinUI would cost less than $1.2 trillion.

Sub-pixel RGB cleartype has only two modes ... It works when the display is LCD and horizontal

Then we need to convey this information, and use greyscale anti-aliasing otherwise. This doesn't seem hard.

have you ever noticed animated text in WPF "snap" at the end of the animation

Nope because I would estimate that 0.0001% of all text is animated and this effect sounds subtle.

[eklipse2k8]

@charlesroddie I’d argue that if the UI teams honestly have bandwidth to address anti-aliasing changes, they should probably work to make the algorithm consistent across the entire ecosystem. Bring greyscale to GDI and WPF and bring Cleartype aliasing to UWP and then allow users to decide what looks best for them.

As displays continue to get denser, the OS can default to cheaper and more performant algorithms.

Consistency is probably the most glaring issue because each one looks different

[reli-msft]

There is a real concern: to support LCD antiliasing you need a surface with six channels (RGB, and three alphas).

[ryandemopoulos]

Thanks to @jevansaks for the history on this topic--for as long as I've been at Microsoft (~11 years now), this conversations has always been around. :) I suppose in some sense, "how to render text" has always been a hot topic in the industry, and extends to Apple and other companies as much as it has to Microsoft / Windows.

I agree that on anchored desktop machines, RGB ClearType is likely the best visual option for 100% scaling. The problem we have is one of ROI--enabling ClearType would actually be pretty expensive and require some work coordinated across multiple teams, and it would _definitely_ produce a lot of bugs (text rendering bugs are one of the most prolific and fickle sources of bugs/issues that we receive).

This leaves us at a difficult spot--do we just wait for display density to gradually change over time, and "live with" the text smoothing we have today while we wait? We don't get many complaints about our text looking blurry anymore (we've done one-off fixes to address some specific issues). I agree with @mdtauk that it's unfortunate that there are situations where WPF text looks better, but I don't know if it's important enough to dedicate fairly heavy resources to fixing--to me, it seems like our time could be better spent fixing a lot of other more impactful issues (including many other WPF parity issues we currently have).

[ryandemopoulos]

Unassigning myself for now; will keep this active to see if others the community jump on this Issue and have other ideas/comments to contribute.

[reli-msft]

Most complains about lacking LCD AA comes from China and Japan —— due to the complexity of the geometries in their writing system. These complains are written in their native languages, not English. Most of them are really vague complains about text not “being clear enough” or “being beautiful enough”. It needs a lot of work to identify exactly what caused their complain.
PC market has been stalled for years, and due to the trend of “Gaming” monitors run at high frequency rather than resolution, the pace of transition to a higher density monitor is significantly slower then our expectations.

[mdtauk]

Without being able to see the rendering code, none of us know what would be required to implement ClearType with WinUI Xaml.

There will be some devices and screens where ClearType should revert to greyscale anti-aliasing - but where a standard screen can use ClearType effectively - it probably should. It could even be made a display setting:

☑ Use ClearType™ for application text

[lukasf]

have you ever noticed animated text in WPF "snap" at the end of the animation

Nope because I would estimate that 0.0001% of all text is animated and this effect sounds subtle.

In modern UWP apps I see anmiations and transitions everywhere. Fluent design is all about motion and stuff. Having all text elements "snap" at the end of every animation, transition, after every smooth scrolling, that would sure look horrible. I do remember that unpleasant snapping in WPF animations very well, and it's good that we don't have that in UWP.

However, it would probably be helpful if we can opt-in to ClearType rendering in UWP, preferrably on a per-control level (which should propagate to children). Then apps which are very text focused could enable it for select parts of the app (or all parts, if they like). But it should not be enabled by default, imho.

[verelpode]

I'm at least happy that text does actually render in WinUI/UWP, unlike icons. Icon rendering hardly works in WinUI/UWP unfortunately. The situation with text rendering could really be much worse than it is currently -- it could be as bad as the icon rendering situation, but thankfully text rendering works much better than the non-functional icon rendering in WinUI. Thus my opinion is that icon rendering should be higher priority than improving the clarity of text rendering on increasingly high-resolution displays.

For example, take a really simple case where you have an instance of Windows.UI.Xaml.Media.Imaging.BitmapImage that was produced from a PNG, and you want to use this PNG BitmapImage as an icon. That should be super simple, right? But no, unfortunately neither Windows.UI.Xaml.Controls.BitmapIconSource nor Windows.UI.Xaml.Controls.BitmapIcon support the ability to render a specified _instance_ of BitmapImage -- surprising but true.

Next example: You have an instance of Windows.UI.Xaml.Media.PathGeometry that you want to use as a monochrome vector icon in multiple places or pages in your app. Again that should be super simple, right? But no, unfortunately exceptions are thrown whenever you try to use/share the same instance of a Geometry in multiple different instances of Windows.UI.Xaml.Controls.PathIconSource, Windows.UI.Xaml.Controls.PathIcon, or Windows.UI.Xaml.Shapes.Path.

Next example: You have an SVG file that you want to use as an icon in multiple places in your app. Again you're out of luck because the SVG renderer contains bugs.

@ryandemopoulos wrote:

it seems like our time could be better spent fixing a lot of other more impactful issues (including many other WPF parity issues we currently have).

I agree. Although text rendering isn't as good as @mdtauk wishes, at least it does actually work, so I think higher priority should be given to fixing the rendering of things that barely work, such as icons. And yes WPF parity issues would also be a big help in the case of developers trying to finish the conversion of large WPF apps to WinUI/UWP.

[be5invis]

@verelpode I think you should open another issue about that?

[verelpode]

@be5invis -- OK, I've now posted more info at https://github.com/microsoft/microsoft-ui-xaml/issues/827#issuecomment-545877729 . For the SVG issue, see issue https://github.com/microsoft/microsoft-ui-xaml/issues/825 .

[verelpode]

@be5invis -- I remembered additional icon rendering problems, more than I mentioned in my previous message, so I followed your suggestion to open a new issue:
An Iconic Disaster #1494

[Gavin-Williams]

I prefer cleartype over grayscale per pixel dithering. When I switched to UWP, I noticed it straight away as having inferior text rendering.

[dragonwolf83]

I've always noticed OneNote UWP being more blurry, this certainly help explains it.

ClearType is an important feature that needs to carry forward until a superior global replacement is introduced. It was marketed as a key Windows differentiator and had many release cycles to bake it into more of Windows. It works, and many find it to be better than known alternatives. This is a major consistency issue where font-handling should use the same anti-aliasing in all apps unless an app chooses to do something different.

As for DPI scaling, that won't work because 1080p panels are still considered standard in low-cost systems. That is not likely to change in next 3-5 years since High DPI can be sold as a premium offering. (Still see 720p TVs sold for low costs)

For devices that don't support it well, this can be handled by:

• Updating ClearType Tuner to detect display type and suggest turning off for that display. Move to Settings app under Display
• Automatic detection during plug & play to default based on display. This is already done via Display Scaling.
• OEM Guidance / Settings

Pixel Snapping could be investigated to see if animations can be better with it or find a better way to get the same level of crispiness.