Xamarin.Forms Drawing Spec

[jassmith]

Xamarin.Forms Drawing Spec

Theming

To make MaterialShell work it not only needs a shell which appears to be material design, but all of its contents also should be material design. Today that means Custom Renderers. This is heavy and undesired from a user work perspective. Instead a better approach is required.

Simple drawing API sample

<Button x:Class="Local.MyButton">
  <Button.Template>
    <DrawingTemplate>
      <Drawing>
        <Grid>
          <RoundedRectangle Background="{x:Bind BackgroundColor}" />
          <Ellipse x:Name="TouchFeedback" Opacity="0" />
          <Text Content="{x:Bind Text}" />
        </Grid>
      </Drawing>
    </DrawingTemplate>
  </Button>
</Button>

View templates, at least for now, must be a DrawingTemplate. The first child of a DrawingTemplate is always a Drawing. Inside of a Drawing you may use Layouts and drawing primitives.

Drawing primitives may be x:named and looked up by name and manipulated in code behind as any other View. In effect a drawing primitive is simply a view who has no renderer but rather must be inside a Drawing in order to work. Drawing is limited to children it can support as all are collapsed into simple drawing commands.

Drawing primative objects may use a Brush type rather than a Color type for providing colors/backgrounds/etc.

When using a MaterialShell, all relevant controls will receive a default Template which themes them according to the Material design guidelines and unifies their look and feel across platform. This theme can be overridden at any layer of the hierarchy simply by blocking its propagation in resource dictionaries.

Once realized a Control may not have its Template changed as DrawingTemplated controls may at times require different renderers.

Types

DrawingTemplate

This is mostly a marker type. In the future we will eliminate the requirement that Templates be DrawingTemplates.

public class DrawingTemplate : ControlTemplate {}

Drawing

A View which has no renderer and is instead consumed by its parent view as a native drawing. Drawing is a very basic layout which can be measured but always sizes all of its children to the same size as itself when laid out.

public class Drawing : Layout<View> {}

New API for Drawing in View

public class View : VisualElement // new API only
{
  public ControlTemplate Template { get; set; }

  protected BindableObject GetTemplateChild(string childName);

  protected virtual void OnApplyTemplate ();
}

Brush

public class Brush : BindableObject
{
  public static readonly BindableProperty OpacityProperty;
  public double Opacity { get; set; }
}

SolidColorBrush

public class SolidColorBrush : Brush
{
  public static readonly BindableProperty ColorProperty;
  public Color Color { get; set; }
}

GradientBrush

public class LinearGradientBrush : Brush
{
  public static readonly BindableProperty GradientStopsProperty;
  [Content]
  public GradientStopCollection GradientStops { get; set; }
}

GradientStopCollection

public sealed class GradientStopCollection : IEnumerable<GradientStop>, IList<GradientStop>

GradientStop

public sealed class GradientStop : BindableObject
{
  public static readonly BindableProperty ColorProperty;
  public Color Color { get; set; }

  public static readonly BindableProperty OffsetProperty;
  public double Offset { get; set; }

  public static readonly BindableProperty StartPointProperty;
  public Point StartPoint { get; set; }

  public static readonly BindableProperty EndPointProperty;
  public Point EndPoint { get; set; }
}

Drawing Primitives

There will be a need for a large number of drawing primitives with associated properties. This document does not attempt to define all of them, just note some of the obvious ones that will need to exist. The drawing primitive API is not intended to be a wholesale replacement for SkiaSharp. It is however intended to be agnostic to the usage of Skia or native drawing backends for the paltform.

A good path forward for Skia would be to add a SkiaDrawing element that would instruct the Drawing to be rendered via Skia. Skia could then render all the stock primitives as well as provide a Skia drawing element which allows for coded drawing.

Shape

namespace Xamarin.Forms.Shapes 
{
  public class Shape : View
  {
    public static readonly BindableProperty FillProperty;
    public Brush Fill { get; set; }

    public static readonly BindableProperty StrokeProperty;
    public Brush Stroke { get; set; }

    public static readonly BindableProperty StrokeThicknessProperty;
    public double StrokeThickness { get; set; }
  }
}

Line

namespace Xamarin.Forms.Shapes 
{
  public sealed class Line : Shape
  {
    public Point Start { get; set; }
    public Point End { get; set; }
  }
}

Ellipse

namespace Xamarin.Forms.Shapes 
{
  public sealed class Ellipse : Shape
  {
  }
}

Text

public sealed class Text : Shape 
{
  // All the same properties as Label more or less
}

Rectangle

namespace Xamarin.Forms.Shapes 
{
  public sealed class Rectangle : Shape
  {
    public CornerRadius CornerRadius { get; set; }
  }
}

BezierLine

This differs quite a bit from UWP, however this is capable of drawing an Bezier curved path/shape, as well as regular polygons and polylines.

namespace Xamarin.Forms.Shapes 
{
  public sealed class BezierLine : Shape
  {
    public IList<BezierPoint> Points { get; }
    public bool ShouldClose { get; set; }
  }
}

BezierPoint

namespace Xamarin.Forms.Shapes 
{
  public sealed class BezierPoint : Point
  {
    public Size LeftControlOffset { get; set; }
    public Size RightControlOffset { get; set; }
  }
}

TODO

• [x] Add API for drawing
• [x] Fill out brush API

Issues

Shapes

There is not currents a great way to draw just an arc. The mechanism in UWP for doing this is a bit... uhg just not fun.

There is also something to be said for the fact that Shapes are currently Views. This is to make sure they can be consumed by standard Layout's which is nice because the user does not need to learn new layouting mechanisms. The downside is drawing primitives only work in the context of a Drawing, but the compiler will not stop you from using them external to one. The counter case is to make Drawing specific layouts which currently seems to be the greater evil.

Ideally Layout would allow any child which implements some ILayoutable interface, of which View and and Drawing Primitives would implement. This would however be a breaking change, so at the moment it seems View is the only viable option.

Brush

ImageBrush is probably going to be needed. Research needs to be done to ensure that every platform can support ImageBrush everywhere brushes are used.

DrawingTemplate

There currently exists no mechanism in core to switch the renderer based on a property as this will require. That will need to be added. Ideally this should be more generic than a switch based on the Template.

[MelbourneDeveloper]

@jassmith ,

This is a very good idea from a XAML standardization point of view, and will provide powerful functionality, However, I'd be careful going down this path. It will definitely contribute to the overall goal of moving Xamarin.Forms toward XAML Standard (https://github.com/Microsoft/xaml-standard). But, along with VisualStates, there's big question marks around how useful this will be in reality.

Graphical acceleration in these kinds of areas must be taken in to consideration. The lack of graphics acceleration is already an issue in some parts of the system like animations. My guess is that if Xamarin.Forms takes on the responsibility of rendering vector imaging, it will be subsequently removed from the underlying platform - and moved from GPU processing to CPU processing. This will be a massive performance hit in some cases. It's not that it shouldn't be done, but thought should be put in to whether or not the processing can be left at the GPU level instead of being moved to the CPU. Consumers of the XF library need to be made aware of what will run on native platform, and what will be rendered by the XF library.

There's also the question of balance between native drawing and cross platform drawing differences. Text is an important point. Should text be rendered the same on all platforms? I.e. should font rendering be passed over to XF to do so as to create uniformity over all platforms? Perhaps. But, I don't feel like that should be the overall project for XF. People subconsciously detect small visual details. The details might be in an animation or so on. Even a small departure from the norm on a given platform can make the user feel uncomfortable. So, it's not necessarily a good idea that all drawing be made uniform. Avalonia is certainly going down that path, but that's a totally separate case, and that's a point of difference between Avalonia and Xamarin Forms.

Also, one more comment is that care should be taken to make sure any primitive types are named the same, and are aligned as closely as possible to other platforms like UWP, and WPF so as to be inline with XAML Standard.

@davidortinau , do you have thoughts on this?

[ChaseFlorell]

One of the great features in material design is the Ripple. Do you expect to support this?
Overall I think this is fantastic!

[rogihee]

I really like this direction. For iOS I was using native view cells using the native elements (i, > options), but the more I apply custom theming I find myself just mimicking a bit the native cells and going more for a nice looking/working theme that is fluent and consistent for the app but not per se native apart from details. I think that if the appropriate underlying rendering library is used GPU will not be a problem, read Skia. The same thing powering Flutter.

Perhaps it's a seperate issue, but it would be nice to make SVG a first class citizen of Forms at the same time. Enabling neat effects such as animated tinting for an image button upon press for example.

My opinion is that in the end you want powerful, rich, native looking (not per se native!) controls that are easy to program with consistent behavior cross-platform, with the odd exception to accomodate a native quirk of a platform. E.g. a ripple effect for Android.

[jassmith]

@ChaseFlorell as a matter of fact I do. The idea being that you can x:Name things in the template and then dig them out using FindByName. Once you have the native element you can apply animations to it like anything else, however there will likely be a special way to get an animation callback. I'm still working out those exact details. With Skia doing to drawing this can easily hit 60FPS on android and even without Skia you can hit 60FPS on the other platforms.

@rogihee SVG really isn't a shipping format sorry :/ I don't want to be responsible for trying to render SVG consistently cross platforms. When are they going to add hinting to SVG's anyway...

[MelbourneDeveloper]

@jassmith , will this us hardware graphics acceleration?

[RichiCoder1]

Coming from Qt's QML, would love to see this. Combined with a few obvious builtin controls, can conver a lot of ground.

Maybe a bit premature or worth another issue , but what's the accessibility/testability story plan for controls built ground up with this?

[jassmith]

@RichiCoder1 a11y is certainly something that will need to be properly worked out. Ideally a Text element would automatically set most of the required properties for simple controls.

[juepiezhongren]

is this the feature for xf 4.0？

[jassmith]

3.0 series

[juepiezhongren]

@jassmith there is nothing about draw and shell in roadmap

[jassmith]

AFAIK the public roadmap does not include anything that has no been scheduled fully

[juepiezhongren]

will all these preview in 2018？@jassmith

[davidortinau]

@juepiezhongren we have not committed to doing these proposals. They are presented here for open discussion on their merits, deficiencies, etc.

I take it from your interest that these specs are interesting to you. Would you share some specific examples of where you see them helping you when building apps with Xamarin.Forms? What problems do you see this solving? What opportunities do you see them opening up for you?

Any real-world examples and stories you can share would be tremendously useful in helping us validate that this could be providing real value.

As always, anyone can feel free to email me if preferred. david.[email protected]

[timahrentlov]

The biggest problem for me is the "we can't do that with the Forms framework" reply that I have to give to potential customers and UX/UI. It discredits the framework and I only get so many chances with a customer.

Forms lack a compositional paradigm where we pr. project can build the UI using UI primitives. This applies to navigation, animations (incl. stuff like hero-animations), UI elements and gestures.

Will this suggestion give us such platform-neutral compositional UI primitives?

[opcodewriter]

@timahrentlov We need to see discuss and see exactly what those limitations are and the reason of those limitations. But honestly I don't even dare to think or look that far. The issue I see with Xamarin Forms is from a much simpler perspective: it still remains a project with very limited development resources. It's unrealistic or useless to discuss what it could be done, when in fact there's no actual enough power and fuel. I don't see the point. Is maybe Xamarin secretly still hoping some individuals from the OSS community will start working and fixing things? Don't get me wrong, I respect the time and effort Xamarin has put into Xamarin Forms.

[juepiezhongren]

@davidortinau
cross platform is hard to make out chracteristic visual design, unique button appearance, unique placeholder disappearing effect etc r really avoid our apps being identical. For example, mac's notice fading, with a slight explosion, is realy easy to impress its customers. Why we love wpf a lot that day, one reason i shall mention, is control template, where we enjoy paths so much for our uniqueness, like cornered hexagon button. We want xf bring these to cross platform client devs.

[opcodewriter]

@juepiezhongren if you're looking for that kind of UI customization, I don't think there's any cross-platform framework which can do that. And about WPF: it's a bad idea to mix WPF with mobile.

[juepiezhongren]

my team is using react native too, where so many jsers contribute a lot, where after 3 months we concluded that it's not productive at all. For xf, it is with drawbacks, with bugs, but basically it is solid solution, eps x.native makes all native api valid. What lacks is only ubi apperance, cusidering flutter' recent mafness. Besides, with shell, flutter shall be useless, without something like flutter.ios, flutter shall be like rn with native api limitless suffering.

[juepiezhongren]

@opcodewriter
we use skiasharp rather frequently, making special control, its performance is satisfying

[juepiezhongren]

@jassmith have u considered to port materialShell to wasm by using webgl, for skia, maybe it is too big to download

[jassmith]

correct we dont want to take a hard dep on any third party libs for the default drawing API. \ could do that

[juepiezhongren]

with xamarin.mono and shell in wasm, any alpha product will cause a dotnet renaissance in china, where hatred against dotnet is too much prevailing

[juepiezhongren]

@jassmith any good news about this issue?

[ChaseFlorell]

what news are you looking for @juepiezhongren? At this point it's a Spec posted to invoke discussion.

[juepiezhongren]

we want to see when draw and shell to be included into roadmap@ChaseFlorell
universal rendering is alrealdy valid for avalonia and flutter

[jassmith]

Timeline on this is going to depend on my personal development speed. The goal here is not to distract the entire team into these endeavors and only spare me onto it. You can track the shell branch if you want to watch it come together. After Shell will come drawing.

[jassmith]

As a note, I intend to stand up a vertical slice of drawing very quickly and then ask for the community to help if they want to get it done faster.

[MelbourneDeveloper]

This is all very nice, but will it use graphics acceleration?

[juepiezhongren]

@jassmith on ios, system.draw to be used; on android, skiasharp to be used || skiasharp for every thing?

[jassmith]

@juepiezhongren it will support multiple backends, so if you want it to use Skia it will use Skia, if you want it to use the native drawing API it will use that instead. By default it will use the native backend and Skia will be opt in so the dependency is not forced on you.

[juepiezhongren]

@jassmith it is really wired that system.draw is available for ios while not for android

[jassmith]

@juepiezhongren adding support for System.Drawing is not something that is within my purview here. Nothing will use System.Drawing however with this spec.

[juepiezhongren]

basically, draw will cause far less renderes

[jassmith]

Not really, Drawings will have to be backed by something that renders them. Unless you mean less need for custom renderers, in which case yes I agree.

[legistek]

I'm in the "do it as close to wpf/uwp as possible" camp, which I know isn't always the most popular position around here.

BUT, this seems like a great compromise between totally lookless with the framework doing ALL the rendering, and the current XF paradigm.

I assume the primitives will have counterparts on each platform and thus hardware acceleration wouldn't be an issue. Windows/iOS/Android/etc are still rendering rectangles, circles, gradients, and the like and will use hardware acceleration to the same extent they already do. This would just give us control designers the ability to design our controls using primitives rather than being stuck with Apple's idea of a button vs Microsoft's.

Do I sum that up correctly?

[jassmith]

@pmoorelegistek thats basically the idea yes. And we are tackling Material first primarily because it doesn't have Blur behind as a major design element. This means we can actually make it work everywhere (looking at you Android).

[jassmith]

when I finish Shell v1 I am immediately moving over to this

[VincentH-Net]

@jassmith This is golden. Use native controls where they fit the UX design and easily add skia-like drawn controls only for the (preferably few) design elements that need that.

No more "can't do that in Xamarin Forms" talks with designers and customers. This removes design limitations while preserving the true native feeling (unlike flutter).

I would love to see this before material shell. This solves a real limitation of XF.
Shell is the nth feature for getting started easy/faster. Getting _started_ has been the focus for way too long, but features like this for getting _further_ will _keep_ XF devs onboard.

[jassmith]

This has to happen before Material Shell because Material Shell depends on this. I think you meant to say you would prefer to see this before Shell?

Shell isn't focused on getting started faster/easier. Its meant to modernize the the Pages in ways we cant do with them being their own controls. It's basically intended to full out replace them and we will encourage people to consider porting. Shell is focused on providing a far smoother and richer experience to end users, its animation can be customized, it can delay/lazy load things before/after animations so they dont provide hiccups. It provides a 0 GPU overdraw content area in Android, compare this to current pages where overdraw is just in the "red go fuck yourself" part of the chart everywhere.

Shell is not about getting people started faster, its about making your app faster. With Shell you can, for example, mark pages/tabs/groups as "frequent" and they will be retained by the system under the assumption the user visits them constantly. This means they wont reload when you navigate away and come back. We can do that because Shell owns the whole hierarchy of pages.

There are also some thoughts we have on how to optimize drawing performance with Shell that while we certainly will never limit drawing to Shell, would potentially make drawing much more efficient with shell as we may be able to do all drawing in the same pass. Grain of salt required here as this spike hasn't been done yet, but it looks decent on paper.

[jassmith]

@weitzhandler I hope to be moving over to this in the next 4-6 weeks. This will be much faster to get working that shell. None of those times are set in stone.

[timahrentlov]

+1

[juepiezhongren]

@jassmith https://github.com/nventive/Uno
that is great.
uni-rendering is great

[juepiezhongren]

forms is going to be a radical explorer, while uno is to be a conservative one
we love both

[RadwanIbrahim]

+1 Great feature it will solve a lot of custom UI problems we want to see it asap

[juepiezhongren]

@jassmith seems that draw is to be done for 3.3.0？

[weitzhandler]

Any update on this?

For us, this is the most compelling feature we hope Xamarin to implement.

We develop mostly LoB apps, and one of our top priorities is development speed.
We care less about native look and feel as long as there is a compelling UI that does a decent job, renders nicely, and takes less development time.

Having to test and adapt each form and page separately on each platform, is a real time killer, requiring any minor UI change to be triple checked on all platforms back and forth.

Shell & Material Shell is a blessing and great news, please push this forward! This is the only thing that will keep your users going elsewhere (Uno, Flutter and others).

[juepiezhongren]

sad news that draw would not appear for 3.3

[touseefbsb]

+1 for this, coming from a uwp background I really want to use XF but it has it drawbacks, currently I am trying flutter but will surely come to XF if this is implemented ASAP.

[jrahma]

I agree this a great feature and must be implemented.. I will also include style for lines for example, solid line, dotted line, etc...

[weitzhandler]

Any news? Plans? Roadmap?

[legistek]

Funny enough, just last night I found myself experimenting with custom renderers and have almost concluded that all this can be done by just defining our own primitives as custom views and building off of them. You don't need much more than a Border, Ellipse, and Line.
I'm anxious to see this officially supported, but I ain't waiting. :)

[andreinitescu]

@jassmith
So Line and Rectangle for example, are going to be a real views? (because in the example I see there's a Grid with Ellipse as a child view). Will
Will I be able to attach add a TapGestureRecognizer to a Rectangle to handle the tap event?
Since these are real views, isn't having actual views for drawing primitive going to be detrimental to rendering speed?

[GalaxiaGuy]

@andreinitescu I believe the intention is for them to be real views on the Xamarin Forms side, but to never be realised as views natively:

_a drawing primitive is simply a view who has no renderer_

So a renderer further up the chain (Drawing? DrawingTemplate?) is responsible for iterating through drawable descendants and drawing them.

I would guess this means the same kind of restrictions that apply to Layouts with CompressedLayout.IsHeadless set would apply (no gesture recognizers, no effects, no transforms etc.)

[andreinitescu]

@GalaxiaGuy I am thinking along same lines but I find it a bit confusing, because of mixing with real views like Grid. Is it really necessary? Instead of deriving from View why not have these as abstract drawing elements (maybe have a base abstract class called DrawingElement with common properties ?)

[minaairsupport]

Nice Job just correct sample

<Button x:Class="Local.MyButton">
  <Button.Template>
    <DrawingTemplate>
      <Drawing>
        <Grid>
          <RoundedRectangle Background="{x:Bind BackgroundColor}" />
          <Ellipse x:Name="TouchFeedback" Opacity="0" />
          <Text Content="{x:Bind Text}" />
        </Grid>
      </Drawing>
    </DrawingTemplate>
  </Button.Template> --> correct this line
</Button>

[KSemenenko]

Maybe it will be in 4.0?

[charlesroddie]

Drawing is good in that it implements controls using graphics primitives independent of target platform. There used to be NControl but this had issues (from pre-netstandard2.0, pre-SkiaSharp days), and isn't updated.

The spec 1. duplicates functionality already implemented in a much more complete form in SkiaSharp, 2. adds unnecessary layers of complexity via XAML/Binding which goes inside the already unmaintainable Xamarin.Forms repo.

A better approach is to make a small control library based on SkiaSharp. NControl++

@jassmith @rogihee SVG really isn't a shipping format sorry :/ I don't want to be responsible for trying to render SVG consistently cross platforms.

SkiaSharp has SVG support.

[timahrentlov]

Any news about this?

[wuxueqicai]

Any news about this?

[legistek]

Also wondering if this is still likely to happen. I'm very much in the camp of wanting a lookless cross-platform UI framework and this seems to be the best way to achieve that.

[charlesroddie]

@legistek You can use SkiaSharp for lookless cross-platform drawing. What are the advantages of a Xamarin.Forms-specific one in your mind?

[legistek]

There's a bit more to a UI framework than drawing.

[charlesroddie]

But this thread is discussing a drawing framework not a UI framework...

[legistek]

I thought it was discussing a drawing framework _for Xamarin Forms_.

[charlesroddie]

SkiaSharp is a drawing framework for Xamarin Forms. The fact it also works on other .Net UI platforms is surely an advantage not a disadvantage.

[legistek]

And as was said in the OP, SkiaSharp could well be the underlying drawing engine on the individual platforms but the idea of this proposal is to add an abstraction layer and support drawing through XAML with all its benefits, the most notable one being data binding. What you call unnecessary complexity I call elegant.

[legistek]

Is this still alive and/or will it be rolled into MAUI?

[samhouts]

@legistek https://github.com/xamarin/Xamarin.Forms/wiki/Feature-Roadmap lists Shapes, Paths, and Brushes as on the roadmap for Xamarin.Forms 4.7.0.

[legistek]

Wonderful news! Thanks!