Xamarin.forms: [Spec] Touch Tracking/TouchStates on VisualElements

[pauldipietro]

_Additional credit for this spec should also go to Chase Florell, who provided the original proposal and proof-of-concept project._

Summary

The current implementation of the VisualStateManager is simple, allowing for three basic states of Normal, Disabled, and Focused. While these VisualStates may be useful, in the context of a Xamarin.Forms application, they do not handle the state of actively being pressed on by the user, or even unpressed and hover states. There is also the potential for multiple fingers touching the screen, as well. By extending the VSM's capabilities for handling these states, developers will be able to more easily implement complex touch or click-based behaviors in their applications, and set a foundation for other enhancements such as those discussed in issue #4232.

API

Extend View

public class View
{
    ///<summary>
    /// When enabled, the View will begin listening for Touches
    ///</sumary>
    ///<remarks>
    /// When a touch is detected, it will raise <see cref="Touched" /> and also toggle the VisualState to the TouchState
    ///</remarks>
    public bool ObservesTouches { get; set; } // should this be bindable?

    ///<summary>
    /// When enabled, the tracked touches will include Views outside the bounds of the current View
    ///</sumary>
    ///<remarks>
    /// Adding this will extend the TouchState to include <see cref="TouchState.Entered" /> and <see cref="TouchState.Exited" />
    ///</remarks>
    public bool TrackBoundaryChanges { get; set; }

    ///<summary>
    /// Raised when <see cref="ObservesTouches"/> is enabled and a touch is detected
    ///<summary>
    event EventArgs<TouchEventArgs> Touched;
}

TouchGesturerecognizer

public class TouchGestureRecognizer : BindableObject, IGestureRecognizer
{
    ///<summary>
    /// Raised when <see cref="View.ObservesTouches"/> is enabled and a touch is detected
    ///<summary>
    public event EventHandler<TouchEventArgs> Touched; 

    ///<summary>
    /// Raised when <see cref="View.ObservesTouches"/> is enabled and a touch is detected
    ///<summary>
    public ICommand Command { get; set; }
}

TouchState

[Flags]
public enum TouchState
{
    Entered = 1<<0,
    Exited = 1<<2,
    Cancelled = 1<<3,
    Failed = 1<<4,
    Changed = 1<<5,
    Pressed = 1<<6, 
    Released = 1<<7,
    Hover = 1<<8
}

TouchEventArgs

public class TouchEventArgs : EventArgs
{   
    public TouchState TouchState { get; }
    public IReadOnlyList<TouchPoint> TouchPoints { get; }
    public long Id { get; }
    public bool IsInContact { get; }
}

TouchPoint

public struct TouchPoint
{
    public Xamarin.Forms.Point Point { get; }
    public bool IsInOriginalView { get; }
}

Usage

The View class will receive two additional properties and one event.
ObservesTouches enables whether the view responds to the new states at all; TrackBoundaryChanges distinguishes whether the view will ignore if the press (via finger or mouse) leaves its boundaries; last is the Touched event which is raised when the touch is detected.

A TouchGestureRecognizer functions similarly to the existing TapGestureRecognizer and exists for the user to define whether the view itself observes touches (false by default), and then define a Command for when the touch occurs.

The TouchState enum defines various states that can occur to the view, which apply to the VisualStates, such as the following:

<VisualStateGroup x:Name="PressedStates">
    <VisualState x:Name="Entered" />
    <VisualState x:Name="Exited" />
    <VisualState x:Name="Cancelled" />
    <VisualState x:Name="Failed" />
    <VisualState x:Name="Changed" />
    <VisualState x:Name="Pressed" />
    <VisualState x:Name="Released" />
    <VisualState x:Name="Hover" />
</VisualStateGroup>

A sample repo can be found here that provides a basic proof of concept for how developers can benefit from these additional states. Using the following snippet of code, while remaining in a pressed state, the bottom right square's text will rotate (as shown in the below GIF):

_Note: As a proof-of-concept, this project uses Effects as the functionality is not built-in, as commented._

        <Grid Style="{StaticResource Grid}"
              BackgroundColor="White"
              HorizontalOptions="Center"
              VerticalOptions="Center"
              WidthRequest="400"
              HeightRequest="200">
            <!-- we could just as easily use the VisualElement.ObservesTouches property -->
            <!-- but this shows how we can do the same thing with a gesture recognizer -->
            <!-- also, if this lands in Xamarin.Forms, there's no need for the Effect-->
            <Grid.Effects>
                <pressed:TouchEffect Touched="TouchGestureRecognizer_OnTouched" />
            </Grid.Effects>
            <Grid.GestureRecognizers>
                <TapGestureRecognizer Tapped="TapGestureRecognizer_OnTapped" />
                <!-- adding this gesture recognizer would automatically toggle "VisualElement.ObservesTouches = true" for the View -->
                <pressed:TouchGestureRecognizer Command="{Binding OnTouchedCommand}" />
            </Grid.GestureRecognizers>

            <Label x:Name="InnerLabel" Style="{StaticResource TextStyle}">
                <Label.Text>
                Welcome to Xamarin.Forms!

                I Rotate!
                </Label.Text>
            </Label>
        </Grid>


    private async void RotateTheView(VisualElement element, TouchState touchState)
    {
        switch (touchState)
        {
            case TouchState.Pressed:
                _isPressed = true;
                break;
            case TouchState.Released:
                _isPressed = false;
                break;
            default: return;
        }

        while (_isPressed)
        {
            await element.RotateTo(360, 500, Easing.Linear);
            await element.RotateTo(0, 0); // reset to initial position
        }
    }

Considerations

The most obvious consideration is to not break any current behavior involving touches/taps. With ObservesTouches false/disabled by default, this presumably shouldn't pose any issues.

Difficulty: Medium/High

[dylanberry]

This is excellent and will help make Xamarin.Forms UI more dynamic and responsive!

[yanshouwang]

Expect this will be implement soon

[KSemenenko]

@yanshouwang are you already working on it?

[PureWeen]

We'll need to reconcile this spec against some of the existing Gesture specs

https://github.com/xamarin/Xamarin.Forms/issues/3492

[PureWeen]

@KSemenenko did you have any thoughts about the spec as it stands?

Here's what I'm thinking.

I'm not the biggest fan of adding all of these to the View
```C#
public bool ObservesTouches { get; set; }
public bool TrackBoundaryChanges { get; set; }
event EventArgs Touched;

- If we have a TouchRecognizer than we should use that for Touch events not have an event on the view.

- From what I can tell *ObservesTouches* is just a way to "activate" touch sensing on the View but I wonder if there are more implicit ways we can make that work. We already have a property called "InputTransparent" so we can easily "disable" a view from touches we just need a smart way to watch events so the thing doesn't go crazy. 

- TrackBoundaryChanges I'm curious if this is required and the motivation for setting this to false. Maybe it's a performance thing?


I feel like we're gaining too many things that influence the touch behavior of a view. 

My thinking is that we can just map ObservesTouches and TrackBoundaryChanges to the existence of

```xaml
<VisualStateGroup x:Name="PressedStates">
    <VisualState x:Name="Entered" />
    <VisualState x:Name="Exited" />
    <VisualState x:Name="Cancelled" />
    <VisualState x:Name="Failed" />
    <VisualState x:Name="Changed" />
    <VisualState x:Name="Pressed" />
    <VisualState x:Name="Released" />
    <VisualState x:Name="Hover" />
</VisualStateGroup>

If these are states someone has expressed interest in then we can do what's necessary at the platform level to track those.

If that proves to be a bad idea then perhaps we could tie this VSG to a TouchRecognizer? So adding a TouchRecognizer "activates" the view for touching?

It just strikes me as confusing to have

• ObservesTouches and InputTransparent on a view (what if these are both set to true?)
• a touch event and a TouchRecognizer

[KSemenenko]

Hello @PureWeen I really like the idea of #3492 #3479, I generally think that it should be a basic GusterRecognizer, from which we can implement such as #2220, #3504, #3480 and also #3494.
But I also really like the ability to use <VisualStateGroup x:Name="PressedStates">

I like your idea, we really can add a TouchGusterRecoginzer to the object and it can cause changes in the visual state. Because, otherwise it will not be clear how to work with gestures correctly, through VisualElement or gesture recognizer.

So if it's ok, then I can start from #3492 and #3479 to implement a TouchGusterRecoginzer who will support:

    Name = "Entered" 
    Name = "Exited" 
    Name = "Canceled" 
    Name = "Failed" 
    Name = "Changed" 
    Name = "Pressed"
    Name = "Released" 
    Name = "Hover"

and further, associate it with VisualState.

[PureWeen]

@KSemenenko sounds great!!

The states will still be apart of the view itself correct?

[KSemenenko]

@PureWeen yes. I think yes.
TouchGusterRecognizer will call the GoToState method for View.
But nothing will be added to the View itself.

And then if you want your View to recognize gestures, then you add TouchGusterRecognizer and after that you can use additional states in VSM.

[PureWeen]

Yay! That sounds great @KSemenenko

I feel like we could also implicitly add the TouchGestureRecognizer if they are wanting to watch that Grouped State but we can cross that bridge later :-)

Something additional to think about is that we want to eventually build in gesture orchestration into all of this. So you could indicate whether the Touch should be stopped by the TouchGestureRecognizer or bubble up the view hierarchy.

I'm not saying that needs to be built in at this point but I'd say keep in mind cases like having this TouchGestureRecognizer inside a ListView. Is there an implementation of TouchGestureRecognizer that will make it easy to do this orchestration? Is there a way to do it so the ListView and ContextActions stay valid even if the View inside the ListView has a touchGestureRecognizer? If that proves too difficult to get working don't let it slow you down too much but it might help to just keep that in mind.

[KSemenenko]

@PureWeen Thanks for your thoughts, I have almost finished the Android version.
and when I’m done, I’ll definitely check the work with ListView and CollectionView.

[PureWeen]

Closed in favor of

https://github.com/xamarin/Xamarin.Forms/issues/3492

[YZahringer]

@PureWeen the TouchStates on VSM, like Entered/Exited/Cancelled/Pressed/Released/Hover is still planned? I don't see any mention of this in #3492

[PureWeen]

Thank you @YZahringer !!

Moved those aspects of this spec to a specific VSM Touch States spec
https://github.com/xamarin/Xamarin.Forms/issues/10314