Microsoft-ui-xaml: Proposal: Add a way to track effective visibility of a control

[dotMorten]

Summary

There's currently no good way to know if a control is currently visible and should be rendering, without having to constantly walk the entire visual tree. This prevents you from reacting to parent controls collapsing a control (think tab and collapse controls), where you for instance might want to stop an expensive SwapChainPanel rendering loop.

Rationale

WPF has always had a read-only bool UIElement.IsVisible link and an event IsVisibleChanged link that allowed control developers to effectively stop/pause unnecessary work if a control isn't visibly active. Like stop an animation, pause DirectX rendering etc, and not waste battery at that point in time.

In UWP there's not really a good way to do this today, except walking the entire UI tree each time the LayoutUpdated event fires. You're really left with only being able to react to loaded/unloaded events.

You can detect if a specific control gets collapsed, but you'd have to monitor every single parent UI Element as well, so it's not really a practical approach.

Scope

| Capability | Priority |
|:--|:-:|
| App can determine if a control is "effectively" visible without walking ancestor trees | Must |

Functional Requirements

I think the WPF equivalent APIs (linked above) are sufficient in behavior and a (near) identical API should be added to UWP:

public class UIElement
{
 +   public bool IsVisible { get; }
 +   public event EventHandler<bool> IsVisibleChanged;
}

[robloo]

This would be really great to add to UWP. It is sorely missed and results in ugly, half-functional code such as the below. Both IsVisible and IsVisibleChanged would be a boost in a number of scenarios related to control development/optimization.

Code

        /// <summary>
        /// Determines if the given UI framework element is visible in the user interface.
        /// </summary>
        /// <param name="element">The element to test for user visiblity.</param>
        /// <param name="container">The container of the element to test, 
        /// it's assumed this is already visible.</param>
        /// <param name="isFullVisibilityRequired">Whether the element must be fully visible in the container.
        /// By default partial visiblity is allowed.</param>
        /// <returns>True if the element is visible to the user, otherwise false.</returns>
        public static bool IsVisible(this FrameworkElement element,
                                     FrameworkElement container,
                                     bool isFullVisibilityRequired = false)
        {
            Rect containerBounds;
            Rect elementBounds;
            Rect intersection;

            if ((element == null) ||
                (container == null))
            {
                return (false);
            }

            if ((element.Visibility != Visibility.Visible) ||
                (container.Visibility != Visibility.Visible))
            {
                return (false);
            }

            // Update the layout just to be sure sizes are correct
            container.InvalidateArrange();
            container.InvalidateMeasure();
            container.UpdateLayout();

            elementBounds   = element.TransformToVisual(container).TransformBounds(new Rect(0.0, 0.0,
                                                                                            element.ActualWidth,
                                                                                            element.ActualHeight));
            containerBounds = new Rect(0.0, 0.0,
                                       container.ActualWidth,
                                       container.ActualHeight);

            intersection = elementBounds;
            intersection.Intersect(containerBounds);

            if (intersection.IsEmpty)
            {
                // No intersection at all
                return (false);
            }
            else
            {
                if (isFullVisibilityRequired)
                {
                    if ((intersection.Width == elementBounds.Width) &&
                        (intersection.Height == elementBounds.Height))
                    {
                        // Full intersection
                        return (true);
                    }
                    else
                    {
                        // Only partial intersection
                        return (false);
                    }
                }
                else
                {
                    // Any intersection is valid
                    return (true);
                }
            }
        }

[Felix-Dev]

WPF has three different Visibility states (opposed to the current two in UWP): Visible, Hidden, Collapsed.

I don't know about the plans for the UWP Visibility enum, (and how the talk of bringing feature parity with WPF to UWP would affect it) but this implementation would make the API proposed above more future-proof:

public class UIElement
{
 +   public bool IsVisible { get; }
 +   public event EventHandler<VisibilityChangedEventArgs> VisibilityChanged;
}

+ public class VisibilityChangedEventArgs : EventArgs
{
 +   public Visibility OldState { get; }
 +   public Visibility NewState { get; }
}

[stmoy]

Thank you for filing this issue. I agree with the rationale that determining if something is visible, like _actually really_ visible is difficult to do on the app-side for the reasons mentioned (like monitoring the ancestor chain). Additionally, we'd also want to account for things like elements contained within open/closed popups when we report if the element is Visible.

Although this feature request is righteous, it would require changes in the underlying platform meaning the earliest we'd do this work is post-WinUI 3.0. (The roadmap can be found here: #717 ) In the meantime, we will keep this item open but in the Freezer for now.

[MikeHillberg]

Note that a related and also useful feature would be the ability to see if an element is on screen, and this proposal is not that; this IsVisible API doesn't detect that an element is clipped or occluded. Clipping is more detectable now with FrameworkElement.EffectiveViewportChanged, but occlusion would be expensive to calculate.

WPF has three different Visibility states (opposed to the current two in UWP): Visible, Hidden, Collapsed.

WPF's Visibility has three states to match the HTML behavior of both display:none (Visibility.Collapsed) and visibility:hidden (Visibility.Hidden). Visibility.Hidden has generally been considered a mistake because you can get the same thing by setting the Opacity to 0, and because the enum is more difficult to work with than a bool.

[verelpode]

I agree. In my opinion, this is a significant problem and a solution would be great to have. But is it possible that a workaround already exists? Is it possible to use Windows.UI.Composition.Visual.IsVisible as a partial workaround? I don't know the answer because the documentation for Visual.IsVisible is only a single sentence, and the source code of WinUI is not yet public. The documentation doesn't explain how this property is affected by the visibility of ancestor Visual instances.

Given an instance of a class derived from FrameworkElement or UIElement, how can we get the corresponding Visual in order to invoke Visual.IsVisible? And if an ancestor element is collapsed, do descendent elements have NO corresponding Visual at all? Can an app say that an element is hidden if it has NO corresponding Visual?

I do realize that Visual has only IsVisible and not an event IsVisibleChanged, but nevertheless Visual.IsVisible could still be usable as a partial workaround, even without the event. For example, imagine a clock element that displays the current time including seconds (either digital or analog; doesn't matter). Such a clock element must use a repeating timer that is triggered once per second. Obviously, each time the timer is triggered, it updates the clock graphic or text. Thus the clock element could invoke Visual.IsVisible each time the timer is triggered, once per second. When Visual.IsVisible returns false, the clock element stops its own timer.

This workaround is useful but less-good than an event IsVisibleChanged because the lack of the event means that the clock element auto-stops but doesn't auto-start (the clock element must be manually re-started later). Whereas if the event existed, then the clock element would BOTH auto-start and auto-stop. As I said, it's a workaround not a proper full solution. But more importantly, how exactly does Visual.IsVisible behave?

@dotMorten wrote:

I think the WPF equivalent APIs (linked above) are sufficient in behavior and a (near) identical API should be added to UWP:

The name IsVisible is easily confused with the Visibility property, therefore I suggest renaming it to IsActuallyVisible and IsActuallyVisibleChanged. This naming is consistent with the ActualWidth/ActualHeight properties. Alternatively, it could be named IsReallyVisible but this name might be too strong, especially if it doesn't detect when an element is clipped or occluded.

@MikeHillberg

Note that a related and also useful feature would be the ability to see if an element is on screen, and this proposal is not that; this IsVisible API doesn't detect that an element is clipped or occluded.

I suggest that it might be best to put the clipping and occlusion testing functionality in a separate feature -- separate to the proposed UIElement.IsVisible feature. As you mentioned, occlusion would be expensive to calculate. Considering the difficulty and expense of calculating clipping and occlusion, and considering that this check isn't always needed, it may be best to implement it separately.

If the UIElement.IsVisible feature is made too complicated/difficult, then app developers may suffer a long delay before the feature is released. When it is eventually released, developers might also suffer reliability problems/bugs as a consequence of the complexity. To avoid these problems, clipping and occlusion testing could be excluded from this proposal #674, and instead included in a separate feature request.

@Felix-Dev suggested:

public class VisibilityChangedEventArgs : EventArgs
{
public Visibility OldState { get; }
public Visibility NewState { get; }
}

I would find those properties confusing because it is unclear whether NewState is identical to UIElement.Visibility versus whether it is intended to report the actual visibility that is affected by ancestors. If NewState is intended to be identical to UIElement.Visibility, then it seems redundant/unnecessary because you can just simply invoke UIElement.Visibility. Furthermore, if this event is triggered via a deferred scheduling mechanism or queue instead of immediately executed, then the info in VisibilityChangedEventArgs could be old and out-of-date. Therefore the simpler and more reliable solution is to leave the EventArgs empty and instead just let apps invoke UIElement.Visibility and/or UIElement.IsVisible -- this makes it clear that the info isn't out-of-date.

[verelpode]

I also suggest that it might be better to define the property in the _opposite_ sense. Instead of reporting whether the element is visible, report whether it is hidden/invisible. At first glance, this reversal appears to make no difference, but actually a difference does exist: The accuracy. The reasoning is that the hidden state is easier to accurately detect/determine than the visible state. If an IsInvisible or IsActuallyHidden property returns true, then the element is definitely hidden/invisible, but if it returns false, then the element is possibly or probably visible but not guaranteed because the element might be invisible via clipping or occlusion. Thus sometimes true is given a more accurate definition than false, and false is not always the 100% exact opposite of true. Giving the property the opposite name has useful implications.

[Felix-Dev]

@verelpode
One of my ideas with the event args was to give the developer info about the previous visibility state of the UI element without having to manually track it while future-proofing this API. Of course, if the visibility of an UI element can only ever have two possible values (visible & collapsed) then there is no need for the OldState property. Combined with your good point about the NewState property, the event args can be left empty as you suggested.

[dotMorten]

The name IsVisible is easily confused with the Visibility property

I don't really agree with that. Seems pretty obvious to me for a read-only property. I don't see the reason for adding support for potential new future visibility states. None of those would matter in the user-stories described above: It's all about whether it's currently rendering or not.

There's also something to be said of precedence in WPF that has proven what it has works fine, despite WPF actually has more than just two visibility states.

[verelpode]

@dotMorten
I meant if the name is IsVisible, then people could accidentally think that IsVisible is implemented simply as follows, and I would immediately forgive anyone who makes this mistake, because it seems like an easy mistake to make.

public bool IsVisible
{
    get {
        return this.Visibility == Visibility.Visible;
    }
}

It's all about whether it's currently rendering or not.

OK, then maybe you'd prefer a name like "IsRendered" or "IsCurrentlyRendered" ?

More importantly, what do you think of the possibility of using Windows.UI.Composition.Visual.IsVisible as a partial workaround until a proper solution is released? I don't know whether it can truly be used for this purpose.

[mdtauk]

If there is a property, then ActualVisibility would fit better, and as an enum to allow more complex options in the future.

Thinking about it as a query to the renderer, you could have enum values like:

• Occluded by <element>
• Rendered in 3D space but out of viewport field of view
• Visible on a different screen (Surface Hub 2X or Foldable/Multi-Screen devices)
• Reserved for Flyout
  and so on...

Whilst the control itself knows if it is supposed to be Visible, Hidden, or Collapsed - the renderer would keep track of the more detailed visible state, which the control/code can query.

[verelpode]

@mdtauk -- I like your choice of naming and the idea of returning an enum, but I can think of a downside. If it reports occlusion, but occlusion testing is expensive to calculate, and apps don't always need this much info, then the ActualVisibility property would perform expensive calculations even when unnecessary. The cost is even worse when you consider the ongoing calculations necessary to _continually_ calculate occlusion in order to trigger the event ActualVisibilityChanged whenever the occlusion changes -- not only calculated when the ActualVisibility property is read, but calculated all the time in order to trigger the event.

To help solve this, the idea might be changed from a property to a method with a parameter to indicate whether a full test should be performed. Possibly something like the following:

public ActualVisibility GetActualVisibility(bool fullCheck);
// Alternatively:
public ActualVisibility CalculateActualVisibility(bool fullCheck);

The above still has a problem: It's unclear whether the event ActualVisibilityChanged is triggered via a full check or only a quick check. So then must we have another property bool IsFullActualVisibilityCheckPerformed? This seems complex and messy and it could result in an unreliable implementation.

If a full ActualVisibility enum is too difficult or time-consuming for the WinUI team to implement, then I'd be happy to have a simple boolean IsInvisible property.

[mdtauk]

@verelpode For what its worth, I wasn't suggesting Occluded need to be included, only that as an enum, it could be one of several statuses that could be reported. I imagine it would be a ReadOnly property, but the renderer would be responsible for reporting what the ActualVisibility would be.

The control or codebehind need not do any work to calculate it.

The renderer should know something like an occlusion, as it would be drawing pixels over the control, or only drawing a partial amount of pixels.
Occlusion would be different from Clipping, and is probably more of a Holographic state, with XAML elements on top of other elements.

[DHowett-MSFT]

Terminal is interested in this. Right now, we are rendering terminal content even when our controls are completely occluded or missing from the visual tree.