Wpf: Narrator doesnt read the Name, level and position of Tree View items of WPF Tree View control

[MohammedAjaz]

PWD Impact:
Users who rely on the screen readers will be impacted if screen reader are not narrating the name for the tree view control as user is unable to understand the purpose of the control.

Test Environment:

WPF_TreeViewControlNarratorIssues.zip
OS: Windows 10 Version 1909 (OS Build 18363.592)
Excel: Dogfood Version 2001(Build 12430.20120)
MSRA V2(Add in): Version 2.3.0.0
Screen Reader (s): Narrator, JAWS (2019.1912.1), NVDA (2019.2.1)

Prerequisite: Install MSRA V2 add in to the excel.
https://docs.microsoft.com/en-us/dotnet/api/system.windows.controls.treeview?view=netcore-3.1

Repro Steps:
Step 1: Enable Narrator and open the excel in desktop.
Step 2: Navigate to ribbon (header) using 'F6'.
Step 3: Navigate to the 'MSRA V2' tab control using right/left arrow keys.
Step 4: Navigate to the 'New' control under the MSRA V2 tab section using tab and press enter to open 'MS Reports' home screen'.
Step 5: Navigate to the 'Business Bar' section using F6.
Step 6: Navigate to 'Data' control present in the black ribbon section using narrator scan mode and press enter to open 'Data Section'.
Step 7: Verify whether narrator is narrating the name for the tree view present in the right pane data section or not.

Actual Result:
1.Screen readers are not narrating the 'Name' for the 'Data Tree View' control present in the right pane 'Data' section.
Ex: Screen readers (Narrator, JAWS & NVDA) narrates as 'tree view'
2.Screen readers are not narrating the name and state for the elements present in the tree view. Screen readers are narrating the junk information.
Ex:
When focus is on 'Sales' expand/collapse control screen readers narrates as below
Narrator: MS.IT.Services.MSRXV2.BIZEngine.MSRXAvailableFieldFolder 2 of 10 expanded
JAWS: Sales open selected
NVDA: level 2 MS.IT.Services.MSRXV2.BIZEngine.MSRXAvailableFieldFolder expanded
Refer attachment (MAS 4.1.2_Data Section_Data Tree_tree items incorrect name_JAWS.wmv, MAS 4.1.2_Data Section_Data Tree_tree items name & state_Narrator.wmv, MAS 4.1.2_Data Section_Data Tree_tree items name_NVDA.wmv)

Note: Same issue is observed for the button controls present in the Product, Volume Licensing, Organization, Geography, Subscription, Time & Other sections left pane also.

Expected Result:
1.Screen readers should narrate the descriptive Name for the 'Data Tree View' control present in the right pane 'Data' section.
Ex: Screen readers (Narrator, JAWS & NVDA) narrates as 'Data tree view'
2.Screen readers should narrate the name, level and state for the elements present in the tree view.
Ex:
When focus is on 'Sales' expand/collapse control screen readers should narrate 'Sales Level 2 1 of 9 expanded/collapsed'
When focus is on the 'Sales' tree item screen readers should narrate 'Sales Level 3 1 of 4'

[SamBent]

This is not a WPF bug.

Name: For UI elements (like the TreeView control), the app is responsible for supplying a "descriptive name"; WPF cannot deduce it. [Note that the x:Name or FrameworkElement.Name property assigns a name for programmatic use, it's not appropriate for exposure to end-users.] For data items (like the 'Sales' item), the WPF uses item.ToString() as a fallback, but this is usually not helpful as it defaults to the full class name "MS.IT.Services.MSRXV2.BIZEngine.MSRXAvailableFieldFolder". Again, the app is responsible for supplying a better name, as WPF cannot deduce one.

The app can supply a better name by setting the AutomationProperties.Name property. For UI elements, do this directly in markup. For data items, use the ItemContainerStyle of the enclosing ItemsControl to set the property on the corresponding container. You can use databinding to make the name depend on properties of the data item. For TreeView (and other hierarchical scenarios) you have to do this at each level of hierarchy.

This is illustrated by the attached sample. The TreeView on the left declares no AutomationProperties.Name values, and behaves as described in the bug. The one on the right is identical except it declares AutomationProperties.Name values, and behaves like the "expected behavior".
TreeViewAuto.zip

Level: There is no automation property corresponding to level; it appears that NVDA deduces level from other information, but Narrator and JAWS don't announce level at all. That's a screen-reader issue, not WPF.

Position: The position information comes from the PositionInSet and SizeInSet automation properties, which WPF sets as of .NET 4.8, as announced here. (In older versions, readers deduce the position and size from the automation tree, often incorrectly.) Older apps can opt in to the newer accessibility behaviors, as described here.

[TrishalaPatne]

I am from the app team. We reviewed your treeviewauto.zip. We have supplied better name in better name by setting the AutomationProperties.Name property to give position. For example we set it to "1 of 6". The issue is narrator reads it incorrectly as "2 of 7 1 of 6".
Just to note if we don't provide better name at the ALL element it read incorrectly as "2 of 7"

[SamBent]

Setting AutomationProperties.Name has no effect on how readers announce positions like "2 of 7". (And setting it to "1 of 6" will just confuse end users.) The incorrect "2 of 7" arises because the reader doesn't see values for PositionInSet or SizeInSet, and tries to deduce the values from the automation tree. But it doesn't really know which elements should contribute, and gets it wrong.

This is already fixed in .NET 4.8, but to see the fix you need to retarget your app to 4.8, or opt-in to the 4.8 accessibility behaviors.

[ryalanms]

@MohammedAjaz: Do you have enough information on how to fix this in your application? Thanks.

[MohammedAjaz]

I have logged the bug on-behalf w.r.t user experience, @TrishalaPatne could help here if above comment from Sam Bent would work for application team

[SamBent]

Closing - there's no action needed (or possible) from WPF. @MohammedAjaz @TrishalaPatne - please file a bug with owners of MSRA, if you haven't already done so.

[TrishalaPatne]

Hello Sam,

We retargeted to 4.8 and implemented positioninset and sizeofset as follows. We have also added runtime as given below in appconfig.

After making this changes, the issue still persists. Narrator is reading incorrectly. It is still saying 2 of 3 when there is only two elements in tree.

[SamBent]

@TrishalaPatne The interesting part of your comment is missing - I don't know what changes you made. But I suspect the problem is either that your machine doesn't have .NET 4.8, or that you declared the AppContext switches in the wrong place.

If your machine has .NET 4.7.2 (or earlier), the switch enabling 4.8 accessibility features won't have any effect. And you're component is an Excel add-in, not an independent app, so settings in your app.config may not be visible at runtime. I don't know enough about add-ins to guide you here; you should ask Excel experts how add-ins are supposed to declare config information.

For testing purposes, try setting the switches via the registry (see doc). That's not a real solution, as it affects every app on your machine, which you don't want to do. But at least you'll know if the switches fix your Narrator problem.

You could also try setting the switches programmatically. This may only work if you do it early enough, before WPF reads and caches the value. This would be a real solution. While it affects only the current app (Excel), including any other add-ins, that's true of any solution; there's no way to get WPF to behave one way for some callers and a different way for others.

[TrishalaPatne]

Hello Sam,

Given, I am to build plug in successfully with position and size parameter, application is picking up 4.8 runtime. Before I retargeted to 4.8, using 4.7 i was getting an build error when i tried to use sizeofset and positioninset.

I have 4.8 installed in my machine and I retargeted whole application to use 4.8. Hence now i can see position and size parameters in accessibility properties which I was not able to see previously when application was on 4.7. Hopefully now you can see code as below.

I have also tried setting up registry key for app context.
However after setting up sizeofset and positioninset parameters as given below, it still narrating it incorrectly.

<Setter Property="AutomationProperties.Name" Value="{Binding Path=ScreenReaderAccessibleFolderName}"></Setter> <Setter Property="AutomationProperties.PositionInSet" Value="{Binding Path=ValueForPositionInset}"></Setter> <Setter Property="AutomationProperties.SizeOfSet" Value="{Binding Path=ValueForSizeInset}"></Setter>

Values are assigned to parameter of object which is initialized when child element is created.
Added following code in app config

<runtime> <!-- AppContextSwitchOverrides value attribute is in the form of 'key1=true|false;key2=true|false --> <AppContextSwitchOverrides value="Switch.UseLegacyAccessibilityFeatures=false;Switch.UseLegacyAccessibilityFeatures.2=false;Switch.UseLegacyAccessibilityFeatures.3=false" /> </runtime>

[SamBent]

For each item, there are four interesting values: (1) the value of your data object's ValueForPositionInset property, (2) the value of the corresponding TreeViewItem's AutomationProperties.PositionInSet property, (3) the value an automation client sees for PositionInSet, (4) the value 'x' in the narration "x of y". You can check (1) by debugging your code, (2) using debugger's LiveVisualTree/LivePropertyExplorer feature, (3) using the Accessibility Insights tool to inspect the item, and (4) by listening to the narration. At which point do they disagree from the expected value?

If (1), you set the wrong value; if (2), the binding isn't working; if (3), WPF isn't responding to automation queries correctly; if (4), Narrator is still using its legacy method to compute x/y. For (1), fix your code. For (2), check the debugger's output window for binding errors - perhaps you merely spelled the property name wrong (case matters); you can add tracing to the binding to get more information in the output window. (3) is covered by our tests, it should be working. For (4), ask the Narrator team; WPF and your app are doing the right thing.

To add tracing to a binding: Value="{Binding Path=ValueForPositionInset, PresentationTraceSources.TraceLevel=High}"
This will log the binding's activity to the debugger output window. Look for "using final value '3'"; if that doesn't appear, the trace should suggest what went wrong.

BTW, WPF will figure out PositionInSet and SizeOfSet automatically (in 4.8). You can override the automatic values, as you've done here, but most people don't.

[wilsonwu360]

-adding comments posted at ADO here-

Trishala Patne commented Yesterday

@Ankur Garg @Chase Coburn @Prasanti Patnaik
Hello Team,
As per discussion on https://github.com/dotnet/wpf/issues/3729, team has recommended to try debugging application as per last comment on bug.
We tried all the recommendations for debugging. I can see that data is being binded correctly for sizeInSet and positionInSet as per following debugger output. I don't see any errors in assessibility tool as well as in ouptut window. Last option from recommendation, is reach out to narrator team that why it is not reading data correctly and it is not wpf issue. Please help us to reach out to narrator team.

System.Windows.Data Warning: 56 : Created BindingExpression (hash=34068253) for Binding (hash=19737894)
System.Windows.Data Warning: 58 : Path: 'ValueForPositionInset'
System.Windows.Data Warning: 60 : BindingExpression (hash=34068253): Default mode resolved to OneWay
System.Windows.Data Warning: 61 : BindingExpression (hash=34068253): Default update trigger resolved to PropertyChanged
System.Windows.Data Warning: 62 : BindingExpression (hash=34068253): Attach to System.Windows.Controls.TreeViewItem.PositionInSet (hash=38178826)
System.Windows.Data Warning: 67 : BindingExpression (hash=34068253): Resolving source
System.Windows.Data Warning: 70 : BindingExpression (hash=34068253): Found data context element: TreeViewItem (hash=38178826) (OK)
System.Windows.Data Warning: 78 : BindingExpression (hash=34068253): Activate with root item MSRXAvailableFieldFolder (hash=50513961)
System.Windows.Data Warning: 107 : BindingExpression (hash=34068253): At level 0 using cached accessor for MSRXAvailableFieldFolder.ValueForPositionInset: RuntimePropertyInfo(ValueForPositionInset)
System.Windows.Data Warning: 104 : BindingExpression (hash=34068253): Replace item at level 0 with MSRXAvailableFieldFolder (hash=50513961), using accessor RuntimePropertyInfo(ValueForPositionInset)
System.Windows.Data Warning: 101 : BindingExpression (hash=34068253): GetValue at level 0 from MSRXAvailableFieldFolder (hash=50513961) using RuntimePropertyInfo(ValueForPositionInset): '1'
System.Windows.Data Warning: 80 : BindingExpression (hash=34068253): TransferValue - got raw value '1'
System.Windows.Data Warning: 89 : BindingExpression (hash=34068253): TransferValue - using final value '1'