Iced: Implement accessibility support

[twitchyliquid64]

Background

Platforms typically mediate an interface between applications with a user interface, and accessibility tools (such as a screen reader), enabling assistive technologies:

• GTK/Gnome - Accessibility Toolkit (ATK)
• Windows - Active Accessibility
• MacOS - NSAccessibility protocol/messaging

Iced should provide the building blocks for making accessible applications, and integrate them automatically where possible.

Requirements

• Integrate with native platform APIs for accessibility, and provide reasonable accessibility integration for widgets built-into iced.
  
  
  
  • Ability to communicate the widget hierarchy
  
  
  • Ability to communicate the type, state, and 'properties' of widgets in the hierarchy
  
  
  • Ability to query for a widget at a specific location.
  
  
  • Ability to provide updates of changes to widgets, or their location in the hierarchy
  
  
• User-code should not need to use platform-specific crates.
• User widgets should be able to provide additional information to accessibility APIs.
  
  
  
  • Additional or better markup/attributes
  
  
  • 'Navigations' or actions
  
  
  • Indicate relationships to logically-related widgets
  
  

Ideas

• Automatic integration
  
  
  
  • [ ] Track widgets in a nested hierarchy
  
  
  • [ ] For each widget in the hierarchy, automatically determining basic information that can be communicated to accessibility APIs (ie: whether its a button/label/input etc, button/label text, focus state).
  
  
  • [ ] Communicate a representation of the hierarchy, basic widget information, and mutations to the platform-specific accessibility interface. This implementation could be shared with layout debugging and unit-testing facilities.
  
  
  • [ ] Implement hit-testing.
  
  
• Widget-specific extensions
  
  
  
  • [ ] Allow widgets to provide additional markup or attributes to an accessibility API
  
  
  • [ ] Allow widgets to indicate actions/navigation options to an accessibility API (this could be implemented at a higher level as messages?)
  
  
  • [ ] Allow widgets to indicate relationships to logically-related widgets
  
  

Imho, this means we first need to implement:

• A widget hierarchy that lives longer than layout/draw iterations
• A stable identity for a widget in the hierarchy.

Related issues

https://github.com/hecrj/iced/issues/282

[myfreeweb]

GTK/Gnome - Accessibility Toolkit (ATK)

Note: GTK4 is now using AT-SPI directly, because ATK is a barely maintained thing that only adds unnecessary indirection.