Nvda: Consult: space/enter keyEvent is swallowed by NVDA

[mo13yu]

I'm working on a project that within a QT application, there is QWidget loads an html. And I need when using NVDA tab navigate to a button inside of the html, if user presses space/enter, not only the button's onClick handler should be triggered, but also the parent widget should handle the keyPressEvent. However I observed the parent widget doesn't get the keyPressEvent.

I've already find out that is because browse mode vs. focus mode are automatically toggled when on web page. Here(https://github.com/nvaccess/nvda/issues/4903) says: "when in browse mode, the enter key is overridden to activate the control using mouse clicks." Could someone show me which part of code is? (seems my parent widget doesn't get mouseEvent either)

I did try manually toggle to focus mode everything works fine. And I also read this from user guide: "NVDA will automatically switch to focus mode if you tab to or click on a particular control that requires it."

My questions are:

1. Is there any programmatic way that I can specify this html button requires focus mode?
2. Is there any programmatic way to toggle focus mode/browse mode in code?
3. What are the html controls that requires focus mode?
4. Is there any way to check what mode is NVDA currently on? (As they are auto-toggled, so I need a way to specify)

Thanks!

[leonardder]

This is a very interesting issue. First of all, it could be good to know how QT renders this HTML (i.e. does it use MSHTML, GECKO or Chromium?). Could you provide developer info (insert+f1) after focusing the particular button?

I wonder, couldn't you make the parent widget react on an event that is triggered by the onclick?

Is there any way to check what mode is NVDA currently on?

You can distinguish the mode by the sound played when it activates. In NVDA's browse mode settings, you can disable Audio indication of focus and browse modes, in which case a switch will be spoken by speech.

[jcsteh]

@mo13yu commented on Jan 6, 2018, 12:43 PM GMT+10:

seems my parent widget doesn't get mouseEvent either

I assume you mean parent Qt widget, not HTML widget? That's tricky. In many cases, NVDA will dispatch a click event using the accessibility API, which will be handled internally by the browser engine. So, it won't be intercepted by Windows (and thus probably not passed to Qt). I guess you'd need to overide the click event in HTML to somehow notify the Qt widget yourself. Or is there some way to monitor web DOM events from the Qt widget?

1. Is there any programmatic way that I can specify this html button requires focus mode?

No. Buttons should never require focus mode from a user perspective. Only controls that need keyboard interaction beyond clicking (e.g. text boxes, sliders, list boxes) cause a switch to focus mode.

1. Is there any programmatic way to toggle focus mode/browse mode in code?

No. You could use role="application" to forceably disable use of browse mode, but that would mean the user wouldn't be able to use browse mode.

1. What are the html controls that requires focus mode?

See above.

1. Is there any way to check what mode is NVDA currently on? (As they are auto-toggled, so I need a way to specify)

No. We can't expose any API to web pages.

[vadimhp]

it's necessary for the web developer to know whether an element was activated using a mouse click or a keyboard's Enter/Space key, even in Browse Mode

[danzen]

We need Browser window key events to fire when NVDA is in browse mode as well. Our canvas framework at https://zimjs.com needs to deal with its own enter events. Currently, it works well in Form mode or if a shift or ctrl ENTER is used. But that is not ideal. If you could indicate the likelihood of this being adjusted that would be great. We have an upcoming government test... Thanks.

[Adriani90]

cc: @feerrenrut

[feerrenrut]

it's necessary for the web developer to know whether an element was activated using a mouse click or a keyboard's Enter/Space key, even in Browse Mode

@vadimhp Why is it necessary? What is the semantic difference for a user activating an element via mouse click or via keyboard? How would you like to expose that semantic difference to an end user?

If you could indicate the likelihood of this being adjusted that would be great.

@danzen Very unlikely at the moment. Please provide more information about what you are trying to achieve.

[danzen]

Thanks for the response, @feerrenrut. We are a Canvas framework and so click events are put on our canvas objects to handle click interactivity. HTML tags are then placed behind the canvas in the same place as the canvas objects. When a user hits enter we capture the enter key being pressed and call the same function that the click events call. If we can't capture a key event in our framework, we can't do this. Any advice would be helpful.

[feerrenrut]

@danzen perhaps you can provide a link to an example, I had a look at the examples on the website linked earlier but could not focus any elements in the canvas with NVDA. If the goal is to mimic a mouse click, then the standard clickable activation should work, but I might be misunderstanding the situation.

[danzen]

Hi @feerrenrut - here is what we did a while ago. We have made a few changes since but mostly to work with Adobe Animate. https://zimjs.com/accessibility/ - sorry, should have posted a more specific link earlier.

[thomasrye]

I wanted to chime in here. #8317 was much more of a direct example of this in my opinion, but it has been merged with this one.

While we wait for some kind of resolution, does anyone know what experience NVDA users do when they come across a button that does not seem to respond to Enter or Space? Are they just used to immediately toggling to Forms mode or trying Ctrl+Enter to make it work and they move on with their task or does it create a lot of confusion for the user?

Just for further input, our particular case involves a <button> that gets it's click event from angular/typescript. The browser (Firefox) recognizes that there is an event on the button, but apparently it is not enough for NVDA to recognize and allow Enter/space alone to active it.

[XLTechie]

Thomas Rye wrote:

While we wait for some kind of resolution, does anyone know what experience NVDA users do when they come across a button that does not seem to respond to
Enter or Space? Are they just used to immediately toggling to Forms mode or trying Ctrl+Enter to make it work and they move on with their task or does it
create a lot of confusion for the user?

It can introduce quite a level of confusion, simply because the user doesn't
really know that nothing happened. Maybe some sort of HTML menu opened or
expanded, that NVDA doesn't speak natively. Maybe something happened elsewhere
on the page, and NVDA doesn't know it.

Take many of the controls on https://getresponse.com, for example. It's been a
while since I tried to use that system, but after a list is created, many of the
functions are operated by controls that open ajax dialogs or overlays that NVDA
knows nothing about. So the user never really knows that nothing happened.

In other cases, yes: other things can be tried to actuate the control.

[feerrenrut]

@danzen I tried https://zimjs.com/accessibility/radiobuttons.html

So far I'm not seeing a problem with NVDA, though here is some advice for zimjs, or at least the examples I looked at:

• Putting this section in a role=application section should make focus mode enabled by default, keys will be passed through directly, however you will have to do a lot more work to provide a user friendly UX. An alternative would be to actually handle the click, if you add onclick="alert();" to the div with id="myCanvasTab1" and press enter while it has focus (with NVDA running in browse mode) then the alert is triggered.
• The elements are all exposed as role="button" and the "name" is overloaded to include semantic information via aria-label. It would be much more friendly to provide the appropriate semantic information. For instance with the div that displays as radio button A id="myCanvasTab1" (I think the label should be A not eh by the way). This should be role="radio", checked state should be exposed with aria-checked="true", then you don't need to programmatically update the aria label with the semantic information.

See https://www.w3.org/TR/wai-aria/#widget_roles

I wanted to chime in here. #8317 was much more of a direct example of this in my opinion, but it has been merged with this one.

@thomasrye thanks for pointing that out. My gut reaction to that example is that you have a web application that has its own keyboard handling code. In this case role=application MAY be more appropriate, though you are forcing the user into a particular kind of experience (hopefully a good one), there is certainly a lot more to consider if going this route.

The browser (Firefox) recognizes that there is an event on the button, but apparently it is not enough for NVDA to recognize and allow Enter/space alone to active it.

This is interesting, and I haven't investigated. Is there a reason that the button doesn't have an onclick? Adding onclick="alert();" shows that pressing enter on the button creates an alert. That is, the button can be activated with the enter key while NVDA is in browse mode.

It seems I don't understand what it is you are trying to achieve here.

• You want to be able to differentiate between keyboard activations and mouse activations?
• You are limited by some framework constraint?
• Something else?

[thomasrye]

@feerrenrut Thanks for the response. (I'm late getting back to you.)

I think it is possible that there is some framework constraint. We are using angular and some data table component that makes the click handling less direct. I will try to look into that more

[mrMagic-web]

Thanks @feerrenrut. Your suggestion helped in my case. I have rapped entire block in role application and gave clickable elements correct button roles.

[Adriani90]

Is there any question still open to this issue? Or can this be closed?

[danzen]

Hi @Adriani90 - you can close it - and thanks all for the information. I think we fixed things some other way but if it comes up again we can re-open. Cheers.

[feerrenrut]

Thanks @danzen