Vscode: Terminal output div container should be more accessible for screen readers

[jooyoungseo]

CC @isidorn

System Info

• Version: 1.46.0-insider (system setup)
• Commit: cd41dd1f6065db76c77c6e2f659bb50e6f558dec
• Date: 2020-05-29T05:51:32.662Z
• Electron: 7.3.0
• Chrome: 78.0.3904.130
• Node.js: 12.8.1
• V8: 7.8.279.23-electron.0
• OS: Windows_NT x64 10.0.18363

Steps to Reproduce:

1. With either NVDA or JAWS loaded, type any commands in terminal (Ctrl+`) input area.
2. Press Shift+Tab two times from the terminal input to focus in terminal output list view.
3. The terminal output list is reachable via tab key thanks to tabindex = "0" attribute; however, screen readers (both NVDA and JAWS) do not read anything with arrow keys unless users manually switch on browse mode.
4. The additional problem can be found with very long terminal outputs that require scroll behavior. Screen reader users have to switch back to forms mode to arrow up/down to scroll through the output history (while not being read any current line) and again, turn back to browse mode to check the results.

• Does this issue occur when all extensions are disabled?: Yes

Suggested Solutions

Currently, the terminal output <div> container denoted with class = "xterm-accessibility-tree" has role = "list".

As a blind developer myself, I am requesting the following for accessibility improvements for screen readers.

1. Please change the value of role attribute from "list" to "document" so that screen reader users can navigate the terminal output via arrow keys using browse mode.
2. Please adjust the tabindex value of this terminal output container in the way that screen reader users can quickly Shift+Tab from terminal input area. In other words, pressing tab key from terminal output area should go to terminal input area; pressing Shift+Tab key from terminal input goes to terminal output area denoted with role = "document" div.
3. Ideally, it would be great if the terminal output can be also included in the F6-keycycle.

[isidorn]

@jooyoungseo thanks for feedback.

1. I believe @Tyriar decided to use the role list because the terminal consistes of multiple lines with each being a listitem. We can experiment changing to document but let's also try to get feedback from the community
2. This makes sense, however it might be controlled by the shell, @Tyriar will know more
3. It is included in the F6 keycicle. Can you provide more details what is not working here?

fyi @marcozehe for role smartness

[MarcoZehe]

If you use role list and listitem, meaning static roles, you need a container that is a role document. Otherwise, screen readers will not read these lists properly. Since they are static text, that is appropriate to do. So the list and list item are appropriate, but the container must be a role document, too.

[isidorn]

@MarcoZehe makes sense
@Tyriar we could put the role: document on the terminal xterm div element which is an ancestor of the list role and it should be good.

[chrisnorman7]

Hi,
The entry part of the terminal is included in the F6 cycle, but not the output.

[isidorn]

@chrisnorman7 aha. Then we would not change the F6 behavior, since the point of F6 is to give a birds view navigation over the workbench components. Once you are close to something you want you should tab into it. Since moving from the entry of the terminal to the output should be easy (if it is not we should fix) I would not change F6 behavior.

[jvesouza]

In my opinion, the ideal is that we could navigate in the terminal output using the arrows to read by character, line and word.

[alex19EP]

@jooyoungseo did you know about control + arrow keys to read terminal lines. if so, then I do not quite understand why you propose these changes.

[MarcoZehe]

The problem with ctrl+Arrow keys is that this will just dump a line into the synthesizer and be done with it. No review, and more importantly, no braille support. So making this output actually navigable with screen reader reading commands is a very welcome enhancement for both speech and braille users.

[alex19EP]

The problem with ctrl+Arrow keys is that this will just dump a line into the synthesizer and be done with it. No review, and more importantly, no braille support. So making this output actually navigable with screen reader reading commands is a very welcome enhancement for both speech and braille users.

ok. now i understand. thanks.

[chrisnorman7]

Honestly, it could be easier.

When I press shift tab from the terminal's input, I hear "Section", then
when I hit it again I get into the output. Same for tabbing from the output
to the input. Whatever "section" is, it sits in between input and output.

Take care,

Chris Norman

On Tue, 2 Jun 2020 at 11:27, Isidor Nikolic [e-mail]
wrote:

@chrisnorman7 https://github.com/chrisnorman7 aha. Then we would not
change the F6 behavior, since the point of F6 is to give a birds view
navigation over the workbench components. Once you are close to something
you want you should tab into it. Since moving from the entry of the
terminal to the output should be easy (if it is not we should fix) I would
not change F6 behavior.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/98918#issuecomment-637446058,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBUZ2JCSLVJAICATZAJ4JDRUTHZHANCNFSM4NPJDSKA
.

[jooyoungseo]

@isidorn, the idea I would like to propose for either F6 cycle or tabindex adjustment is about a more easier and screen-reader-friendly approach.

Let me give you an example. I am a Python programmer who has to frequently switch between code editor pane and terminal output.

When I execute a code block from code editor into terminal, I have to do the following:

1. Press 'Ctrl+`' to focus in terminal input.
2. Pres 'Shift+Tab' key twice to focus in terminal output.

Wouldn't there be a more efficient way to quickly switch between code editor and terminal output?

[isidorn]

All good feedback,
Though let's wait for @Tyriar to hear his thoughts.

@jooyoungseo you would basically like a command that gives focus directly to the terminal output

[chrisnorman7]

Unless the thing which sits between the input and the output, that NVDA
simply reads as "section" is important, why not remove its tabindex
attribute?

Take care,

Chris Norman

On Tue, 2 Jun 2020 at 16:21, Isidor Nikolic [e-mail]
wrote:

All good feedback,
Though let's wait for @Tyriar https://github.com/Tyriar to hear his
thoughts.

@jooyoungseo https://github.com/jooyoungseo you would basically like a
command that gives focus directly to the terminal output

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/98918#issuecomment-637611577,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBUZ2ORYEP7XR42IY7DMZTRUUKHNANCNFSM4NPJDSKA
.

[jooyoungseo]

Unless the thing which sits between the input and the output, that NVDA simply reads as "section" is important, why not remove its tabindex attribute? Take care, Chris Norman

Yes, that's exactly what I want so that we can press shift+tab just one time from terminal input to output area, and vice versa.

[Tyriar]

The problem with ctrl+Arrow keys is that this will just dump a line into the synthesizer and be done with it. No review, and more importantly, no braille support. So making this output actually navigable with screen reader reading commands is a very welcome enhancement for both speech and braille users.

why not remove its tabindex attribute?

The ctrl+arrow 'navigation mode' was built with help from JAWS and the main reason it exists is because the list is and must be virtual. As I understand it, navigation with a screen reader is not possible using a virtual list as we need to update the contents of the DOM during navigation and we don't get events to trigger when a screen reader is being navigated.

The buffer must be virtual for the same reasons as the editor, we cannot create 1000+ lines of dom content, easy with their own set of spans as browsers don't like that and VS Code would lock up whenever you press enter (triggering a layout after a row changes).

you would basically like a command that gives focus directly to the terminal output

Navigation mode is that command; ctrl+up and ctrl+down.

[isidorn]

@Tyriar ok, but it still holds that we can add a role 'document' to the parent container.

[Tyriar]

If that will improve the experience that's easy 👍

[chrisnorman7]

Wow, thanks for working so hard on this all. That's great.

[isidorn]

Yeah it should improve the experience.
As for the other focus discussions I am not sure what action items we can extract out of those. It would be great if we found something and put it in a concise new issue.

[derekriemer]

We might well need to add buttons to the document to allow the user to scroll the terminal up/down. I don't know that Orca, jaws and NVDA will properly scroll the terminal in such a way that you guys can dynamically update the DOM.

[jooyoungseo]

Adding scroll buttons would be possible. However, in my opinion, the "document" role would suffice because it will then be treated as a read-only multi-line text box where you can navigate the terminal output from top all the way to the bottom regardless of its length. Please correct me if I am wrong.

[jooyoungseo]

@Tyriar, I hope you don't mind me asking this again. Could we expect to experience this improvement in July 2020 milestone? I am asking this because see the June and July milestone label up there. I am very looking forward to this improvement.

[Tyriar]

@jooyoungseo fixed 🙂

[jooyoungseo]

@Tyriar, after testing the latest commit (15ada625f20086007e2c4aa0d760234360cd648f), I have found that the implemented "document" role has not not made any navigation improvements for both NVDA and JAWS.

@MarcoZehe, would you mind helping us addressing this issue?

I have tested with the following code-insiders:

Version: 1.48.0-insider (system setup)
Commit: 15ada625f20086007e2c4aa0d760234360cd648f
Date: 2020-07-30T14:43:12.691Z
Electron: 9.1.0
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.19041

[MarcoZehe]

The document role is on a div with class xterm-accessibility-tree. Keyboard focus, when shift-tabbing from the terminal input twice, lands on the outer div with classes 'terminal xterm'. The document role must be on something that is either the focused element itself, so the latter div, or something outside that, but not below it in the hierarchy. In essence, the role is on the wrong div.

[jooyoungseo]

Thanks very much for your advice @MarcoZehe!

@Tyriar, would you mind adjusting the position of "document" role following @MarcoZehe's advice so that we can get the real improvement?

Appreciatively,

[isidorn]

Thanks for trying this out and for letting us know.
@Tyriar let me reopen this so we try to address feedback from @jooyoungseo and @marcozehe

[Tyriar]

Oh? I actually verified this one and was able to use browse mode to navigate the list items by use ins+space and then shift+i, am I doing something wrong?

[MarcoZehe]

You should not need to use Insert+Space first. If done right, tabbing into this area would then turn document mode on automatically, tab out, and it returns to application mode.

[Tyriar]

@MarcoZehe wouldn't browse mode break terminal input if it's switched to automatically when focusing the terminal?

[MarcoZehe]

Not if you put it where tab stops land, as I indicate above. It is just two containers up from where the role is now. The entry field is outside that container. Browse mode would be applied to all descendants of that xtrm thing, but not to the input, which lives outside. So the moment focus leaves this particular area, NVDA will return to application mode.

[Tyriar]

@MarcoZehe when I change it to be 2 containers above I land on the terminal and it says "document, terminal input edit" (because the textarea is under .terminal.xterm), the input still seems to work fine but i and shift+i input instead of navigating? I still need to enter browse mode with shift+ins manually?

[Tyriar]

If I tab into the terminal it says "document", and pressing i or shift+i just says "i" without doing anything?

[MarcoZehe]

Hm, when I open a terminal, I am in the edit field. I then shift-tab twice (the first stop is for what I don't know). And that is where NVDA should then go into browse mode. So if putting the role on the xterm div doesn't work because it does include the input, maybe move that tabindex="0" down to where the role="document" is instead. Does that work?

[Tyriar]

the first stop is for what I don't know

That is for this: <div tabindex="0" class="focus-trap" aria-hidden="true"></div>, the purpose of that is to move keyboard focus to the hide panel button and skip everything else but there was a bug causing that not to happen. But anyway, that would break what we're trying to go for here where shift+tab allows us to navigate the terminal so I'll remove it. It's also a bit ugly as it depends on both xterm.js internals and panel internals (the reason it broke).

maybe move that tabindex="0" down to where the role="document" is instead

Having both tabindex="0" and role="document" on the terminal xterm element just ends up saying "document", i and shift+i end up just saying "i" so I'm not seeing this automatic switching to browse mode in NVDA. I could add an aria-label there to say something like "Enter browse mode to navigate terminal rows, document"?

[MarcoZehe]

That surprises me, I would have thought the moving of the tabindex attribute to the same element that contains role="document" would work. It usually does.

[jooyoungseo]

@Tyriar,

In the latest version of VSCode insider, the terminal output aria-live event is not automatically read by JAWS (v2020.2006.12.400).

Version: 1.48.0-insider (system setup)
Commit: 2277c8e2a3e1cc630a6397301ba54a1dccd8a60d
Date: 2020-08-07T11:32:09.993Z
Electron: 7.3.2
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Windows_NT x64 10.0.19041

[jooyoungseo]

@Tyriar,

It would be still ideal for screen reader users to not manually switch between browse mode and focus mode. I think there should be another factor causing the "document" role to not work, which we might want to further investigate.

[Tyriar]

I'll move the document to the terminal xterm element I was talking about so you can verify whether it's better, maybe I was doing it wrong.

[jooyoungseo]

@Tyriar, aria-live event for the terminal output is working again for JAWS (tested with the latest insider commit: bd08768fd310d34cdd4423a53e5f3ca77340fea5).

To make the "docvument" role work, what if we do the following?:

Currently, the <div class = "xterm-accessibility-tree"> has "document" role with no tabindex attribute.

On the other hand, tabindex = "0" is set within the outer <div class = "terminal xterm">.

Please move the tabindex = "0" from the outer to the "xterm-accessibility-tree" container having the "document" role. And, please make sure the outer container not to have any tabindex.

[Tyriar]

Please move the tabindex = "0" from the outer to the "xterm-accessibility-tree" container having the "document" role. And, please make sure the outer container not to have any tabindex.

@jooyoungseo moving the tabindex will change a bunch of other stuff though like outline styles, focus/blur events, etc. Plus .xterm-accessibility-tree doesn't exist when a11y mode is disabled so it would fork the functionality (probably leading to regressions on one of them).

[Tyriar]

PR to move the role to the main .terminal.xterm element as discussed, I'll pull this in and it should be available in tomorrow's Insiders build. If people could verify whether this improves the situation that would be great.

[jooyoungseo]

Thanks very much for your hard work @Tyriar.

In my experience with the latest commit (d1f267742169584db010716d92addf50d4cd8139), neither JAWS nor NVDA auto-switches between browse and document mode in the terminal input and output.

What do you think @MarcoZehe and @zersiax? Would you mind you all testing the latest commit for this feature?

[jvesouza]

using orca, when I'm at the terminal and pressing shift + tab, orca just announces that I'm in a document but doesn't switch to browse mode.
Nothing else is read, even if I try to navigate with the arrows.