Viewers: Unify mouse/key/touch bindings between cornerstone and vtk.js

[pieper]

Request

What feature or change would you like to see made?

Probably vtk.js interactor should match cornerstone conventions. Or we should define and OHIF standard and apply it to both cornerstone and vtk.js. Could also look at corresponding interaction styles in Slicer.

Why should we prioritize this feature?

Currently going between 2D and MPR view is confusing due to the different bindings.

[salimkanoun]

Yes !
Is there a hope that the 2 displays get the same "features" with the same tools in MPR than the main view : same zoom, pan interaction, Roi and measurement ?

[pieper]

There's definitely some work in this direction - vtk.js has some svg overlay support under development.

[dannyrb]

Do you have a hit list of the most jarring differences, or what you would like to see support for? We could unify things a bit:

• Default to WWWC tool as left click
  
  
  
  • Verify that viewports level similarly depending on mouse direction + delta
  
  
• Set pan as middle click (currently shift click)
• Update zoom/scale strategies to match

We could add a stack scrollbar if it still makes sense.

I haven't played with VTK.js's touch bindings enough to know how well they align with what we currently have. I'd likely want to prioritize how the Viewer looks at display resolutions that more commonly support touch first.

[pieper]

I believe you've hit the main ones. Default left click wc/ww, and middle-mouse pan were the two that jumped out at me as odd. @swederik suggested filing this as an issue so he probably has others that annoy him 😜

[salimkanoun]

just a suggestion about controls,

I some software i very like having the middle mouse button to have a "software scroll". You click the middle button, keep it pressed and then go up and down and this trigger a stack scroll, more the go up or down more the speed increase (like a dynamic ciné).
It gives you a smoother scrool and prevent from repetitive wheel turns when you have a to read a stack of 1000+ slices.

Did you try such a control ? what's your though about it ?

Is this workstation the zoom was at right click and the pan was wheel + right click together.

It don't know if there a plan maybe to constitute a panel of users and have ideas / feedback of controls.
I think there is no idea gold standard for user interaction but maybe putting our brain together will lead to find one ?

Best regards,
Salim

[pieper]

A mode that works really nicely for MPR viewing is to access the crosshair using the a key, such as shift (this is what we do in Slicer). It's especially useful to link the zooms across the MPR views and to pan the crosshair point to the center.

Agreed though that there's no gold standard - we can try to do our best to combine the best examples we've seen.

One comment I've heard from radiologists is that they don't like having to hold down mouse buttons while moving the mouse because it leads to strain if they are doing it all day every day.

[dannyrb]

You click the middle button, keep it pressed and then go up and down and this trigger a stack scroll, more the go up or down more the speed increase (like a dynamic ciné).

That's interesting. I do think that's a better user experience.

A mode that works really nicely for MPR viewing is to access the crosshair using the a key, such as shift (this is what we do in Slicer). It's especially useful to link the zooms across the MPR views and to pan the crosshair point to the center.

I really like that as well!

Agreed though that there's no gold standard

Popular editors will sometimes expose the ability to change all interactions/hotkeys to match the software you may be transitioning from. For example, VS Code supports "Sublime Text", "Web Storm", etc.

Having the ability to choose from popular defaults like "Osirix" or "Slicer" would make transitioning easier, and open the door for easier sharing/customization of preference sets? (a bit of a digression, but if we try to define "the gold standard", we may be here a while ^_^)

One comment I've heard from radiologists is that they don't like having to hold down mouse buttons while moving the mouse because it leads to strain if they are doing it all day every day.

This isn't something I've heard before, but it makes a lot of sense. The cornerstone-tools support for multiple clicks AND mouse drag now makes more sense.

[salimkanoun]

Execllent idea the transitional settings. I think it's pretty nice if it can be customizable, then the question will focus on what is the best option that should be the default one ?
With tools in hands and active community I think the solution will apear.

I don't have idea what will be the best user interaction but by looking at this discussion I'm pretty confident that we will find it !

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[pieper]

This should be addressed as part of the update to the core off screen rendering functionality. Related to #2238