Model-viewer: Implement better heuristic to enable user interactions than basic focus

[cdata]

As of #288 , we began blocking user interaction unless a <model-viewer> has focus. This enables users to interact with the page in more natural ways (such as scrolling while passing over <model-viewer>) at the cost of requiring explicit interaction to focus <model-viewer> before the controls would work.

@mrdoob immediately pointed out that this was not very intuitive when interacting, especially on mobile. @yuinchien also pointed out the same, and has prototyped some potential alternative heuristics to decide if <model-viewer> should be interactive.

This issue covers discussion and work necessary to enable improved detection of explicit user interactions with <model-viewer> so that it feels intuitive without disrupting other activities such as scrolling the page.

[yuinchien]

I played with horizontal gesture detection on , as @cdata pointed out, most of the mobile page scroll is vertical, allowing auto-focus to happen when detecting user interaction feels a lot more accessible.

const modelViewer = $('#demo-1');
let startX;
let startY;
let isFocused = false;
const THRESHOLD_X = 30;

function onTouchStart(e) {
  let touchobj = e.changedTouches[0];
  startX = touchobj.pageX;
  startY = touchobj.pageY;
}

function onTouchMove(e) {
  let touchobj = e.changedTouches[0];
  if( Math.abs(touchobj.pageX-startX)>THRESHOLD_X && !isFocused) {
      modelViewer.focus();
  }
}

function onFocus() {
  isFocused = true;
}
function onBlur() {
  isFocused = false;
}

modelViewer.addEventListener("focus", onFocus);
modelViewer.addEventListener("blur", onBlur);
modelViewer.addEventListener("touchstart", onTouchStart, false);
modelViewer.addEventListener("touchmove", onTouchMove, false);

[elalish]

On a related note, we also have the inverse problem: when the element covers the whole screen on mobile (either because it's big, or because the user zoomed the page on it before it got focus), then once it gets focus it is impossible to scroll the page, because all touch events are routed to rotating and zooming the model and there is no way to defocus.

[elalish]

One possible solution would be to route touch events to the model regardless of focus, but to reroute those events back to the page when the model limits are hit (high/low elevation, near/far zoom) so that the page can still be scrolled and pinched. The downside is that it would have a bit of sudden mode shift, but it seems fairly consistent with the way scrolling works on pages with scrollable sub-elements. What does our resident expert think @yuinchien?

[cdata]

Thanks for continuing the discussion here @elalish

Indeed, we have discussed what you are describing as _yet another_ possible interaction mode. We might want to support all-of-the-above, and let users decide what works best for them.

I want to also point out that this issue is related: https://github.com/GoogleWebComponents/model-viewer/issues/500

Internally, SmoothControls already supports an interaction mode that does not require focus. This emulates the default (and only) interaction mode of OrbitControls, which swallows input events regardless of focus state. #500 asks that we simply make which mode is used externally configurable. Even if we did this, it would probably only be a half measure because it brings back all the old problems we had with OrbitControls (primarily: that it makes scrolling down a page really hard on mobile).

[elalish]

This also seems to be related to #284, so we might want to find a solution that solves that as well.

[cdata]

Unfortunately, I do not think that #284 is related here. The problem in #284 is that the canvas is obscuring a little bit of built-in browser UI. The only way to fix that is to allow UI events to completely pass-through the canvas, which implies that we cannot observe them directly from said canvas.

All that to say that the fix for #284 might be more along the lines of changing the target element that is listening to events for the purposes of manipulating the camera.