Vega-lite: Support for touch and pinch in interactions

[mattijn]

Currently there is only support for mouse click, mouse move and scroll for vega-lite declared interactions.

It would be great if this was extended with touch touchmove and pinch.

[domoritz]

How difficult is this and do we need anything else in Vega @arvind?

[kanitw]

I think this is already possible. Perhaps, someone should provide an example and we can close this.

[mattijn]

This is surely working in Vega, https://vega.github.io/vega/examples/zoomable-scatter-plot/.
It is just not implemented in Vega-lite.

[domoritz]

https://github.com/uwdata/falcon also works on mobile.

[domoritz]

This issue is about seeing whether it's possible to manually customize the events in VL to capture touch as well. #4661 is about supporting this by default.

[mattijn]

At this falcon website it seems the anchorpoint is snapping to the left or right edge handlers of the interval.

[domoritz]

At this falcon website it seems the anchorpoint is snapping to the left or right edge handlers of the interval.

Can you elaborate?

[mattijn]

[mattijn]

Touch works indeed, I can't get the pinch to work.

Example for creating interval selection box (rectangle will be orange if using touch events):
Click on mobile

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "data": {"url": "data/cars.json"},
  "selection": {
    "MOUSE": {
      "type": "interval",
      "on": "[mousedown, mouseup] > mousemove",
      "translate": "[mousedown, mouseup] > mousemove"
    },
    "TOUCH": {
      "type": "interval",
      "on": "[touchstart, touchend] > touchmove",
      "translate": "[touchstart, touchend] > touchmove",
      "mark": {"fill": "#fdbb84", "fillOpacity": 0.5, "stroke": "#e34a33"}
    }
  },
  "mark": "rect",
  "encoding": {
    "y": {"field": "Origin", "type": "ordinal"},
    "x": {"field": "Cylinders", "type": "ordinal"},
    "color": {
      "condition": {
        "selection": {"or": ["MOUSE", "TOUCH"]},
        "aggregate": "count",
        "type": "quantitative"
      },
      "value": "grey"
    }
  }
}

And an example where one can pan when two touch events are captured. So the panning is only captured when you touch with one finger somewhere outside the chart and pan the chart with another finger:
Click on mobile

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "data": {"url": "data/cars.json"},
  "selection": {
    "MOUSE": {
      "type": "interval", "bind": "scales",    
      "translate": "[mousedown, window:mouseup] > window:mousemove!"
    },
    "TOUCH": {
      "type": "interval", "bind": "scales",
      "translate": "[touchstart[event.touches.length===2], touchend] > touchmove"
    }    
  },
  "mark": "circle",
  "encoding": {
    "x": {
      "field": "Horsepower", "type": "quantitative",
      "scale": {"domain": [75, 150]}
    },
    "y": {
      "field": "Miles_per_Gallon", "type": "quantitative",
      "scale": {"domain": [20, 40]}
    },
    "size": {"field": "Cylinders", "type": "quantitative"}
  }
}

For the last only the TOUCH selection is captured, not in combination with a MOUSE selection.

I can't get the pinching to work within zoom since I can't find a method to access the pinchDistance expression

Also when looking to the underlying created Vega, it doesn't combine the mouse and touch events within a single on directive, but it rather creates two different code blocks.

[domoritz]

Thanks for the video! I filed https://github.com/uwdata/falcon/issues/123.

I guess we can close this issue here then and discuss how we can support this natively in #4661.