Sway: Implement drawing tablet protocol

[ddevault]

This was merged into wlroots.

[emersion]

Probably need to implement the fallback for clients not supporting the protocol first.

[ghost]

How is this feature going?
I cloud totally drop off i3wm, once pen pressure is working :)

[Emantor]

Pull requests are always welcome.

[ronchaine]

What would it require to implement this? I am pretty swamped by work but I might have time to start looking at this during the weekend.

I haven't touched sway code yet except for building it, so might be a bit rough ride though.

[ddevault]

Take a look at how it's implemented in rootston & wlroots, then add something similar to sway. Shouldn't be terribly complicated, I think.

[lehmanju]

I would totally implement this, input/cursor.c looks like the relevant file. handle_tool_axis just lacks testing against pressure values!?

[jchv]

It looks like this may have stalled. I would like this feature so I am going to attempt to implement it. Looks like I need to:

• Create a new set of files, include/sway/input/tablet.h and sway/input/tablet.c:
  
  
  
  • sway_tablet, sway_tablet_pad and sway_tablet_tool structures mirroring the wl_roots tablet v2 structures for handling events, lifecycle
  
  
  • sway_tablet_configure, sway_tablet_pad_configure functions for initializing the sway_tablet and sway_tablet_pad structures (sway_tablet_tool would be initialized on handle_tool_proximity like it was in rootston. This seems a bit asymmetrical, but given the nature of drawing tablet tools, that literally don't logically exist until they come into proximity, it makes sense to me.) These would also setup event handlers, most of which would probably be implemented in tablet.c as well.
  
  
  • I assume most of the cursor functionality, including positioning, pressure, tilt, etc. belongs in cursor.c, as it is today. We just need to also plumb through to wlr_tablet_v2_* for many attributes.
  
  
• Add struct sway_tablet*, struct sway_tablet_pad*, struct sway_tablet_tool* fields to sway_seat_device.
  
  
  
  • It looks like wlr_input_device.data points to sway_input_device, which _seems_ like an inappropriate place in the abstraction layer to jam fields like this; but since the data field is taken, we can't use it for the tablet structures like rootston did. So, it seems, similar to sway_keyboard, the right place to put it is the sway seat device structure. I hope I'm understanding this correctly.
  
  
• Change seat_configure_tablet_tool implementation to use sway_tablet_configure to create a sway_tablet and store it on sway_seat_device.
• Implement WLR_INPUT_DEVICE_TABLET_PAD for seat_configure_device, calling seat_configure_tablet_pad, which calls sway_tablet_pad_configure`.
• Implement forwarding of input events back through wlr_tablet_v2_* functions.

Thoughts:

• The handling of cursor positioning is a little nebulous to me here; I can see that the logic today works and handles mapping to regions properly, I need this functionality personally so I will try to keep that working. But I'm not sure what coordinate spaces we're dealing with.
• Trying to follow the hierarchy for the tablet API, through v1 and v2, has felt fairly tricky. I'm still not sure if I have a proper basic understanding after reading through code for about an hour or two. So I may be really off with how this needs to be designed. (I hope not.)
• I know fallback events are needed. Can someone explain what would need to happen? It looks like tablet events work _today_ as mouse events. Is it just a matter of keeping that working, or is it more involved?

This is my very first time digging into the code so I really know absolutely nothing. I'm on Freenode now (as jchw) if anyone wants to discuss this in realtime.

[jchv]

Small update: right now having a bit of challenges working out the organization of devices. If you put the sway_tablet in the sway_seat_device, it is not possible to access from within cursor code where it is unfortunately needed. This leads me to believe a pointer to sway_tablet is needed from sway_tablet_tool, and sway_tablet_tool should be stored in wlr tool->data so it can be accessed from events. However, since the tablet tool is discovered by the proximity event, it, too, lives in cursor, which leads to a challenge trying to get the tablet device for a given tool.

I think one of two things needs to happen to make this work:

• Somehow, tool discovery/configuration needs to happen within seat.c where seat_get_device is accessible. But since it is a cursor event, it will likely need to be called from cursor.
• A linked list of tablets needs to be stored on the cursor or seat. I think this is redundant though because there’s already a linked list of seat devices. So I think this is a no-go.

Also still figuring out how to properly debug this code. If I get anything working at all I will post the preliminary code, for now nothing.

[jchv]

OK, I've hit a milestone, I am now getting proper pressure sensitivity in GIMP and buggy pressure in Krita (need to debug.) I don't know of any native Wayland programs that support pressure, and also my tablet doesn't support tilt, so there is likely work to be done. It's still a little too early to really need much testing, and the code desperately needs to be cleaned up, but please feel free to check it out at https://github.com/jchv/sway/tree/drawing-tablet-support, compile it, test it, etc.

To get things working in GIMP, you will need to put xwayland-stylus into Screen mode, and of course enable brush dynamics that use your stylus's pressure or tilt capabilities. Krita works out of the box, but has some trouble triggering in some cases.

I also have yet to implement the fallback events, so you will find some situations where the stylus will not work, probably mostly with native Wayland clients. It doesn't look like it will be hard to get fallback events working, I think I just need to take the case where the surface doesn't support the tablet API and do what the code used to do.

[ysooqe]

@jchv Thank you for implementing this!

I don't know of any native Wayland programs that support pressure

There is Xournalpp which uses GTK3 and therefore should run natively under Wayland and it supports pressure sensitivity, so maybe thats a program you are looking for.

One question though, does this implementation also include things like support for the buttons of the pen/tablet?

[jchv]

@ysooqe

One question though, does this implementation also include things like support for the buttons of the pen/tablet?

This is something that is important but probably not going to work yet. I suspect this is handled in the Tablet Pad even if your pen has buttons on it (like Wacom.) On that note, I haven’t even yet checked to see if erase works :)

I may have some difficulty testing tablet pad code as it seems my tablet pad gets rejected due to a libinput bug, but I’ll try to remedy that soon.

This should all be in the PR when it is ready.

[jchv]

Briefly stalled to work on https://github.com/NixOS/nixpkgs/issues/52490, which is preventing my Wacom tablet from working properly in libinput.