Makie.jl: Thoughts on transformations

[jkrumbiegel]

Most of this draws from https://matplotlib.org/users/transforms_tutorial.html, they had this stuff figured out pretty well already.

We probably need (at least?) 4 different levels of transformations in Makie:

• Display (actual pixels)
  
  
  
  • allows you to place anything pixel perfect in the scene
  
  
  • allows all kinds of absolute placements or measures that are not supposed to change with scaling of the plot
  
  
  • so everything like font sizes, marker sizes, line thickness, absolute margins and paddings, this all needs to go be specified in the display transform
  
  
  • if unit support is built in, it also works on this level: pt, mm, dip, px, all these things are in the end convertible to a specific pixel number, so can be used interchangeably
  
  
  • allows picking of things that are being clicked on by going down the transform chain from here
  
  
• Figure / Scene
  
  
  
  • if just one scene exists this covers the whole image, with the lower left corner being (0, 0) and the upper right (1, 1))
  
  
  • because in Makie, subscenes are allowed, this would need to be more flexible to allow nesting, but it's still basically the same thing
  
  
  • allows you to place e.g. a title centered at the top if you put it at figure coordinates (0.5, 0.9)
  
  
  • allows placement of subplots in figure relative space
  
  
• Axis
  
  
  
  • for normal rectangular 2D axes these also go from (0, 0) at lower left to (1, 1) at upper right
  
  
  • allows to place text / annotations relative to the axis / subplot
  
  
  • for 3D axes it's probably more complicated because the boundaries can change when rotating the axes..
  
  
  • there are also logarithmic axes, polar axes, things that are a bit more complicated
  
  
  • still it is important for layouting purposes that an axis always separates between its inner part where the actual plot is and the outer part where labels etc are
  
  
  • basically, any kind of container that says "i want to be aligned with other container's important edges at these top, right, bottom and left lines" serves the same purpose and can be included in a layout engine, it doesn't have to be an actual axis
  
  
  • a layout engine would basically, as far as I understand it, and as far as the Rhea prototype works right now, always deal with the actual display level and try to align things in this space. So it would want to set the left and right edges of a figure at let's say 0px and 1000px of a window that is 1000px wide, the first subplot's left edge would be at 20px, etc. So it should be possible to calculate for all axis decorations how far away from the axis frames they reach (the axis area is the part that's flexible in the layout, the decorations don't change size). Let's say the left axis label is 10pt wide and 20pt away from the left axis spine, then the layout engine knows to reserve 30pt space on the left side of the axis and will adjust axis width accordingly.
  
  
• Data
  
  
  
  • this is the level at which the actual data lives, so that you can plot points from 0 to 1,000,0000 in one subplot next to another subplot that goes only from 0.1 to 0.5 but they're both the same size
  
  

Things that should be possible:

• create axis label text of 8pt size with 5mm gap to the axis boundary
• create scatter plot markers that are always 3dip wide with a stroke width of 0.5pt
• create scatter plot markers whose width is meaningful in data coordinates (and therefore increases / decreases in size when the axis limits are changed)
• an axis that spans half of the figure width and height with its main area (excluding labels)
• lines that always span the full (or a percentage of) height or width of an axis, but the other dimension is in data coordinates (axvline, axhline)
  
  
  
  • this is a blended transform, where the x and y parts refer to different reference frames (possible as long as the transforms are separable)
  
  
• zoom into an axis while leaving all text and lines intact, by simply changing how the data transform maps to the axis transform (which is equal to changing the x and y limits of the axis). right now zooming into an axis just magnifies everything, this could be a useful function sometimes but most of the time not what you want.
• more advanced blends: create a text annotation that is always 1/5th axis width + 20pt to the right of a point at (0.3, 0.3) in figure space
• every object in a scene should have a place in this chain (or tree) of transforms so you can calculate any conversion you want by going up and down this chain
• if these transformations / matrix multiplications all happen at the GPU level, the performance is much better
  
  
  
  • vispy does this, e.g., by building nested transforms as functions into GLSL code that is created dynamically through string interpolation. Only the parameters of these functions are then updated on the GPU. Maybe this is easier to realize with Julia's compilation powers?
  
  

Depending on how you think about it, figure and axis could both just be called a container, so you could merge the two and make multiple nested versions possible. This would integrate neatly with arbitrary nested grids from the layout engine.

[asinghvi17]

Thanks a lot, @jkrumbiegel! Just a couple of thoughts:

• As of right now, it wouldn't be too hard to do the axis/figure coordinates (although maybe they would be better off being called Scene coordinates, since Scenes are basically the merged containers you're talking about.

• One thing to be careful of is determining which container the relative coordinate is relative to, since it's Scenes all the way down.

[yakir12]

Just chiming in here, rotation of the camera in a zoomed-in scene should rotate around the new center of the data, not the center of the data before the zoom. Matlab got that fixed only in the last (few?) version.