Openseadragon: Troubles with correct coordinate transformation when capturing a selection as image

[jakobgager]

I try to capture a part of an image as base64 png (canvas.toDataURL).
To this end, I use the selection plugin and work with the onSelection event.
The rect from the event is transformed from image coordinates to appropriate canvas coordinates. Here I use

let nxy = viewer.world.getItemAt(0).imageToWindowCoordinates(new OpenSeadragon.Point(rect.x, rect.y));
let nwh = viewer.world.getItemAt(0).imageToWindowCoordinates(new OpenSeadragon.Point(rect.width, rect.height));

to get the transformed coordinates. With these I create e new canvas and use the canvas.getContext('2d').drawImage function to get the cropped portion of the tiled image.
Finally, the toDataURL function of this canvas is used. For details on the image portion topic see e.g. https://stackoverflow.com/a/34244421/2870069.

I do not use the viewport transformation functions as I work with a sequential multi-image.

The issue now is that if I use the initial view I get an image which is slightly of vertically. If I start to zoom the portion of the image moves around the canvas in a not so obvious way.
I just wonder, if I took the wrong transformation functions to define the selection.

Thanks alot!

Btw. I just recently stumbled across OpenSeadragon when looking for a way to handle large image data and was really impressed by the performance of the tool! Many thanks!

[rsimon]

Not sure how far off things are in your case. But I have implemented something similar for Annotorious and that seems to match the selection ok. Here's the code I'm using:

https://github.com/recogito/annotorious-openseadragon/blob/main/src/util/ImageSnippet.js

[jakobgager]

@rsimon thanks for your code! I'll have a look at it.
In the mean time I also discovered picturae/openseadragonselection/issues/40 which seems like more of less the same issue. Unfortunately this is still open.
Grüße aus Favoriten!

[rsimon]

Ah - I wasn't aware of this issue. Without knowing how the rect is computed, it's hard to tell what's going on in that code.

But, yes, it's tricky ;-) There are multiple coordinate systems involved, and there are multipe ways to get there. In my case:

1. I started with the bounds of the selection in the browser. (The selection is also rendered as an SVG shape, so I can get the bounds via .getBoundingClientRect();
2. Offsetting that with the bounding client rect of the OSD canvas gets you x/y offsets relative to the OSD canvas
3. __Important__: the OSD canvas has different height and width attributes set than its physical width/height on the screen, you need to account for that (see kx, ky factors in my code).

The advantage of this approach is that you don't need to deal with OSD's internal viewport coordinate system, nor the original base image coordinate system. It's just about the coordinates of your shape relative to the browser.

Grüße aus Favoriten!

:-D Grüße aus Meidling!

[jakobgager]

Your approach works perfectly! Now I create an overlay using the addOverlay function from openseadragen (so they handle the transformation for me) and get the bounds of the created div to be used with your code.
I have no idea what's wrong with the original approach, but I'm fine with the current solution.
Thanks again!