Image-sequencer: A new Overlay subpart module

[tech4GT]

Our current overlay module works perfectly for complete Images but in some cases we just want part of the Image to be overlaid. Like over at image-sequencer-app we are trying to overlay one warped image onto another but that does not work correctly since the area around the warped image is black.
I think we can create a subpart-overlay module which can take the coordinates of the object in the image to overlay and can overlay just that.
The problem this boils down to is making a function which will determine whether a given point is inside the quadrilateral defined by the 4 input points.
I would love to hear your thoughts in this @publiclab/is-reviewers
cc @jywarren

[tech4GT]

[jywarren]

Hi! Is this resolved in https://github.com/publiclab/image-sequencer/pull/1082 ?

[tech4GT]

Actually I made it work for that specific use case using a custom overlay module but this is still something we might wanna have. It may come in handy, Don't you think?

[jywarren]

Sure can you just write out the additional fix you're proposing? Thanks!

On Mon, May 27, 2019, 2:09 PM Varun Gupta [e-mail] wrote:

Actually I made it work for that specific use case using a custom overlay
module but this is still something we might wanna have. It may come in
handy, Don't you think?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/1080?email_source=notifications&email_token=AAAF6J7ICIYUHR5YJPEMTTDPXQPUXA5CNFSM4HPYFJH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWKJV6Q#issuecomment-496278266,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAF6J4ZKZKY3QQ27BXW2BLPXQPUXANCNFSM4HPYFJHQ
.

[tech4GT]

Sure! I am thinking of using morphological operations to determine the boundary of the object in the image and then only overlay the pixels inside the boundary! Maybe someone over at the soc team can pick this up, the challenge here is that we cannot conventional methods since those with increase the runtime by a factor of image width!! I think exploring the morphological operations route is well worth it! :smile:

[jywarren]

using morphological operations to determine the boundary of the object in the image and then only overlay the pixels inside the boundary

This would be an optimization, right? Are we all the way there with correct final bounding box in the un-optimized mode, first? But yes, once we have the basic version working, an optimization would be great!

[blurry-x-face]

Hi @tech4GT could you explain this a bit more, I would like to work on this!

[tech4GT]

Hi @blurry-x-face
So it's been a while but as far as I remember this module was supposed to overlay a part of the image onto another. For instance, say I give the module the four corner points of a quadrilateral, I then want only the image inside this quadrilateral to be overlaid. The challenge, as I explained earlier is to determine which pixel is inside the quadrilateral and which one is not. I did try to come up with a solution but that was infeasible due to the complexity being O(height*width*width) which would take forever to complete.

[blurry-x-face]

@tech4GT how about if we crop and then overlay, wouldn't that work?

We could create a meta module using crop, rotate and overlay to achieve that!

[tech4GT]

Yeah that's the challenging part, how do we crop an arbitrary quadrilateral out of the image using just the four corner points.

[blurry-x-face]

I'll try and make something and let you know if anything works out, thanks!

[blurry-x-face]

@tech4GT I was able to make this, this runs in O(n^2) and also uses web assembly so its fast!

If this looks good I can open a PR for it!

[tech4GT]

Sure! Great work!

[blurry-x-face]

Thanks! :smile: