Image-sequencer: Automated feature detection (and matching) - long-term

[jywarren]

Let's not think too much about this YET - but one add-on feature we could add to this feature for image distortion (#299) would be a way to automatically examine 2 images and find how they ought to overlap. This has several steps:

1. finding interest points in 2 images
2. matching interest points between 2 images
3. calculating the corner transformations required to use the matches (if any) to perform a distortion using #299

Here's one in-depth tutorial that goes through all the necessary steps: https://hub.packtpub.com/object-detection-using-image-features-javascript/

This would be great to develop as an independent module, outside of Image Sequencer!

[gitmate[bot]]

GitMate.io thinks possibly related issues are https://github.com/publiclab/image-sequencer/issues/216 (Add WebGL in CLI/Node context (long-term)), https://github.com/publiclab/image-sequencer/issues/166 (Edge detection module), https://github.com/publiclab/image-sequencer/issues/284 (Shorten long URLs), and https://github.com/publiclab/image-sequencer/issues/205 (add "drag to crop" UI feature to Crop module).

[gitmate[bot]]

GitMate.io thinks possibly related issues are https://github.com/publiclab/image-sequencer/issues/216 (Add WebGL in CLI/Node context (long-term)), https://github.com/publiclab/image-sequencer/issues/166 (Edge detection module), https://github.com/publiclab/image-sequencer/issues/284 (Shorten long URLs), and https://github.com/publiclab/image-sequencer/issues/205 (add "drag to crop" UI feature to Crop module).

[jywarren]

This should be possible in OpenCV.js!

• http://answers.opencv.org/question/5348/evaluation-of-interest-point-detectors-and-descriptors/
• https://docs.opencv.org/2.4/doc/tutorials/features2d/feature_detection/feature_detection.html
• https://stackoverflow.com/questions/23637916/find-interest-point-in-surf-detector-algorithm
• https://stackoverflow.com/questions/29133085/what-are-keypoints-in-image-processing

steps might be:

1. run SURF on Image 1 to find interest points
2. run SURF on Image 2 to find interest points
3. find best matches of common points from each
4. in relation to image 1, rotate, scale, and translate image 2 to best overlap the top 2-3 common points between the 2 images
5. flatten into 1 image

Amir shared: https://docs.opencv.org/3.3.0/dc/dc3/tutorial_py_matcher.html

https://docs.opencv.org/2.4/modules/ocl/doc/feature_detection_and_description.html?highlight=feature%20description

[jywarren]

An initial version wouldn't need to do full 4-corner transform but could start with just rotation/scale/translate!

[rockstar777]

@jywarren Can I work on this issue?
I found intersting to do it. :smile:

[jywarren]

That'd be really awesome. Have you been able to merge a more basic module first? Perhaps #165 has some, if you haven't (apologies, it's been a busy week so I forget!)

However, we'd really like your help with this. It seems like a big project but we'd love to hear how you might try to solve it? What would be your first step?

👍 🎉

[rockstar777]

Thanks @jywarren
I had'nt merge that before but i would try that in #165
Yes, It is big project but let's try to solve by detecting interesting points in two images using opencv.js.
My first step for this issue would be I will get comfortable in OpenCV.js, then try to implement code for detecting interesting points, then moving to matching points and distortion of images.

[jywarren]

yes, i think we could have a "finding points" module, a "matching points"
module, and then a distortion one... solve it piece by piece in separate
PRs, you know?

On Tue, Oct 2, 2018 at 4:10 AM puneet bindal [e-mail]
wrote:

Thanks
I had'nt merge that before but i would try that in #165
https://github.com/publiclab/image-sequencer/issues/165
Yes, It is big project but let's try to solve by detecting interesting
points in two images using opencv.js.
My first for this issue would be I will get comfortable in OpenCV.js, then
try to implement code, then moving to matching points and distortion of
images.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/300#issuecomment-426186487,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ5jq1fILuMh86SNqckIGIq_vmg8oks5ugx8ygaJpZM4U6VT2
.

[jywarren]

There are actually feature detection and description abilities in jsfeat here: https://github.com/publiclab/image-sequencer/issues/165

I /believe/ this is in pure javascript!

[jywarren]

and there's a very cool demo that seems to be running here: https://inspirit.github.io/jsfeat/sample_orb.html

[rockstar777]

hi @jywarren,
I have created a GSOC proposal for this issue, please review and help me to improve my proposal.
Thanks (apologies for being inactive for last few months)
:)

[rockstar777]

Also, please mention if it is okay to include this issue in GSOC '19.
Thanks

[jywarren]

Of course it's fine to include! Did you post your proposal on our website
at
https://publiclab.org/notes/warren/02-28-2019/call-for-summer-of-code-2019-proposals
? Thank you!

On Tue, Apr 2, 2019 at 11:42 AM puneet bindal [e-mail]
wrote:

Also, please mention if it is okay to include this issue in GSOC '19.
Thanks

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/300#issuecomment-479058442,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJzYn0KwJu2iK1klexZIvu8LkAa__ks5vc3pHgaJpZM4U6VT2
.

[rockstar777]

Yes,I had Posted it from username puneetbindal8989.
Link: post link
Thanks :)