Image-sequencer: Various module ideas to be broken out into separate issues

[jywarren]

Here's a list of possible modules and links to other code we could use to implement them.

Module Candidates, related ideas

• [ ] mask - which accepts a data-url (or a previous step output) and masks everything except 100% black pixels. Alternative (more powerful but similar): the overall brightness (averaged RGB) value of each pixel of a specified mask image is used to determine opacity of the output image. See below "referencing earlier states" brainstorm.
• [x] contrast - with "auto stretch" as a follow-up
• [x] text overlay: https://github.com/mattdesl/fontpath-simple-renderer (see #431)
• [x] histogram - https://www.npmjs.com/package/histogram -- see #119
• [x] Flood-fill (paint bucket), which would accept a "drop point" (moved to #541)
• [ ] Luminosity, etc - https://github.com/yahoo/pngjs-image has lots of useful general-purpose image getters like image.getLuminosityAtIndex(idx)
• [x] some way to add in a new image (respecting alpha) -- add-image (with blend mode, default normal?) - #256
• [x] edge detection -- done! in #168

Referencing earlier states

Complex sequences with masking could require accessing previous states (or nonlinearity):

• [x] flood-fill an area (see above)
• select only the flooded area (could use overlay module somehow)
  
  
  
  • roundabout: lighten everything to <50%, then flood-fill with black? Not 100% reliable.
  
  
  • roundabout 2: flood fill, then blink-diff with original
  
  
• then add step which recovers original image, repeat flood-fill/blink-diff for second region
• reference above masked states in a mask module

Notes:

pattern-fill module to fill areas with patterns in a JS canvas:

var c=document.getElementById("myCanvas");
var ctx=c.getContext("2d");
var img=document.getElementById("lamp");
var pat=ctx.createPattern(img,"repeat");
ctx.rect(0,0,150,100);
ctx.fillStyle=pat;
ctx.fill();

Masking:

ctx.save();
ctx.beginPath();
ctx.moveTo(0, 0);
ctx.lineTo(160, 600);
ctx.rect(0, 0, 160, 600);
ctx.closePath();
ctx.clip();
ctx.drawImage(img, 0, 0);
ctx.restore();

https://github.com/inspirit/jsfeat has a range of features:

JavaScript Computer Vision library

The project aim is to explore JS/HTML5 possibilities using modern & state-of-art computer vision algorithms.

Examples and Documentation

Features

• Custom data structures
• Basic image processing methods (grayscale, derivatives, box-blur, resample, etc.)
  
  
  
  • grayscale (Demo)
  
  
  • box blur (Demo)
  
  
  • gaussian blur (Demo)
  
  
  • equalize histogram (Demo)
  
  
  • canny edges (Demo)
  
  
  • sobel deriv (Demo)
  
  
  • scharr deriv (Demo)
  
  
  • find more at Examples and Documentation page
  
  
• Linear Algebra module
  
  
  
  • LU (Gaussian elimination) solver
  
  
  • Cholesky solver
  
  
  • SVD decomposition, solver and pseudo-inverse
  
  
  • Eigen Vectors and Values
  
  
• Multiview module (Demo)
  
  
  
  • Affine2D motion kernel
  
  
  • Homography2D motion kernel
  
  
  • RANSAC motion estimator
  
  
  • LMEDS motion estimator
  
  
• Matrix Math module for various matrix operation such as traspose, multiply etc.
• Features 2D
  
  
  
  • Fast Corners feature detector (Demo)
  
  
  • YAPE06 feature detector (Demo)
  
  
  • YAPE feature detector (Demo)
  
  
  • ORB feature descriptor (Demo)
  
  
• Lucas-Kanade optical flow (Demo - click to add points)
• HAAR object detector (Demo)
• BBF object detector (Demo)

[jywarren]

GitMate.io thinks the contributor most likely able to help you is @ccpandhare.

[ccpandhare]

These are pretty good resources!

[jywarren]

Great source of FTOs! @tech4GT @Mridul97

[tech4GT]

@jywarren Thanks!

[Mridul97]

Thank you! :smile:

[aashna27]

@jywarren are we ready to have a masking module ? if so I would like to work on it . Also if you could elaborate on " Luminosity" I would like to look into it.

[jywarren]

that'd be really cool. I wonder if it could be implemented using the blend
module, where we use the 2nd image as the opacity value for the first
image? This might not be the 100% most efficient way (or maybe it is,
dunno) but it could re-use code in a clever way :-)

On Fri, Jan 4, 2019 at 1:56 PM aashna27 [e-mail] wrote:

@jywarren https://github.com/jywarren are we ready to have a masking
module ? if so I would like to work on it

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/165#issuecomment-451534888,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJzMdsaO71M7pqunuqqtiEZj7rjxiks5u_6OsgaJpZM4R3kPW
.

[harshkhandeparkar]

Flood fill has its own issue.

[blurry-x-face]

Mask Module

@jywarren I was trying to make the mask module and came up with this by applying Alpha Masking.
Could you brief a little more about this, Or does this looks good?
Thanks!!

[jywarren]

Oh, this looks really nice!