Image-sequencer: Module idea: matrix math

[jywarren]

Many image operations are made possible with matrix operations, including blurs, edge detection, and lens distortion.

http://aishack.in/tutorials/image-convolution-examples/

If we had a general purpose matrix math module (or just a matrix math core function?), we could use it as the basis for lots of cool modules!

[tech4GT]

Awesome!! This is uncanny, I was thinking about this last week itself!! This can help us with a lot simple convolution based steps you are right. One thing we need though is to dynamically adjust the inputs if we want the matrix size to be variable.
For example if the user wants a 3x3 kernel, we would take 9 inputs and if the user wants a 4x4 kernel inputs change to 16. I think the best solution here would be take all the inputs as a space separated string listing the inputs row-wise. What say?

[jywarren]

lol isn't "canny" a matrix math method?

On Tue, Sep 4, 2018 at 12:20 PM Varun Gupta [e-mail]
wrote:

Awesome!! This is uncanny, I was thinking about this last week itself!!
This can help us with a lot simple convolution based steps you are right.
One thing we need though is to dynamically adjust the inputs if we want the
matrix size to be variable.
For example if the user wants a 3x3 kernel, we would take 9 inputs and if
the user wants a 4x4 kernel inputs change to 16. I think the best solution
here would be take all the inputs as a space separated string listing the
inputs row-wise. What say?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/324#issuecomment-418430379,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ4MGWcEJu8htkLIdeU9EED3qA0hNks5uXqhAgaJpZM4WP5UL
.

[tech4GT]

Haha😂

[jywarren]

aha, well we could start with a narrower 3x3 implementation to try this
out. Isn't 4x4 not centered around a single pixel? Or does it operate on 4
pixels? Or do I not really understand matrix math LOL

On Tue, Sep 4, 2018 at 12:52 PM Jeffrey Warren [e-mail] wrote:

lol isn't "canny" a matrix math method?

On Tue, Sep 4, 2018 at 12:20 PM Varun Gupta [e-mail]
wrote:

Awesome!! This is uncanny, I was thinking about this last week itself!!
This can help us with a lot simple convolution based steps you are right.
One thing we need though is to dynamically adjust the inputs if we want the
matrix size to be variable.
For example if the user wants a 3x3 kernel, we would take 9 inputs and if
the user wants a 4x4 kernel inputs change to 16. I think the best solution
here would be take all the inputs as a space separated string listing the
inputs row-wise. What say?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/324#issuecomment-418430379,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ4MGWcEJu8htkLIdeU9EED3qA0hNks5uXqhAgaJpZM4WP5UL
.

[tech4GT]

Oh yeah you're correct!! Sorry for that, it's going to be odd so 3x3, 5x5 and so on...

[Mridul97]

I would love to work on this issue. I think we have to create a module that takes pixels of the input picture and the 9 values of 3x3 kernel as arguments and returns the pixels of the picture after applying convolution. For starters we can keep kernel to be 3x3 and then later try to make it general. Your thoughts please? @jywarren

[jywarren]

This sounds pretty interesting and i like your idea of specific first
general later. Would you be interested in opening a PR to test it out?

On Mon, Sep 10, 2018 at 10:27 AM Mridul97 [e-mail] wrote:

I would love to work on this issue. I think we have to create a module
that takes pixels of the input picture and the 9 values of 3x3 kernel as
arguments and returns the pixels of the picture after applying convolution.
For starters we can keep kernel to be 3x3 and then later try to make it
general. Your thoughts please? @jywarren https://github.com/jywarren

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/324#issuecomment-419932270,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ_OgDdDazGtEMyMj4lyIl24_lciuks5uZnaVgaJpZM4WP5UL
.

[Mridul97]

Yes, I will open a PR as soon as I complete coding the module. Thanks!

[Mridul97]

Hey, I have opened a PR #336. Please have a look!

[Mridul97]

Hi, I was trying to test the module but I am not getting how to convert ndarray to imagedata. Could you please help me with that? @jywarren @tech4GT

[tech4GT]

@Mridul97 You wouldn't need to do that if you use the pixelManipulation API available in image sequencer, if you want to though you can look at this file https://github.com/publiclab/image-sequencer/blob/main/src/modules/_nomodule/PixelManipulation.js
We basically use the save-pixels npm package to convert to an image

[tech4GT]

Look at this code https://github.com/publiclab/image-sequencer/blob/88364decf6a6111a0be31fd98010576d5c6f918e/src/modules/_nomodule/PixelManipulation.js#L80-L95

[Mridul97]

Thanks! @tech4GT

[Mridul97]

I have opened a PR #358. Please have a look!

[jywarren]

I think this is basically complete -- we should break out the follow-up issues, but I also wanted to highlight what seems to be odd about the "sharpen" kernel of:

https://publiclab.github.io/image-sequencer/examples/#steps=convolution{constantFactor:1.0|kernelValues:-1%200%20-1%200%205%200%20-1%200%20-1}

Could be I"m doing something wrong...

[tech4GT]

@jywarren This is wierd behavior, I'll look into this issue right away! Thanks!

[jywarren]

Don't stress out! Pace yourself! 👍🏼

On Sat, Oct 6, 2018, 1:28 PM Varun Gupta [e-mail] wrote:

@jywarren https://github.com/jywarren This is wierd behavior, I'll look
into this issue right away! Thanks!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/publiclab/image-sequencer/issues/324#issuecomment-427591912,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABfJ6H_D5WUAzHXEObMxSsna_vTa4Zkks5uiOf6gaJpZM4WP5UL
.

[tech4GT]

Actually my trip got cancelled, I figured might as well write some code, haha 😃

[jywarren]

Oh I hope its all right!! I'm about to leave on a trip for 2 weeks starting w a 15h flight 😴😴🛫

Still planning to keep up with PRs but will be a little slower than usual!

[tech4GT]

Awesome!!👌

[jywarren]

I was wondering if the sharpen above is still seeing issues? no rush though, just curious.

[tech4GT]

Oh, I completely forgot about this, will look into this as soon as I get time, pretty busy with the end of semester though 😄