Opencv4nodejs: question about perspectiveTransform

[Gektor40k]

Hello!
In the c++ code which I'm porting to node.js, using function:
CV_EXPORTS_W void perspectiveTransform (InputArray src, OutputArray dst, InputArray m)
(from core.hpp) - I tried to find an analog of this function in opencv4nodejs, but there is only perspectiveTransform, which is a function of Mat class.
There are other arguments and the result of the work.
Can I find a suitable option, or it does not exist?

[justadudewhohacks]

For any function, if the first argument is the source mat itself, then the function will be part of the Mat instance. Also outputs are usually always returned in opencv4nodejs and never passed as arguments.

Thus perspectiveTransform will translate to:
const dst = src.perspectiveTransform(m)

Hope this clarifies it.

[Gektor40k]

Thank you, i think its should work!
Then the additional question, unfortunately, could not find information about it.
InputArray src is not Mat, it is an array of Point2.
Can I make a new Mat from this array? What constructor or function should I use?

[justadudewhohacks]

Yes, I think in general in OpenCV you can represent a std::vector of Points and Vecs (or Arrays of Points and Vecs in case of Javascript) as Mats.

It will probably look somehow like this in case of Point2:

As a 1D 2-channel matrix:

const matData = [
  points.map(pt => [pt.x, pt.y])
]

const src = new cv.Mat(matData, cv.CV_32FC2)

As a 2D 1-channel matrix:

const matData = [
  [], // array with x coordinates
  [] // array with y coordinates
]
points.forEach(pt => { 
  matData[0].push(pt.x); 
  matData[1].push(pt.y) 
})

const src = new cv.Mat(matData, cv.CV_32FC1)

Not sure which one is the correct way to do it.

[Gektor40k]

Yes! Thank you! All is working!

[woojoo666]

neither method is working for me. I tried

const matData = [
  [0, 0],
  [10, 0],
  [10, 20],
  [20, 20]
];
const srcCorners = new cv.Mat(matData, cv.CV_32FC2)
const dstCoordinates = srcCorners.perspectiveTransform(homography)

and this silently fails (exits without getting to the last line, perspectiveTransform() is never called).
I believe the matrix type CV_32FC2 is the problem, because I can create CV_32F matrices without issue.

If I try the other method:

const matData = [
  [0, 10, 10, 0], // array with x coordinates
  [0, 0, 20, 20] // array with y coordinates
];
const srcCorners = new cv.Mat(matData, cv.CV_32FC1)
const dstCoordinates = srcCorners.perspectiveTransform(homography)

this one gives me the error:

const dstCoordinates = srcCorners.perspectiveTransform(homography)
                                  ^
Mat::PerspectiveTransform - OpenCV Error: (scn + 1 == m.cols) in cv::perspectiveTransform, in file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matmul.cpp, line 2268, status -215

Could you help me out? I'm really having trouble finding any sort of documentation on this stuff. And my homography matrix is correct, I checked it by manually printing out each cell value.

[justadudewhohacks]

Hi,

Regardless from the first error, I am not quite sure but I think perspectiveTransform expects a 4x4 single channel Matrix, doesn't it?

The reason why you can't create a Mat with CV_32FC2 is because you provide it data with values for a single channeled Mat. For a 2 channel Mat it would be something like this:

const matData = [
  [ [0, 0] , [0, 0] ],
  [ [10, 10], [0, 0] ],
  [ [10, 10], [20, 20] ],
  [ [20, 20], [20, 20] ]
];

[woojoo666]

findHomography gives me a 3x3 matrix, am I supposed to pad it with something before giving it to perspectiveTransform? And how would I create a 1D 2-channel mat? The method you gave looks to be a 2D 2-channel mat

[justadudewhohacks]

Ok, it should definitely work with the Mat returned from findHomography.

A 1D Mat is just a Matrix with a single row only:

const matData = [
  [ [0, 0] , [0, 0], ... ]
]

[woojoo666]

isn't that what I did?

const matData = [
  [0, 0],
  [10, 0],
  [10, 20],
  [20, 20]
];
const srcCorners = new cv.Mat(matData, cv.CV_32FC2)
const dstCoordinates = srcCorners.perspectiveTransform(homography)

is this not creating a 1D 2-channel matrix, and then calling perspectiveTransform on it? What am I doing wrong?

[justadudewhohacks]

No, in your example you are creating a Matrix with 4 rows, 2 columns each (4x2) Matrix, with flat values (single channeled).

Note the array boundaries in my example.

[woojoo666]

oh wait I see, I needed another level of brackets. Now it doesn't silently fail anymore, but it is now giving me the error

const dstCoordinates = srcCorners.perspectiveTransform(homography)
                                  ^
Mat::PerspectiveTransform - OpenCV Error: (scn + 1 == m.cols) in cv::perspectiveTransform, in file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matmul.cpp, line 2268, status -215

I tried appending a 1 to each point to make them 3 dimensional, but that didn't fix it either.

[justadudewhohacks]

According to http://answers.opencv.org/question/110408/assertion-failed-scn-1-mcols/, this might be due to your homography being empty.

How do you actually calculate the homography? If you are using cv.findHomography then you should input an array of source and an array of destination points cv.findHomography(srcPoints, dstPoints) as stated here: https://opencv4nodejs.herokuapp.com/docs/calib3d#findHomography

[woojoo666]

I manually printed out my homography, and it is definitely not empty. In fact, right now I am manually doing a perspective transform by extracting the homography matrix using Mat.at(r,c), then doing matrix multiplication, and then drawing the transformed points. That method seems to be working for me, so I don't think the homography matrix is the problem

[woojoo666]

I only modified the matchFeatures.js example a tiny bit.

Here is my code
And here is a diff between matchFeatures.js and my code

[justadudewhohacks]

Hmm, sorry I am not familar with how exactly the source Matrix for perspectiveTransform has to look like. But from the docs it looks like they should accept 2D or 3D vectors, so your Matrix probably either has to have 2 or 3 rows and be a single channel Mat, or it has to be a single row Mat of 2 or 3 channels, but not sure which one is the right way to go. Also it would be probably better to accept an array of Vec2 or Vec3 instead of a Matrix here in the implementation of the binding.

[amberneil]

Just wanted to chime in here, I've struggled with this problem for a day or so.

It didn't end up being an empty homography result, which is the diagnosis all over Google, it was my formatting of the matrix that you have labelled "matData" in https://github.com/justadudewhohacks/opencv4nodejs/issues/161#issuecomment-383359698. This structure worked for me:

const matData = [
    [[0, 0]],
    [[10, 0],
    [[10, 20]],
    [[20, 20]],
];
const srcCorners = new cv.Mat(matData, cv.CV_32FC2);
const dstCoordinates = srcCorners.perspectiveTransform(homography);

The nested arrays take a minute to wrap your head around, and the cv.CV_32FC2 parameter tripped me up for a long time. Any other type than cv.CV_32FC2 gave me the "OpenCV Error: (scn + 1 == m.cols)".

[rainabba]

@justadudewhohacks I'm learning my way around homography (for form/image alignment) and was excited to see https://opencv4nodejs.herokuapp.com/docs/calib3d#findHomography, but the app looks to be dead. Will that be coming back?

[rainabba]

@woojoo666 I'm still stuck on using findHomography(). Mind sharing your implementation for us to learn from?

[saadshahd]

@justadudewhohacks How to extract the dest XY from the result matrix?