Javacv: MatExpr.asMat() lead wrong result

[chenchuangc]

I am using javacv calculate ssim ( with the opencv )
environment ： mac-os
dependency is

  <dependency>
      <groupId>org.bytedeco</groupId>
      <artifactId>javacv-platform</artifactId>
      <version>1.4.3</version>
    </dependency>

refer opencv code is here :
https://docs.opencv.org/2.4/doc/tutorials/gpu/gpu-basics-similarity/gpu-basics-similarity.html
with the method

Scalar getMSSIM( const Mat& i1, const Mat& i2)

i transform the code to java ,but it seems something strange , the code is below
please focus on the step5 and step6

    public static Scalar calSSIM(Mat imgOne, Mat imgTwo) {
        double C1 = 6.5025, C2 = 58.5225;

        Mat I1 = new Mat();
        Mat I2 = new Mat();
        imgOne.convertTo(I1, CV_32F);
        imgTwo.convertTo(I2, CV_32F);

        Mat I2_2 = I2.mul(I2).asMat();
        Mat I1_2 = I1.mul(I1).asMat();
        Mat I1_I2 = I1.mul(I2).asMat();

        //step1
        Mat mu1 = new Mat();
        Mat mu2 = new Mat();
        GaussianBlur(I1, mu1, new Size(11, 11), 1.5);
        GaussianBlur(I2, mu2, new Size(11, 11), 1.5);
        Mat mu1_2 = mu1.mul(mu1).asMat();
        Mat mu2_2 = mu2.mul(mu2).asMat();
        Mat mu1_mu2 = mu1.mul(mu2).asMat();

        //step2

        Mat sigma1_2 = new Mat();
        Mat sigma2_2 = new Mat();
        Mat sigma12 = new Mat();

        GaussianBlur(I1_2, sigma1_2, new Size(11, 11), 1.5);
        sigma1_2 = subtract(sigma1_2, mu1_2).asMat();

        GaussianBlur(I2_2, sigma2_2, new Size(11, 11), 1.5);
        sigma2_2 = subtract(sigma2_2, mu2_2).asMat();

        //step3
        GaussianBlur(I1_I2, sigma12, new Size(11, 11), 1.5);
        sigma12 = subtract(sigma12, mu1_mu2).asMat();


        Mat t1, t2, t3;
        Scalar scalar = new Scalar(C1);
        //step4
        t1 = add(multiply(2, mu1_mu2), scalar).asMat();
        t2 = add(multiply(2, sigma12), new Scalar(C2)).asMat();
        t3 = t1.mul(t2).asMat();

        //step5
        //if i use the line blew  replace the next line ,the resulet will not right ,the diff is only wether use MatExpr.asMat() in the calculating process
//      t1 = add(add(mu1_2, mu2_2).asMat(), new Scalar(C1)).asMat();  
        t1 = add(add(mu1_2, mu2_2), new Scalar(C1)).asMat();
        t2 = add(add(sigma1_2, sigma2_2), new Scalar(C2)).asMat();
        t1 = t1.mul(t2).asMat();

        Mat ssim_mat = new Mat();
        divide(t3, t1, ssim_mat);
        Scalar ssim = mean(ssim_mat);

        return ssim;
    }


  //step6
   //in order to test the difference 
   //here i use two same image for test , the right result should be 
   // SSIM: (1.0, 1.0, 1.0, 0.0)
  // but in the step 5 ,if i use the  t1 = add(add(mu1_2, mu2_2).asMat(), new Scalar(C1)).asMat();
  // the SSIM is 
  // SSIM: (1.0, 1.0010865, 1.0010098, 0.0)

    public static void main(String[] args) {
        String im1 = "/Users/admin/work/learn/30.picture_/06.work_process_/02test.jpg";

        Map<String, String> imageMaps = new HashMap<>();
        opencv_core.Mat imageOne = imread(im1);
        opencv_core.Mat imageTwo = imread(im1);
        System.out.println("SSIM: "+calSSIM(imageOne,imageTwo));

    }

i am sorry for boring you ,but i can not find the reason anywhere

[saudet]

Do you get the same result with JavaCV 1.4.4?

[chenchuangc]

#

Do you get the same result with JavaCV 1.4.4?

yea , i have just change the dependency to JavaCV1.4.4

 <dependency>
      <groupId>org.bytedeco</groupId>
      <artifactId>javacv-platform</artifactId>
      <version>1.4.4</version>
    </dependency>

but the issue is still exist

[saudet]

Sounds like an issue with OpenCV. Are you sure it doesn't behave like that in C++ either?

[chenchuangc]

Are you sure it d

i have not test it in c++ , because there is no MatExpr.asMat() method in cv::MatExpr Class
so i may suspend is there any thing wrong in javacv when transform?

[saudet]

In C++, asMat() is a cast operation like (Mat)add(...).

[chenchuangc]

In C++, asMat() is a cast operation like (Mat)add(...).

but there is not such action in the code of c++ , the code with opencv is like this

 Mat t1, t2, t3;

    t1 = 2 * mu1_mu2 + C1;
    t2 = 2 * sigma12 + C2;
    t3 = t1.mul(t2);              // t3 = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))

    t1 = mu1_2 + mu2_2 + C1;
    t2 = sigma1_2 + sigma2_2 + C2;
    t1 = t1.mul(t2);               // t1 =((mu1_2 + mu2_2 + C1).*(sigma1_2 + sigma2_2 + C2))

    Mat ssim_map;
    divide(t3, t1, ssim_map);      // ssim_map =  t3./t1;

    Scalar mssim = mean( ssim_map ); // mssim = average of ssim map
    return mssim;

you mean i may get the same error when i add an (Mat) in the calculating process?

[saudet]

They are calling the function using the + operator. Try this:

t1 = (Mat)(mu1_2 + mu2_2) + C1;

[chenchuangc]

They are calling the function using the + operator. Try this:

t1 = (Mat)(mu1_2 + mu2_2) + C1;

ok, i will try it latter,thank you so much!

[chenchuangc]

They are calling the function using the + operator. Try this:

t1 = (Mat)(mu1_2 + mu2_2) + C1;

yea i have test it in opencv in c++ just change the place you show
and the result shi not correct

➜  ssim_test git:(master) ✗ ./ssim_test 02test.jpg 02test.jpg
start....
argument[0] is: ./ssim_test
argument[1] is: 02test.jpg
argument[2] is: 02test.jpg
 MSSIM:  R 100.1009740% G 100.1085902% B 99.9999990%
end....

then , does it mean , we should use MatExpr in calculate?
thank you so much !

[saudet]

Maybe, we should report this upstream and get the opinion of OpenCV developers:
https://github.com/opencv/opencv/issues
Could you do that?

[chenchuangc]

ok i will do that

[saudet]

Thanks for reporting this issue upstream! Let's wait and see what they say...

[saudet]

Let's close this issue as it is now being tracked upstream. In any case, thanks again for reporting!