Scalatest: Need relative floating point comparison

[mkolod]

Since floating-point values vary in magnitude, relative comparisons are important because floating-point precision changes depending on magnitude of the number. Absolute testing may be inadequate in certain cases, especially when ScalaTest is used for numeric computing tests (Apache Spark, Breeze, etc.). It would be nice to have relative comparisons in addition to absolute "almost equal" comparisons (C++ Boost has them, for instance). However, one needs to take care to do the relative comparisons correctly - relative comparisons are fine away from zero and they get inaccurate around zero (see here). I co-authored an Apache Spark/MLLib implementation but I think having such comparisons would be useful in ScalaTest. I could contribute the changes back to ScalaTest if I get feedback as to the optimal API desired by the community (e.g. similar to the === implementation with TripleEqualsInvocationOnSpread). Currently, the Spark solution uses the ~== and !~== methods with +- method for relative tolerance. Here's a test suite demo of the Spark implementation. Please let me know if I can contribute such a feature to ScalaTest. Of course, the Spark pull request is still in flux, but I wanted to give you an idea of what I meant.

[bvenners]

This is already support in Scalactic TripleEquals and Tolerance traits, which you can use in assertions in ScalaTest. And it is supported in ScalaTest matchers as well. Here are some examples:

scala> import org.scalactic.TripleEquals._
import org.scalactic.TripleEquals._

scala> import org.scalactic.Tolerance._
import org.scalactic.Tolerance._

scala> 2.0 === 2.01 +- 0.1
res0: Boolean = true

scala> 2.0 === 3.01 +- 0.1
res1: Boolean = false

scala> import org.scalatest.Assertions._
import org.scalatest.Assertions._

scala> assert(2.0 === 2.01 +- 0.1)

scala> assert(2.0 === 3.01 +- 0.1)
org.scalatest.exceptions.TestFailedException: 2.0 did not equal 3.01 +- 0.1
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1538)
at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
... 43 elided

scala> import org.scalatest.Matchers._
import org.scalatest.Matchers._

scala> 2.0 should equal (2.01 +- 0.1)

scala> 2.0 should equal (3.01 +- 0.1)
org.scalatest.exceptions.TestFailedException: 2.0 did not equal 3.01 plus or minus 0.1
at org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160)
at org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:6231)
at org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)
... 43 elided

[fommil]

hi @bvenners ... this came up in my team recently too but there is something subtle in the feature request :smile:

the idea is to have a more complex typeclass instance for tolerance on double, such that the comparison is relative rather than absolute.

[bvenners]

That's an interesting idea. Yes, it makes sense. How would it be expressed? As a percentage perhaps?

[fommil]

Yeah, that'd be a good reminder: +-%

[jimka2001]

I see this is closed? Was it actually implemented? If no, could someone post some user code which would implement the request? The problem is that I don't know what absolute value of epsilon to specify, because it completely depends on the order of magnitudes of the numbers being checked.