Vavr: Look into mutation testing

[roookeee]

I see that VAVR has some pretty high test coverage (>90%) but I stumbled upon this commit which has this test:

@Test
    public void shouldShuffleHaveSameElementsDeterministic() {
        final Seq<Integer> shuffled = of(1, 2, 3).shuffle(new Random(514662720L));
        assertThat(shuffled.indexOf(1)).isNotEqualTo(-1);
        assertThat(shuffled.indexOf(2)).isNotEqualTo(-1);
        assertThat(shuffled.indexOf(3)).isNotEqualTo(-1);
        assertThat(shuffled.indexOf(4)).isEqualTo(-1);
    }

Combined with the other tests of said commit there is no test that actually tests whether the shuffle is deterministic (just checking length, checking all elements are contained). Have a look at mutation testing to fix these kind of bad tests in VAVR so that your high test coverage is more reliable

[danieldietrich]

Thanks for your suggestion. This project is a community effort. Help is welcome!

Update: I followed your link and looked into that lib. I don't want to introduce just another dependency. The number of cases where mutation testing can be applied is small. Maybe there is even only this one. Instead, I suggest to rewrite the test as follows:

• test that the size of the shuffled collection is the same as the original collection
• test that the elements stayed the same

In order to ensure that the shuffled collection isn't equal to the original collection, we could increase the size of the collection the way that the probability is really small that we will receive the same collection. The we could additionally

• test that the shuffled collection does not equal to the original collection (if that fails we could repeat the test for a few times). Even better: We should fix the random order using a seed

[mincong-h]

Hi @roookeee and @danieldietrich , if you don't mind, I can give it a try.

[roookeee]

Go ahead I don't mind at all :)

[mincong-h]

Done in https://github.com/vavr-io/vavr/commit/7544480410fe235c3314d6eff897a6584c340ebb

[danieldietrich]

Thx!