Reactor-core: expectNoEvent does not fail test when event is actually emitted

[PyvesB]

Expected Behavior

According to the Javadocs of StepVerifier.expectNoEvent(Duration):

Expect that no event has been observed by the verifier for the length of the provided Duration

When I create a Mono that emits an item, I would expect the verification to fail.

Actual Behavior

expectNoEvent does wait for the specified duration, but does not seem to have any effect on the expectations.

Steps to Reproduce

Consider the following unit test which is part of the project:
https://github.com/reactor/reactor-core/blob/5e9231edb8e3362730ae7c9b7f7dceeb336c5d17/reactor-test/src/test/java/reactor/test/StepVerifierTests.java#L1063

Change the publisher to Mono.just("notNever") so that it actually emits an event:

@Test
public void verifyVirtualTimeNoEventNever() {
    StepVerifier.withVirtualTime(() -> Mono.just("notNever")
                                           .log())
                .expectSubscription()
                .expectNoEvent(Duration.ofDays(10000))
                .thenCancel()
                .verify();
}

The test will still pass.

Your Environment

• Reactor version(s) used: latest from master
• Other relevant libraries versions (eg. netty, ...):
• JVM version (javar -version): 13
• OS and version (eg uname -a): reproduced on Windows and macOS

[berry120]

Looks like this works as expected (test fails) in 3.1.2, then is broken (test passes when it shouldn't) in 3.1.3 onwards.

From a brief look, it appears as though the completeLatch in DefaultStepVerifierBuilder isn't being counted down to 0 until after the NoEvent has executed, so it doesn't see any issue and therefore doesn't fire. (I could be right off the money with that suggestion though, I'm not familiar with the codebase.)

[berry120]

I believe this is also a related SO question: https://stackoverflow.com/questions/58486417/how-to-verify-with-stepverifier-that-provided-mono-did-not-completed

[OlegDokuka]

Due to my mamory that was a constant flaw in design of reactor-test. Most likely that will be fixed once reactor-test will be rewritten from scratch / significantly. I have been trying to do that a couple of times before but ended up with the mentioned fact.

The problem is hidden in blocking behavior of expectNoEvent. Which block the thread. Thus in any case observed events will be after the lock release therefore all of them will be considered as apperead after (e.g no way to detect failure of expectation)

I guess it worth to say that for now the use case of expectNoEvent is only in case of virtualTimeScheduler. In all others it does not work

[berry120]

@OlegDokuka Thanks for the explanation.

I guess it worth to say that for now the use case of expectNoEvent is only in case of virtualTimeScheduler. In all others it does not work

It looks like the virtual time scheduler is being used in the code above however, but it still doesn't work there?

Completely understand fixing it is a big exercise - could we put something in the Javadoc that clarifies the behaviour in the meantime (or maybe deprecate the method entirely if it's broken in most cases?)

[PyvesB]

I did have a play around with the virtual scheduler, and if I recall correctly it doesn't work either.

[simonbasle]

I've been attempting to fix this without making much progress 😢
The original example can be in a gray area because it is using virtual time with an operator that doesn't care about time (Mono.just(...) isn't impacted by the clock), but then in the SO question a similar issue occurs without virtual time...

But the workaround you suggested @PyvesB is interesting. Maybe we can add something similar to StepVerifier.LastStep API: expectTimeoutAfter(Duration). It could replace expectNoEvent(duration).thenCancel(), and hopefully avoid that issue...

[simonbasle]

@OlegDokuka could you open a separate issue for the synchronous nature of clock-based methods? I think thenAwait(Duration) is already sensitive in that regard? Ideally an issue with a good demonstration of the problem so that we have a good foundation to work with :bow:.

[simonbasle]

opened #1931 to introduce the alternative discussed above @PyvesB @berry120 @OlegDokuka