Spring-cloud-contract: @AutoConfigureWireMock does not work on meta-annotations

[fdw]

Similar to #1201, @AutoConfigureWireMock does not reset the stubbing/mocking/request counters if the test class is annotated with a meta-annotation containing @AutoConfigureWireMock.

If you'd rather re-open #1201, please do :)

[marcingrzejszczak]

Do you have an example of such usage?

[fdw]

For the meta-annotation:

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@Inherited
@SpringBootTest
@AutoConfigureWireMock(port = 0)
@AutoConfigureMockMvc
@TestPropertySource("classpath:application-test.properties")
annotation class IntegrationTest

A test that works when directly annotated with @AutoConfigureWireMock needs an additional WireMock.reset() in the @Before method when we use this @IntegrationTest.

[fdw]

Do you need anything else from me?

[marcingrzejszczak]

A sample would be great cause I can't replicate this :grimacing:

[fdw]

After testing a lot of different variants, I think my problem is not so much the meta-annotation (that one works as well as the direct annotation), but rather #1027. I'm sorry for the unnecessary work.

[fdw]

Unfortunately, this issue still persists.

We have a meta-annotation like this:

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@Inherited
@SpringBootTest
@AutoConfigureWireMock(port = 0)
public @interface ComponentTestAnnotation

This meta annotation is used on a base class for tests:

@ExtendWith(SpringExtension.class)
@ComponentTestAnnotation
public class BaseComponentTest {
[...]
}

Finally, we have an actual test like this:

@SpringJUnitWebConfig([...])
public class ActualTest extends BaseComponentTest {
[...]
}

And in the appropriate application.properties is wiremock.reset-mappings-after-each-test=true

However, the mapping and request count are not reset after each test.

With some debugging, the problem seems to lie here:
if (testContext.getTestClass().getAnnotationsByType(AutoConfigureWireMock.class).length == 0) (in WireMockTestExecutionListener, lines 97-98)
Because the class is not directly annotated with AutoConfigureWireMock, the annotation is not found.

If I understand everything correctly (which I'm not sure of 😉), using Spring's AnnotationUtils should help: if(AnnotationUtils.findAnnotation(testContext.getTestClass(), AutoConfigureWireMock.class) != null).

[marcingrzejszczak]

Wow, thanks for the analysis. Are you willing to add a PR with a test and a fix to 2.2.x branch?

[fdw]

I can try, but I'm not sure when I have the time. Do you have a pointer for the tests that you want?

[marcingrzejszczak]

I'd create a test that does exactly what you mentioned.

@Target(AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@Inherited
@SpringBootTest
@AutoConfigureWireMock(port = 0)
@interface ComponentTestAnnotation {
}

@ExtendWith(SpringExtension.class)
@ComponentTestAnnotation
class BaseComponentTests {
[...]
}

@SpringJUnitWebConfig([...])
public class AutoConfigureWireMockMetaAnnotationTests extends BaseComponentTests {
[...]
}

In the AutoConfigureWireMockMetaAnnotationTests I would create a simple test that verifies if the stub got reset. Yuo can create it here spring-cloud-contract-wiremock/src/test/java/org/springframework/cloud/contract/wiremock

[fdw]

I tried to start a PR, but I have some problems setting it up:

• Intellij does not find a shaded. package to import: Error:(21, 1) Groovyc: unable to resolve class shaded.com.google.common.collect.ImmutableMap. I'm not sure if this bug is the reason: https://youtrack.jetbrains.com/issue/IDEA-93855
• mvnw on the CLI doesn't download all the dependencies apparently: package org.eclipse.aether.util.repository does not exist

I followed the readme as best as I could and installed the Intellij plugins, but the problem persists.

[marcingrzejszczak]

Hi! You need to first run ./mvnw install, that will build the shaded module where you'll find the shaded... classes. It works in such a way that it will unpack the shaded source files and attach it as sources to the project.

[fdw]

Unfortunately, I can't run ./mvnw install, as some tests fail. Is master not the correct branch to start this?

[fdw]

Hi, I would still be interested in opening a PR, but the problems persist: I can't even build the project.

[Jakzi666]

We just had the issue as described above, is there any chance that this may be fixed soon?

[marcingrzejszczak]

Hi, I would still be interested in opening a PR, but the problems persist: I can't even build the project.

Hey @fdw ! Do you still have problems with building the project? What exactly seems to be the problem?

We just had the issue as described above, is there any chance that this may be fixed soon?

We have limited capacity - pull requests are welcome!

[fdw]

I just tried again; checking out master and running ./mvnw clean install -P integration. I'd expect that to work, but unfortunately, I get an error:

[ERROR] Tests run: 1196, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 111.804 s <<< FAILURE! - in JUnit Vintage
[ERROR] should allow to set dynamic values via stub / test matchers for [spock]  Time elapsed: 0.312 s
[ERROR] should allow to set dynamic values via stub / test matchers for [testng]  Time elapsed: 0.451 s
[ERROR] should allow to set dynamic values via stub / test matchers for [mockmvc]  Time elapsed: 0.394 s
[ERROR] should allow to set dynamic values via stub / test matchers for [jaxrs-spock]  Time elapsed: 0.22 s
[snip]

Is it enough if I just work inside spring-cloud-contract-wiremock, or do I need the rest to work?

[marcingrzejszczak]

Which test is failing and why? Are you running this on Windows or Linux?

[fdw]

I'm on Linux. Unfortunately I cannot really tell what failed; I get a looong wall of text (enough to exhaust my scrollback), and then the pasted lines. But I can tell you that it's in "Spring Cloud Contract Verifier".

[marcingrzejszczak]

I can't really help you cause the build works fine for us on our CI machines. You'd have to tell us which concrete test fails and why.

[fdw]

Can you tell me if I even need to run these tests, or are the ones in "spring-cloud-contract-wiremock" enough?

[marcingrzejszczak]

Running those in sc-wiremock is enough

[fdw]

Okay, thanks, then I'll try to prepare a PR :)

[fdw]

FYI: I'm still working on this, but the CLA is a bit more of a problem than I thought ;)

[fdw]

PR is here, thanks in advance for the review :)

[marcingrzejszczak]

Hey @fdw I had to revert the commits cause both 2.2.x and master branches were broken for us. Maybe if you run the tests in random order then you can replicate this issue?

[fdw]

I've rebased my branch on the latest 2.2.x branch and ran the tests several times. The only error I got was:

[ERROR] MavenReportException: Error while generating Javadoc: 
Exit code: 1 - /home/fdw/Media/Code/spring-cloud-contract/spring-cloud-contract-wiremock/src/main/java/org/springframework/cloud/contract/wiremock/restdocs/SpringCloudContractRestDocs.java:49: error: bad use of '>'
 *          .jsonPath("$[?(@.foo >= 20)]")
                                     ^

I get the same error on 2.2.x :/

How can I run them in a random order?

[marcingrzejszczak]

-DrunOrder=random

[fdw]

I've run the tests on my rebased branch several times now with -DrunOrder=random, but I couldn't reproduce this error :(
Should we try again and hope that it's fixed now, or do you have another idea?

[marcingrzejszczak]

Once I have some spare time I'll try to merge this locally and see if I can replicate this locally

[fdw]

Hi, @marcingrzejszczak, I rebased again onto the current 2.2.x branch and ran the tests again (with -DrunOrder=random, of course), but I still couldn't reproduce the problem. Do you have any news?

[marcingrzejszczak]

It's constantly failing for me locally

[ERROR] Failures: 
[ERROR]   MetaAnnotationWithResetAfterEachTestApplicationTests._03_test:85 
Expecting:                                                                                                                                                           
 <"Hello World">                                                                                                                                                     
to be equal to:                                                                                                                                                      
 <"bar">                                                                                                                                                             
but was not.                                                                                                                                                         
[INFO] 
[ERROR] Tests run: 91, Failures: 1, Errors: 0, Skipped: 0

[marcingrzejszczak]

OK I think the issue was related to https://github.com/spring-cloud/spring-cloud-contract/issues/1425 . I've fixed that one and reverted the revert and I no longer have this issue.

[fdw]

Awesome, thank you so much!