Spock: Field of type Specification loses setupSpec method when used inside another Specification

[renatoathaydes]

I have a Specification that uses another Specification in a field. I try to call the setupSpec method on it, and it's apparently missing.

Here's an example:

import spock.lang.Specification

abstract class Base extends Specification {
    def setupSpec() {}
}

final class BaseImpl extends Base {

}

class Impl extends Specification {
    Base base = new BaseImpl()

    def 'test'() {
        when:
        base.setupSpec()
        then:
        noExceptionThrown()
    }
}

This spec fails due to:

Expected no exception to be thrown, but got 'groovy.lang.MissingMethodException'
    at spock.lang.Specification.noExceptionThrown(Specification.java:118)
    at bar.Impl.test(Foo.groovy:21)
Caused by: groovy.lang.MissingMethodException: No signature of method: bar.BaseImpl.setupSpec() is applicable for argument types: () values: []
    at bar.Impl.test(Foo.groovy:19)

If I instantiate the Base type directly after making it non-abstract, then it works fine.

I know this is weird, but looks like a bug to me... is there any reason I can't use a Specification field and call its methods like this?

Tested with Spock 1.3-groovy-2.5 and 2.0-groovy-3.0 and the result is the same.

[renatoathaydes]

If I actually override setupSpec in BaseImpl, it works fine.

Adding @Shared to the field doesn't make any difference.

Removing extends Specification from Base also makes the problem go away.

[renatoathaydes]

Methods with names that are not "special" in Spock work fine... I guess this means the Spock AST transformer for some reason removes the method from the Specification? Is that really necessary?

[kriegaex]

Whoa, that's fancy. What is the use case? I am not asking sarcastically, but being genuinely interested.

[renatoathaydes]

I am using Geb, which means I need to extend GebSpec. That means I can't extend my usual base spec class (Base in this example) which "knows" to set up and tear down resources we need in many tests.
So, instead of extending Base, I decided to use composition instead, in a new Geb-specific base class. And I ended up having to do something similar to the code shown in the example.

I've managed to work around this issue for now, so this is not particularly urgent to me, at least... but when Groovy code semantics are changed like this, a lot of confusion ensues :) it would be great if breaking Groovy semantics could be kept to a minimum (though I understand Spock MUST "break" semantics to do the great stuff it does, like being able to catch Exceptions from when blocks and handle them in then blocks, which is awesome).

[kriegaex]

That is quite interesting, thank you. When assessing if and what to do about an advanced usage scenario like yours, I guess it helps Leonard and Björn understand the intent better. Maintainers usually feel much more inclined to improve something in their products, if they actually know they are doing it for a legit use case. Or maybe they can suggest an alternative approach.

[renatoathaydes]

No worries!
The solution I came up with is obvious:

class Base extends Specification {
    def setupSpec() { setupSpecification() }

    // implement the actual logic here so we can call this from other spec
    def setupSpecification() { ... }
}

[leonard84]

@renatoathaydes you might want to look at Geb 4, @erdi did a lot of refactoring to make it more composable, so you don't need this workaround. Also, manually instantiating specs and calling their functions doesn't sound like a good solution. Can't you use traits or static functions for the shared functionality?

That being said, Spock intentionally transforms the fixture methods so that it can call them in the correct order when inherited. A side effect is that users can't call them, which I would consider a good thing, as it is IMHO always a code smell.

[renatoathaydes]

Can't you use traits or static functions for the shared functionality?

No. As I explained above, it's a base class that I am trying to use, and because Spock forces tests to extend Specification, we need to call methods on the base class that does extend Specification, nothing weird about it.

[renatoathaydes]

you might want to look at Geb 4

I will when it's stable and when we manage to upgrade to Groovy 3 (we have many thousands of tests, we can't afford to use something unstable as that would force us to potentially have to make thousands of changes on upgrades).

However, given just how difficult it's being going from Spock 1.3 and Groovy 2.4 to Groovy 2.5, due to lots of small bugs just like this one, I suspect that will take a long time... going to Spock 2 and Groovy 3 is looking pretty much unfeasible right now, just too many changes that break things.

[kriegaex]

My two cents: Geb 4 is stable. Actually, there is a 5.0 milestone release already, which builds on Spock 2. For 4.0, Marcin only excluded Spock 2 because it was a milestone despite each Spock milestone being quite stable, which I think was not a good decision. BTW, running Spock 2 and Groovy 3 tests was possible with Geb 3 already, AFAIR. It should be possible with Geb 4 too, definitely with Geb 5 milestones. Anyway, I think it would make sense for you to try and upgrade to one of a more recent Geb release or milestone, instead of facing several migrations within a probably quite short timeframe.

I am surprised you are calling this a bug and are talking about many small bugs and it being difficult to upgrade at all. Maybe your project uses lots of "off the beaten path" features or maybe even undocumented stuff. This might be a good chance to reason about your tests and getting them canonicalised a bit more, while you have to migrate anyway.

Having said all that, I want to apologise, just in case I was smart-assing too much, knowing next to nothing about your project situation. I am just thinking out loud, hoping that it might inspire new ideas more than getting on anyone's nerves.

[leonard84]

I think the issue itself can be closed, if you want to further discuss certain aspects I invite you to open a discussion.

[erdi]

Yep, Geb 4.0 has been stable for a while now and it introduced GebTestManager and DynamicallyDispatchesToBrowser so that it's easy to use Geb's test framework integration without the need to extend from GebSpec - take a look at how concise GebSpec and GebReportingSpec are in that version, they serve as an example of using GebTestManager and DynamicallyDispatchesToBrowser.

[renatoathaydes]

I confused Geb 4 with Geb 5. We're using Geb 4.1. I did not know something changed in Geb 4 that allowed me to not be forced to extend from GebSpec, I will have a look into it.

Maybe your project uses lots of "off the beaten path" features or maybe even undocumented stuff.

Well, here are some things we did that caused lots of trouble:

• using final local variables in specs (used to work, doesn't anymore)
• using static methods in traits. This seems to be a problem specially when initializing trait fields.
• using static final fields in traits. For whatever reason, sometimes the fields are not found, sometimes they're even observed as null, i.e. before being initialized, which should be impossible.
• various @CompileStatic "refinements" that stopped compiling code that previously compiled.

I don't think any of these can be called "off the beaten path". We had a meeting today discussing how to avoid these pitfalls, and I presented a strategy to keep our tests reliable and easy to upgrade (by mostly avoiding the above). But our tests have been written over the course of 6 years with multiple engineers, some temps, a few juniors, who know very little Groovy and Spock... I think this is the reality anywhere with more than a couple of devs...

Everyone in the team still agrees Spock and Groovy have been great choices for writing our tests, even hinting at going back to JUnit caused most of our devs to express disagreement with the idea immediately! So, yeah, we're doing the best we can and we enjoy using Spock, but some ugly code here and there is to be expected and hopefully won't break every major release :)

[kriegaex]

Sounds like great potential new specs for Spock's test bed. If you could contribute some, probably the people on the receiving side would gladly accept them in order to analyse them and avoid future regressions.

_I myself am also mostly a "Groovy via Spock user", hence not very experienced in Groovy, using Spock mainly to test Java code. I never happened to be working in a project where the code under test was anything other than Java._

[renatoathaydes]

@kriegaex Do you mean contribute use cases to existing Spock tests? I would gladly do so (it wouldn't help only you and other Spock users if things remain working in future releases!), just give me some pointers so I know how to get started (e.g. what already exists, how to determine what addition could be useful and so on).

[kriegaex]

Yes, that is what I mean. Well, in general, logic implies that whatever causes you problems and is not a bug in your own test code but you see as a bug or shortcoming in Spock, is not covered by tests yet. Or let us use BDD terminology: It is not specified behaviour for Spock. Otherwise there would be failing Spock specs already.

You can express both faulty regressions (i.e. specification gaps) and desired features as Spock specs and add them to its test/spec harness.

Many tests live in module spock-specs, especially under src/test/groovy. Why they are split into base packages spock and org.spockframework there, I have no idea. But you find lots of specs there. E.g. I see that Björn regularly adds tests to org.spockframework.smoke when creating PRs with new features. Maybe this is where you want to specify the expected behaviour for your bug reports or feature requests.

[leonard84]

It is not specified behaviour for Spock. Otherwise there would be failing Spock specs already.

While we have a lot of test cases for things that should work, we don't have test cases for everything that doesn't.
The interaction with traits is such a thing, as they are not officially compatible with AST transforms.

[Vampire]

Btw. just to answer your initial question if you are still curious.
The method does not get deleted, but it is made private.
The cause for this is, that the methods then are not overridable and can be called targeted individually at the right time and in the right order.
If you have setupSpec in Base and BaseImpl and it would not be private, only the one in BaseImpl would be called.
By making the methods private in the transformation, they are independent methods and can be called independently.
By adding a setupSpec to BaseImpl, your example works as you can call private methods fine using Groovy, but not private methods of superclasses as those are not inherited, which is the point. :-)

[renatoathaydes]

I figured that this is what was happening. The whole issue was due to the fact that we assumed we needed base classes to make sure specs don't forget to call the setup/cleanup methods. We have been using Spock for many years, and didn't understand that Spock "overrides" the Groovy rules when it comes to the lifecycle methods, and that even if you "override" setupSpec, the base class' setupSpec method is ALWAYS called anyway. We made setupSpec final and added an abstract method like setupIntegrationTest that the specs can override instead (it does nothing so forgetting to call super in them is fine).

The problem is that when you add setupSpec via traits instead of base classes, only one of them runs.

Example:

trait B {
    def setupSpec() {
        println 'B setupSpec'
    }
}

trait A {
    def setupSpec() {
        println 'A setupSpec'
    }

}

class C extends Specification implements A, B {

//    def setupSpec() {
//        println 'C setupSpec'
//    }

    def 'test'() {
        expect: 2 + 2 == 4
    }
}

Running this prints:

B setupSpec

As B is "applied" on top of A, hence it overrrides its clashing methods.

If you un-comment the setupSpec in the C class, it doesn't compile:

The method setupSpec should be public as it implements the corresponding method from interface bar.A
. At [145:5]  @ line 145, column 5.
       def setupSpec() {
       ^
1 error

Hence, we still need base classes for this, unfortunately, and I can't see a better way to handle this than using the workaround I came up with...

Once we have all the information, everything makes sense though... so this is fine as it is.

[kriegaex]

We have been using Spock for many years, and didn't understand that Spock "overrides" the Groovy rules when it comes to the lifecycle methods, and that even if you "override" setupSpec, the base class' setupSpec method is ALWAYS called anyway

This has been nicely documented since Spock 1.3:
https://spockframework.org/spock/docs/1.3/all_in_one.html#_invocation_order

The problem is that when you add setupSpec via traits instead of base classes, only one of them runs.

I rarely ever use traits, but this is exactly what I would have expected. I understand however, that you had different expectations for some reason.

[leonard84]

If you look at Multiple inheritance conflicts in the documentation it is clearly stated that:

In this case, the default behavior is that the method from the last declared trait in the implements clause wins. Here, B is declared after A so the method from B will be picked up:

So even without Spock doing anything it wouldn't have worked. It is actually more surprising, that fixture methods from traits are picked up at all, since it is not supported by Spock.

To resolve conflicts properly would be to use different methods

trait B {
    def setupSpecB() {
        println 'B setupSpec'
    }
}

trait A {
    def setupSpecA() {
        println 'A setupSpec'
    }

}

class C extends Specification implements A, B {

    def setupSpec() {
        setupSpecA() 
        setupSpecB() 
        println 'C setupSpec'
    }

    def 'test'() {
        expect: 2 + 2 == 4
    }
}

[renatoathaydes]

Just to clarify: I was showing how traits don't work in this case because someone here suggested we try that.

We use base classes, as I've been saying, and that's why I faced the problem of trying to "merge" two different base classes, and why I had to call setupSpec from another class. Hope that makes sense now.

This has been nicely documented since Spock 1.3:

I understand however, that you had different expectations for some reason.

@kriegaex no need to be rude, man... yes, everyone should read the docs fully, and know everything... sorry that we didn't. But that was mentioned just to try to show where we're coming from: we still need those base classes anyway as I pointed out with the trait example... I didn't expect the traits would work or anything, I was just pointing out that it won't as someone here suggested we try that.

[renatoathaydes]

To resolve conflicts properly would be to use different methods

@leonard84 yes, as I already said we're doing!

[kriegaex]

@kriegaex no need to be rude, man...

Pardon me? Who is being rude now? I merely pointed to the documentation in order to clarify that Spock does not perform any dirty, undocumented tricks. I meant to do you a favour, giving you a chance to read it.

I understand however, that you had different expectations for some reason.

This was not meant sarcastically, either. I am sorry if I cannot convey any tone of voice or mimics in a comment, so you must have misunderstood me. Did I give you any reason to think that I meant to be rude? Was I ever rude or condescending towards you before? Actually, IMO you know much more about Groovy in general and probably also a bit more about Spock than me. Aren't you the guy who designed this reporting tool? Sorry for stepping on your toes, mentioning that you didn't know something that is documented. By no means do I think I fully understand the ins and outs of Spock or know the documentation by heart. So I want to extend my sincere apologies, if something I wrote came across in the wrong way.

[renatoathaydes]

@kriegaex ok, I'll do my best to keep things cordial as we're all trying to enjoy what we're doing and learn from each other, not attack each other for no reason whatsoever.

Did I give you any reason to think that I meant to be rude?

I might have overreacted, sorry about that... I've been working for a full week (finished yesterday, finally!!!) just on upgrading Spock, Geb and Groovy so our tests are kept well maintatined... there's probably over 10,000 tests (I haven't counted the grand total lately) so changing even small things can take a lot of effort.

With that in mind, it's a bit disheartening when someone tells you that you should "just" not do things wrongly (as if I had written everything myself), or "just" use the latest versions of everything, which is how some advice in this thread has come across.

OTOH, I can simpathize, as a maintainer of a few libraries myself (yes, I maintain spock-reports, for the love of it as I am not actually using that anymore in any project, but I know that a lot of people are), with the fact that it would be nice if people just read the docs and used the libraries properly.

Hopefully we can all do better in the future 😃 .