Java: Should we use a test generator?

[FridaTveit]

As suggested by @beatbrot:

How do you think about creating Unit-Tests out of the canonical-data.json?
I could work on that, If you want me to :)

What do we think about creating an automatic test generator @exercism/java? 🙂

[sjwarner-bp]

That would be interesting. I think the F# track have had good success with this.

Depends how much we already deviate from the current suites though :thinking:

[FridaTveit]

Personally I think creating a test generator would be an interesting task and it could make it easier to maintain the Java track in future.

However, most of our good first patch issues and hacktoberfest issues come from updating the tests. I think it's important we have ways for new people to contribute so if adding a test generator would make that more difficult then I would be less keen to add one.

And with our current setup with version files and script to check if they're up to date, maintaining the tests isn't too difficult 🙂

Also as @sjwarner-bp mentioned, it would make it more difficult to deviate from the current suites. I don't think we do that often, but we do at least once and it's nice to have the option.

So I think there would be pros and cons to adding a test generator 🙂 It might be worth looking at any other tracks (e.g. F#) to see how theirs works?

[beatbrot]

Question is also: Do we want to have a generator that generates actual Java Code with @Test methods?

Or would we rather want to have a @Test-Method that processes the json-File?

[Smarticles101]

Personally, I would say a test generator would be quite difficult to make (as mentioned here).

Would it be worth making a test generator? Essentially we would be making a thing to maintain the tests. Instead of us maintaining the tests we would have to maintain the thing that maintains the tests.

Like @FridaTveit mentioned, other tracks have test generators which would be hugely beneficial to look at.

Another question that's important is what language would we build the test generator in? At first one might assume Java, but since it's not an exercise, it doesn't have to be written in Java. There's probably better languages to write it in. The tests use bash to run.

Question is also: Do we want to have a generator that generates actual Java Code with @test methods?

Or would we rather want to have a @Test-Method that processes the json-File?

@beatbrot how would it process the json file? I imagine a true "test generator" would generate actual java code, but I'm curious to hear your thoughts on a json processor

[beatbrot]

@beatbrot how would it process the json file? I imagine a true "test generator" would generate actual java code, but I'm curious to hear your thoughts on a json processor

Well, another alternative is to let an abstract class handle the loading from the JSON-File. For each entry in the JSON file, the class would call an abstract method test(...) that has to return the value calculated by the solution. This is then compared to what was expected by the json-file.

The abstract class should make use of generics similar to how java.util.function.Function does it. We might also have a BiAbstractTest, that accepts two input parameters.

I agree that this class would be rather complicated to implement, considering that these are beginner exercises, however: If the test-writer does not care about the internals of this abstract class, creating tests would be much quicker!

[jmrunkle]

Closing this as an abandoned idea. Since we can often create better assertions ourselves manually for each test class, I am generally against this approach anyway (it is also likely to require more work from the maintainers).