Testthat: No proper DSL for nested example groups within `describe` blocks

[louis-vines]

I've been using testthat for several years now in package development and recently came across the BDD flavour of writing tests with:

describe("...",{
  it("...", {

  })
})

Having written Ruby code in the past I obviously like this as its really reminiscent of the RSpec way of doing things.

However I can't really see how to properly structure nested example groups. For example in RSpec if we want to describe the behaviour of a function in multiple different contexts we write as:

describe "something" do
  context "in one context" do
    it "does some behaviour" do
    end

    it "does some more behaviour" do
    end
  end

  context "in another context" do
    it "does different behaviour" do
    end
  end
end

It feels like the BDD approach in testthat is missing that middle context keyword (I know there is a context function in testthat but it's usage seems much more for grouping large groups of tests into a relevant section).

To make this point clear if I wanted to nest this same example as above using the testthat DSL I assume from the describe documentation that I should nest my examples as so:

describe("something", {
  describe("in one context", {
    it("does some behaviour", {
    })

    it("does some more behaviour", {
    })
  }) 
  describe("in another context", {
    it("does different behaviour", {
    })
  }) 
})

But this doesn't feel write as I feel like describe should be used to denote a thing (function probably) we are describing, while the inner blocks here are grouping together examples based on whether a context is true or not.

Should there be an extra keyword/function (like context but probably a different name as this is already taken) to group together nested examples within a describe block?

[louis-vines]

FYI: if this seems like a sensible proposition I'm more than happy to give the implementation of this ago, if given some guidance - I haven't contributed to any tidyverse packages before so would need to get up to speed on best practices.

[hadley]

I think describe() is effectively retired in testthat — we don't ever use this style, so it really belongs in its own package with a maintainer that cares about this style of testing.

[cderv]

@louis-vines if this could help, I know this is used by some CRAN package authors like ompr
https://github.com/dirkschumacher/ompr/blob/master/tests/testthat/test-model-api-milp.R

There are others you can find if you look in cran source repo on github.
If you are interesting in investing in this DSL for a new independant package, you could get in touch with their author to see if they would continue to rely on it or not.

Just a thought. 🤔

[kaz-yos]

This is very nice! I used this style in emacs stan-mode via emacs-buttercup (emacs port of Javascript's Jasmine), so it looks very natural to me.

[jimhester]

I actually did use describe / in for all of the tests in fs, I kind of like it.

[hadley]

To be clear, I'm not proposing that we remove describe() + it(), but I will not implement any new features for it.