Mobx-state-tree: [Suggestion] Using MST models for generative property-based testing

[lostfictions]

One thing I like about MST is how its modeling of data enables a lot of different things: runtime and design-time checking, serialization, etc.

Another potential use case for MST models that I haven't seen discussed -- but that seems like low-hanging fruit -- is simplifying or automating generation of test cases for property-based testing in the style of QuickCheck or Hypothesis (or jsverify and testcheck.js in the JavaScript world.)

(For folks not familiar, here's an article that makes a great case for property-based testing in JavaScript.)

As it stands, testing MST models with a library like jsverify right now requires duplicating a lot of the same type information to generate snapshots. Take the basic TodoStore example from the readme:

const Todo = types.model("Todo", {
  title: types.string,
  done: types.boolean
}).actions(self => ({
  toggle() {
    self.done = !self.done
  }
}))

const Store = types.model("Store", {
  todos: types.array(Todo)
})

const jscTodo = jsc.record({
  title: jsc.string,
  done: jsc.bool
})

const jscStore = jsc.record({
  todos: jsc.array(jscTodo)
})


describe("todos", () => {
  jsc.property("toggling a todo twice", jscTodo, t => {
    const todo = Todo.create(t)

    const cloned = clone(todo)
    cloned.toggle()
    cloned.toggle()

    return deepEqual(getSnapshot(cloned), getSnapshot(todo))
  })

  //....
})

Note how jscTodo and jscStore are essentially identical to their MST counterparts Todo and Store. This might not always be true -- for example, MST refinement types might need additional code to describe how to generate conformant values -- but it's worth noting that most types in jsverify and co. have direct equivalents in MST, including unions, literals, etc.

I don't have a concrete suggestion of how generation could be integrated in MST, but I wanted to float the idea as it seems like a potentially really powerful and valuable one that aligns closely with MST's goals of providing great DX.

[mattruby]

Wow! That's a really neat idea!

[lostfictions]

I took a shot at a proof of concept today! It's a quick-and-dirty implementation that just tries to recursively convert MST types into jsverify ones, but the results already seem promising. You can see some tests here and the implementation here.

It's really quick and compact to make some assertions and let the generator come up with all the possible combinations of test cases (below tests using example models from the MST repo):

property('marking all todos completed leaves none active', TodoStore, store => {
    store.completeAll()
    return store.activeCount === 0
})

If you're not mutating the models but instead just want to verify properties of, say, a view function given some random state, it's even easier and more compact:

property('box width is always greater than 0', Box, box => box.width > 0)

You can combine generators and models freely, and TypeScript inference works:

property('boxes given a positive dx will have a greater final x', Box, jsc.nat, (box, dx) => {
    const startX = box.x
    box.move(dx, 0)
    return box.x >= startX
})

Thoughts? This is something that might be nice to have in the library itself, especially since it's dependent on some internal functions (I had to dump a copy of mobx-state-tree into my repo since the npm module includes definitions for stuff like type-flags, but -- maybe erroneously -- no implementations...)

[luisherranz]

Well, that certainly looks awesome.

[mweststrate]

Closing for now, as I think this is something for outside the core package, but not that the recently added reflection api's should make this a bit easier already