Google-cloud-go: Provide interface for mocking out libraries such-as datastore.Client through dep injection

[cobookman]

In unit tests its nice to be able to use dependency injection + mocks on databases / remote resources. For example take datastore, say I want to mock out a datastore Get to test my code. If the datastore library provided a Datastore interface I could simply dependency inject a mock datastore client.

While you can write these interfaces yourself, it'd be nice for the library to provide them.

Example. Say there was a interface for the datastore.Client's methods under datastore.ClientInterface.

func getDoc(ctx context.Context, client datastore.ClientInterface, id string) (*Doc, error) {
  k := datastore.DecodeKey(id)
  c := new(Doc)
  err := client.Get(ctx, k, c)
  return c, err
}

in the unit tests....

type MockClient struct { 
  datastore.ClientInterface
}

func (m MockClient) func Get(ctx context.Context, k *pubsub.Key, d interface{}) error {
   d.(Doc).Text = "HI WORLD"
   return nil
}


func TestGetDoc(t *testing.T) {
  ctx := context.Background()
  mc := MockClient{}
  doc, err := getDoc(ctx, mc, "FooBar")
  if err != nil { 
    t.Fatal(err)
  }
  if doc.Text != "HI WORLD" {
    t.Fatal("didn't pull the doc correctly")
  }
}

[jba]

Unfortunately, your example wouldn't compile, because MockClient doesn't implement all the datastore Client's methods.

There's a workaround for that (see below), but in this case it would be simpler to have

type Getter interface {
    Get(context.Context, *datastore.Key, interface{}) error
}

func getDoc(... Getter ...)

or even

func getDoc(... interface { Get(...) } ...)

In any case, as a general rule we try to avoid writing interfaces unless we really need them for something in production code. It makes the code easier to read and understand.

Also, I'd recommend the datastore emulator. Then you can write unit tests (or at least tests that don't go off-box) that look exactly like tests against the real datastore service.

The workaround for having to implement every method of an interface FooInterface is embedding:

type FooMock struct {
    FooInterface
}

FooMock has all the methods of FooInterface, but panics when they're called. You can "override" whichever methods you like with working implementations.

[cobookman]

Also, I'd recommend the datastore emulator. Then you can write unit tests (or at least tests that don't go off-box) that look exactly like tests against the real datastore service.

that makes running unit tests on something like travis hard to do.

Also I dont think its possible to do that against all of the cloud services. And Ideally one could run a unit test on a function which integrates with many cloud services.

func getDoc(... interface { Get(...) } ...)

That would get kind of out of hand having to dependency inject every single datastore.Client method.

Unfortunately, your example wouldn't compile, because MockClient doesn't implement all the datastore Client's methods.

Also edited the above with the workaround mentioned below for clarity to others viewing this thread.

[derekperkins]

Also, I'd recommend the datastore emulator. Then you can write unit tests (or at least tests that don't go off-box) that look exactly like tests against the real datastore service.

that makes running unit tests on something like travis hard to do.

It's easy to install those dependencies on any CI, usually a one-line bash command in your setup scripts.

[cobookman]

It's easy to install those dependencies on any CI, usually a one-line bash command in your setup scripts.

I'll do that for now. But was hopping I could simply issue a go test to test the code w/o having a bunch of setup scripts.

[cobookman]

How best does one startup / teardown the emulator before and after each unit test though. Aka give the datastore emulator a blank state before every unit test.

Also this does get kind of crazy when say I want to run unit tests on a program which uses pubsub, datastore, gcs, big query, .....etc.

This also starts to breakdown for say testing pubsub. I want to test if a function emits a pubsub message. Am I supposed to in my unit test listen for sent pubsub messages? Creating a mock would allow me to just double check that the right value was sent to pubsub.

[derekperkins]

@cobookman I think there's an endpoint for clearing the whole thing, but what I think works better is using namespaces. If you use a UUID generator inside each unit test to create a unique namespace, everything runs completely separate without the need to clear anything. That's probably the fastest way to do it.

[derekperkins]

Also this does get kind of crazy when say I want to run unit tests on a program which uses pubsub, datastore, gcs, big query, .....etc.

That's our main problem with testing currently, we're just using so many parts of the ecosystem that it's hard to keep important functions inside a true unit test. It feels like full integration testing is the only way to really stay on top of things.

[adams-sarah]

Closing for now. Feel free to reopen if necessary.