Badger: Badger Mock

[thomashargrove]

Just wondering if there is a mock/fake implementation of Badger that we can use for testing our code that integrates with badger. Possibly one that is backed by a simple in-memory store?

If one does not already exist and we make one, would there be interest in contributing it to this repo?

[campoy]

This is an interesting idea.

What client are you currently using? I wonder if we could create a gRPC server that can be easily started for this kind of tests.

[thomashargrove]

We are just using badger directly in our process. There is no real client that I see.

[arsham]

Until badger provides mock/fake constructs, you can do this. I'm not sure if this would work for you, but I've had good results:

// Badger wraps an actual badger connection.
type Badger struct {
    db *badger.DB
}

// NewBadger opens a database at loc.
func NewBadger(loc string) (*Badger, error) {
    opt := badger.DefaultOptions(loc)
    opt.Logger = logrus.StandardLogger()
    conn, err := badger.Open(opt)
    if err != nil {
        return nil, errors.Wrapf(err, "opening db at: %s", loc)
    }
    return &Badger{
        DB: conn,
    }, nil
}

// Update issues an Update call on the underlying connection, with provided txn
// transaction.
func (b *Badger) Update(fn func(txn Transaction) error) error {
    return b.db.Update(func(txn *badger.Txn) error {
        return fn(&transaction{txn: txn})
    })
}

// View issues a View call on the underlying connection, with provided txn
// transaction.
func (b *Badger) View(fn func(txn Transaction) error) error {
    return b.db.View(func(txn *badger.Txn) error {
        return fn(&transaction{txn: txn})
    })
}

// GetSequence returns the output of GetSequence from the underlying connection.
func (b *Badger) GetSequence(key []byte, bandwidth uint64) (Nexter, error) {
    return b.db.GetSequence(key, bandwidth)
}

// A transaction encapsulates the badger Txn.
type transaction struct{ txn *badger.Txn }

func (t *transaction) Set(key, val []byte) error    { return t.txn.Set(key, val) }
func (t *transaction) Get(key []byte) (Item, error) { return t.txn.Get(key) }
func (t *transaction) Delete(key []byte) error      { return t.txn.Delete(key) }
func (t *transaction) NewIterator(opt badger.IteratorOptions) Iterator {
    return &iterator{
        itr: t.txn.NewIterator(opt),
    }
}

// An iterator encapsulates the badger Iterator.
type iterator struct{ itr *badger.Iterator }

func (i *iterator) Item() Item                   { return i.itr.Item() }
func (i *iterator) Close()                       { i.itr.Close() }
func (i *iterator) Seek(b []byte)                { i.itr.Seek(b) }
func (i *iterator) ValidForPrefix(p []byte) bool { return i.itr.ValidForPrefix(p) }
func (i *iterator) Next()                        { i.itr.Next() }

In your application accept an interface that implements Badger (from above code) and stub that one.

[manishrjain]

The way our unit tests work is they create a temporary directory, run Badger itself, optionally close it and finally that directory gets deleted. If that's slow, you could use /tmp which is typically run off RAM.

[campoy]

Sorry, @thomashargrove

I somehow thought this was on the dgraph repo - therefore my question about clients.
As Manish mentions, we don't use mocks ourselves since it's quite easy to run off disk or RAM.

Does that solve your problem, or are you interested on something else?

[thomashargrove]

Thanks for all the replies. While spinning up badger on a RAM disk is not terribly slow, it still would be an issue for a lot of tests. We will build the memory-backed fake and share it when finished to see if others find it useful.

[thomashargrove]

Also @arsham thanks for the snippet. We will use that as well for a subset of our tests where it works better than a fake.

[campoy]

Thanks, I'll close this issue then.

Please do share your fakes if you'd be ok with them being part of the project!

[thomashargrove]

Here is the mock we built for the subset of the API we use:

https://github.com/salesforce/sloop/tree/master/pkg/sloop/store/untyped/badgerwrap

It contains a fake in-memory implementation in mock.go and pass-through in badger.go. This helps us in making fast self-contained unit tests for code that works on badger transactions.

Also, if anyone is interested in a visual tool to debug historic issues in Kubernetes please check out Sloop!

[burdiyan]

As an idea, what if Badger would use an abstraction for the filesystem instead of directly using real implementation from os package. Then anyone could pass a fake implementation of the filesystem while using the concrete implementation of Badger with all its functionality.