Eventstore: What is the best way to clear an event store for unit tests?

[kmalakoff]

I am using the node client and ran into an issue that the author suggested I raise here.

When I hard delete streams and then subscribe to the $all stream, events in those deleted streams are emitted. The admin UI works as one would expect, eg. blocks access to view hard deleted streams. I would expect that events in hard deleted streams are no longer emitted even when subscribing to $all.

For a little background, I am writing unit tests and trying to clear the EventStore between runs. I opened 3 issues in the node client to figure out a good way to clear the event store / reset the event store to a known state programatically.

• https://github.com/nicdex/node-eventstore-client/issues/40
• https://github.com/nicdex/node-eventstore-client/issues/39
• https://github.com/nicdex/node-eventstore-client/issues/38

Maybe I should ask here: what is the best way to clear an event store for unit tests?

[gregoryyoung]

So $all is a special stream usually used for things like replication purposes. It is optimized for such use cases. There are two ways you should look for tests. 1) use specific streams without any dependency 2) use --mem-db and throw away the node between tests.

[kmalakoff]

Hi Greg,

Thank you for the quick response.

1) is probably not a great idea because my codebase relies on building read models using the $all stream which I would like to exercise and they will slow down over time as previous test data accumulates. I also have a test suite for restoring from checkpoints where I prefer to assume a known, clean start state.

2) is --mem-db and throwing the node away between tests something that can be run programmatically (eg. supported by the node-client)? I'm using Docker to standup the test EventStore container so I'm assuming the --mem-db flag will be there, but throwing away the node from my API container via the node-client sounds like it might not be supported. Before I discuss with the node client author, I'd like to confirm this is possible programmatically.

I was explaining to to the author of the node client that I would ideally like clearing the event store programmatically for testing support as a first class citizen. For example, perhaps you could expose an API for clearing the Event Store if an opt-in configuration setting (eg. test: true, test_reset_available: true) is provided to the Event Store. I'm not sure about the complexity on your side, but it might be a good solution rather than trying to layer this on top of production APIs and processes. Just a thought...

Thank you!

[pgermishuys]

@kmalakoff closing this issue for now. Please don't hesitate to reopen if you have any further questions/issues related to this.

[kmalakoff]

@pgermishuys as you can see from the 3 issues I opened on the node event store and from my feedback to @gregoryyoung that an acceptable solution has not yet been presented and clarified so please reopen.

The important part is that there should be a way provided to programmatically reset the event store to a state where all user data has been cleared specifically if the event store is run in a separate container / on a separate server for test.

• Hard delete and scavenge doesn't work: https://github.com/nicdex/node-eventstore-client/issues/38
• Hard deleted events still are emitted on $all: https://github.com/nicdex/node-eventstore-client/issues/39
• Just generating new UUIDs will accumulate data which will slow down tests and cause developers to create accounting workarounds for snapshot testing given the lack of deterministic state when tests start.
• There may not be a way to programmatically clear a memory backed database in another container (I'm still waiting to hear about this).

Please bear in mind that testing is very different than production...thank you!

[gregoryyoung]

Just generating new UUIDs will accumulate data which will slow down tests and cause developers to create accounting workarounds for snapshot testing.

Not really. You might try measuring.

There may not be a way to programmatically clear a memory backed database in another container (I'm still waiting to hear about this).

A restart of an in-memory node is 100% assured to drop all data.

[gregoryyoung]

Also in general as a strategy you should not be running unit tests through your db but having an in memory equivalent. Very few tests should actually be hitting the database.

[kmalakoff]

@gregoryyoung unfortunately, I don't think this is going to be a fruitful framing of debate. Maybe we can try a different angle?

We could go back and forth on whether accumulating data by generating new UUIDs is a good testing pattern given the potential side effects, and we could debate the pros and cons of working around and the impact of those side effect (FYI: I would argue this is an anti-pattern).

Also, we could discuss whether tests should exercise external services (FYI: I have already implemented an in memory mock for unit tests, but also want to perform integration tests and be able to also run those unit tests with an event store if I please to confirm my mock works like the node-client and event store together).

If I was to try to boil this down to a clear need and solution, I would argue that "EventStore is missing drop database".

Hopefully it is clear that I have been trying to give examples to convince you to implement drop database. I'll try to give another...I've been working with Adaptech Solutions and implemented seed database scripts to set up two services to known states with some necessarily identical UUIDs across them so that I can develop and test some integration scenarios. Whenever one of the developers changes commands / events or as I iterate through writing and debugging the seeding scripts, I need to manually go to the event store, delete the data folder, start everything up again, run seed in both services, and then continue developing. It would be so much easier to just have a single seed script that drops the database, seeds the data to a known state, and allows me to spend less time on non-value adding activities.

We could debate other ways to do this (for example, adding event versioning, creating a cross-platform script that handles all of those steps across services by knowing possible locations of data directories, etc), but I'd prefer not to. 😉

I hope that I have convinced you that a flavour of drop database is valuable enough to developers! If not, maybe we should discuss and debate specifically the pros and cons of drop database since I think it is what I want and need. Of course, I could be wrong, but it seems like it will provide me the sort of opportunities and solutions to problems that I'm expecting and hoping for. 🤞

[gregoryyoung]

--mem-db and restart does a "drop database" it will also prevent any files from being written (so nothing to clean up).

[kmalakoff]

Awesome and thank you for looking this up!

Unfortunately, I have searched the documentation and cannot find a reset API call so that it can be programatically triggered between tests to reset the event store in a separate Docker container from where the tests are running.

Also, why or why not should there be a drop database programatic command / API for all storage variants?

@hayley-jean I'm not sure yet if this is a question or a feature request or a documentation request. Depending on whether a remote API exists or not and whether I can convince @gregoryyoung that there should actually be a drop database not even for non --mem-db databases, it will change the categorization.

[hayley-jean]

Hi @kmalakoff

Currently there is no drop database functionality that can clear the Event Store database while it is running.

At the moment, your only options to start with a fresh Event Store DB are :

1. Start Event Store with the --mem-db flag.
2. Run your tests.
3. Stop and start Event Store.

Note: You can shut down Event Store with an http request to http://<ipaddress>:<port>/admin/shutdown and if you are running Event Store in upstart, it should start the node back up for you.

1. Start Event Store with the --db flag set to a known location.
2. Run your tests.
3. Stop Event Store.
4. Delete the db.
5. Start Event Store again.

[kmalakoff]

Thank you for clarifying...

It sounds like I have control to get a docker image working with something like upstart and calling the shutdown command, but I do not have control over deleting the database from my API container running the tests.

I've opened a feature request for Drop Database: https://github.com/EventStore/EventStore/issues/1335 so we can explicitly debate its merits.

I recommend that we leave this open until there is a full solution to this problem (eg. a general purpose drop database may not be agreed as the solution and instead a partial solution like an API for deleting a memory database may be recommended instead, at least in the short term). Unless there is a fully programmable solution for my use case with the memory database option? (eg. I'm treating restoring to a specific, known state as my acceptance criteria)

[dimaip]

I'm coming from exactly the same perspective as @kmalakoff and I would also love to have that API to programmatically reset the store.

[iberodev]

And is there a way to embed the EventStore.ClusterNode dll in a nuGet package that I could add as a dependency to my integration tests project so that I don't need to assume there is a server installed and running on the build/test machine? That'd be ideal, so that I could start and stop the in memory server directly from the test fixtures.

Anybody has an example on how they're doing it?

[jen20]

@iberodev there is a package to do that - EventStore.EmbeddedClient, but it relies on your running either .NET Framework or Mono currently, since the server isn’t compatible with Core CLR.

[iberodev]

@jen20 thank you. I use the Core CLR (netstandard2.0) so I'll have to look for another approach for integration tests then.

[jageall]

there is an example here of how you can do test using docker controlled from the unit tests:
https://github.com/linedata/reactive-domain/blob/33d04df1b959fbab9e01af47d5f22a5d9d1d374a/src/ReactiveDomain.Testing/EmbeddedEventStoreFixture.cs

[jageall]

The current design of Event Store does not and cannot support the concept of resetting the database while on line. Offline truncation is the only form which is supported either for resetting logs during failed elections or for resetting the data store. The only option here is to take control of a docker container for tests.