Lagom: Back office UI

[jroper]

Lagom is in a perfect position to be able to provide a back office UI to itself, in particular to its persistent entities. Unlike using CRUD relational database schema, where there exist many tools to assist users in manually manipulate the data when necessary (most obviously, plain old SQL), event sourcing makes this far more difficult due to the data being stored serialized inside columns of the database.

Given that Lagom already has serializers to persist events to JSON (of course, this can be overridden, and protobufs used instead, but the default is JSON and JSON is not a bad option anyway), we should be able to easily provide a backend UI that allows persistent entities to be interrogated directly, using the JSON serializers to render state and events in a human readable format, including the following functions:

• List all entity types.
• Allow selecting an entity type and entering an entity id.
• List the current state of any entity given the above type/id.
• List all events for that entity, paged.
• Display the state of an entity given an event offset or timestamp.

We could also allow custom rendering templates that take the deserialized event/state and returns HTML.

We could also allow write operations, including issuing commands directly to an entity from the back office UI, as well as directly appending events that a command wouldn't otherwise allow (this could be useful in scenarios where a bug has caused an entity to get into a corrupt state, and once that bug is fixed, there are a very small number of instances of entities where new events need to be appended where it's not worth coding up a new command just to fix the corruption).

Initially this could be provided by the user manually coding the JSON for the command/event, which we could then do a round trip validation of using configured serializers, including returning the round tripped JSON back to the user to confirm that that's the command/event that they want to issue (and also potentially doing a dry run of applying the command/event, and returning the state to the user for them to confirm that that's the state that they want), before issuing it. Optimistic locking (including the current offset) can be used to detect concurrent modifications between when the user validated the command/event and submitted it. We could also possibly read the schema of the events/commands (for Scala, will require play-json schema support, for Java, will require Jacksons schema support) to assist the user in generating command/event json.

[ktoso]

Such could be implemented rather in https://github.com/akka/akka-management and then utilised in any play or lagom app I'd propose.

[jroper]

I think there's a number of features of this that will depend on the restrictions that Lagom imposes and functionality that Lagom provides in order to provide a useful UI. For example, this will only really work if JSON is used as the serialization format, and some features will require more detailed interrogation of the JSON schema than Akka serialization will allow, and so it will likely require direct interaction with Lagom's JSON serializers. Another issue is that vanilla Akka peristace doesn't have any concept of what "the state" of an entity is, that's just implied on an actor, and so there's no way that will be able to be serialized and sent (of course, I think Akka typed persistence does, so maybe that will be able to support that).

So I'd say before it can be done in akka-management, some of Lagom's features will need to be moved to Akka. So I'd say we should implement in Lagom first, and then use that to identify which Lagom features are needed in Akka, to work out what needs to be moved to Akka. It would actually be good to have JSON support in Akka persistence - so that persistence plugins can store events natively as JSON rather than byte arrays (since many databases do have native support for JSON).

[kolemannix]

My team would derive huge value from this. We've invested some energy in building a simple back office HTTP / JSON api for our persistent entities using Akka Persistence Query directly: we create a plain akka-http service on an internal port and expose some endpoints for fetching paged events, and current entity state, as well as sending certain commands directly to some PersistentEntities.

Unfortunately, by using Akka Persistence Query directly for this. we are depending on implementation details of Lagom which is not ideal.

The direct appending use case for manually healing an entity would be fantastic. Unfortunately, bugs do occur in production, and with event sourcing they can often be more costly than with a fully mutable model where a simple migration is capable of wiping the mistake away.

In summary, +1