Marten: Async Daemon Improvements

[jeremydmiller]

For the moment, I'm combining all the Async Daemon improvements in this issue because I think everything is going to be closely related. This issue will only refer to an improved version of today's polling daemon, but should be built in such a way that we could introduce replication slots or messaging technologies as an alternative to polling. The desired improvements and new functionality identified so far is:

• Should actually be adequately documented finally
• Easy recipe for integrating the daemon into .Net applications using the generic HostBuilder as an IHostedService
• Must support multi-tenancy (originally #952).
• Ability to trigger a projection rebuild and resumption without downtime (originally #596)
• Support clustering and failover in server side applications (originally #969)
• Zero downtime projection rebuilds
• Optimized projection rebuilds
• Higher throughput
• Take advantage of logical event stream sharding to parallelize asynchronous projection building
• Should be resilient to event ordering issues

Optimization Ideas

• The runtime code generation of the projection definitions should make the projection application faster in general
• Look to parallelize the CPU intensive tasks of doing document changes in memory within a single EventPage
• Do a look ahead in the event streams and look for an event that would cause a delete of the aggregate. If so, bypass
  any of the other event updates and just delete the document
• Bypass DocumentSession and work at the IStorageOperation and UnitOfWork abstractions. That'll lighten the load as
  compared to the current projections. That also makes the multi-tenancy story much easier

Rebuilding Projections

Fill this out when Oscar, Babu, and Jeremy have our call.

[jeremydmiller]

WORK IN PROGRESS, MORE SOON...........

Runtime Architecture Concept

• The Async Daemon should be a BackgroundService that's easily spun up by the new HostBuilder integration
• Inside of that background service should be

Notes

• When the daemon spins up in a new instance, it needs to register itself as a

Ideas

• Use the Postgresql Notify mechanism as a way to send messages between nodes?????
*

[jeremydmiller]

I'm closing this in lieu of all the new, finer grained issues