Eventstore: The .NET Core MegaThread(TM)

[jen20]

This thread is intended to summarise progress, act as an umbrella thread, and be somewhere to point people asking about .NET Core support for Event Store.

Work to date and roadmap

Getting to Core CLR is a priority for us internally. To repost the current state from the response to pull-request #1680 which summarises work to date and the roadmap:

Currently @jageall has done some work to target the Client API at .NET Standard 1.6. Urgent patches excepted, the next release of the EventStore.Client package will work across .NET Framework 4.6.1, Mono 4.6 and .NET Core 1.0 and above as a result. The current Core package will be deprecated, but likely have a new version which includes the main client package as a dependency in order not to break people currently depending on it. This is a divergent approach to that taken in this pull request, though I am going to leave it to James to go through and work out how to reconcile the two.

If you need to run the client on an earlier version of .NET or Mono, you will need to stick to the last release, or build it yourself - .NET 4.6.2 and Mono 4.6 will represent the earliest supported versions.

As @bartelink says, I disagree that current state [of the client] was a mistake. The alternative at the time would have been no support for .NET Core as a client rather than the work now proposed now being done then. However, now that the landscape is clearer and there is less churn in the world of .NET Core, we need to move to more standard approaches.

Unfortunately it is not as straightforward as it sounds to "just port the client": the client internalises it's dependencies, but has internals visible to the rest of the system for test purposes. Consequently, many of the server dependencies also require upgrading (which is part of the work done). As part of this, we are switching the project formats to use the newer SDK csproj format, and to pulling dependencies from a public MyGet feed instead of checking them into this repository directly. Some of this can be seen in #1703, some is not yet public. This is not a straightforward substitution either - our build system is complicated by the need to build native code (V8) across multiple platforms, and to statically link binaries for Linux.

Once the client is published, we intend to port the server to .NET Core also. Although not completely decided, it is likely that Mono and .NET Framework support for the server will be dropped for the next major release of Event Store, and that .NET Core will be the only supported runtime across all of the operating systems we support for running the server, and likely also for the embedded client. This will allow us to aggressively adopt attractive new constructs in .NET Core such as Span without having to give too much thought to compatibility with other runtimes.

Again, this is not a straightforward process: much of the code base has (mistakenly) used Mono as a proxy for "Not Windows" for several years now, and that all needs to be corrected. Our packaging will also change substantially, as .NET Core does not yet have the ability to statically link the runtime. There will likely be several release candidates before the next major version of Event Store ships, and everything here is subject to change based on further research or (constructive) discussion.

Finally, as part of porting the server to .NET Core it is likely that we will make it more modular. Not everyone uses projections, and certainly not everyone wants to build V8 in order to work on the managed core of Event Store or the client. It is likely that both projections and persistent subscriptions will be moved to a plug-in model rather than being part of the core system in the next major version as a result, and we will likely extract interfaces which allow other aspects of the system to working a similar model in future - we have a number of commercial-only projects under way which will also benefit from this: asynchronous geo-replication being one example.

Event Store is one of the most complex open-source .NET projects around thanks to the long history of support for running on Mono and Linux (most of our production clients have been using this for years). Because of the critical nature of a database, it is also not desirable to rush this work. Consequently it is going to take longer than some people think it should - we acknowledge that there is a desire to see this work done and that there are benefits to doing it, but this is not going to be a "quick fix".

You'll likely see incremental work towards this being merged into the codebase over the coming days, weeks and months and we will be creating a new label and milestone for .NET Core related work to help track this.

Related issues and pull requests

• #1710 starts the work to move to the new SDK project format and switch to NuGet for packages (via a MyGet feed). It is currently blocked on us upgrading various aspects of CI, but should be merged soon.
• #1703 is superseded by #1710 but is part of the same work.

[haf]

Could you make the $ce-, $et-, $streams and $all projections built in, if you decide to remove V8, please.

Also, perhaps fix your existing client for .net core before migrating the rest of your server? It obviously connects at least half-successfully to the server, so perhaps work through that state machine to start with?

[bartelink]

Also, perhaps fix your existing client for .net core before migrating the rest of your server? It obviously connects at least half-successfully to the server, so perhaps work through that state machine to start with?

AIUI https://github.com/EventStore/EventStore/pull/1680 can accomplish what you ask but is not being used as part of this set of work - the issue with EventStore.ClientAPI.NetCore is that clients <= 4.1.1 have concurrency bugs, the fixes for which (and more and less significant new features) are not included in that. Per the above, I believe a netcore client is the first milestone though ?

[jageall]

$all isn't a projection, and the others you mention are all C# and dont run in v8 in any case. For example, $streams is here, and the other standard projections are also in the same folder.

https://github.com/EventStore/EventStore/blob/master/src/EventStore.Projections.Core/Standard/IndexStreams.cs

I am not sure that saying the client connects half-successfully is really even vaguely accurate as it has 1000s of hours of use. I have observed problems with reconnection and some settings not working as expected. Although that is generally poor documentation of what those options are doing. That said, if you are talking about the core client it isnt 100% up to date with the full .net client

The first part of what we are trying to do is get both the full client and the core client building off the same codebase with the same tests, which will eliminate differences in the code base as a source of problems.

I suspect that you may also be referring to the other issue that you commented on around persistent subscriptions. If you have specific scenarios in mind please make sure we have an issue for them.

I personally would like to add some more testing around this area and possibly adding some other options to better support running with a connection in a webserver. We are talking about getting long running client tests against the powerpull cluster running with some of the newer features such as persistent subscriptions.

(edited by @jen20 to remove email formatting and quoting of the last message)

[jen20]

@haf As @jageall and @bartelink correctly identified, the original text of the issue reads:

Once the client is published, we intend to port the server to .NET Core also.

This clearly states the ordering in which artefacts will emerge.

[jageall]

I hadn't realised there was a concurrency bug fix. I'll pause what I am doing here and get a 4.1.1 build of the core client out. It wasnt apparent in the PR titles. Will work on it tonight.

I think it is in the PR you did ? I never had an issue raised in the core client about their being concurrency fix so hadnt prioritised getting it merged across over getting the clients unified.

If it is in that pr are you able to verify or share some test that does verify @bartelink?

[haf]

I am not sure that saying the client connects half-successfully is really even vaguely accurate as it has 1000s of hours of use.

How can you say that after all the debugging I've been doing to try and fix it? It's an issue that adversely affects me. I'm happy that @bartelink made you aware of some fixes. I'm going to pull those in as soon as they are available and see if they pass our load tests.

[jen20]

@haf I have edited your post to remove the unnecessary abuse of an open-source contributor. Please do not do so again in future, and refrain from commenting on this issue further unless you have a constructive technical point to make.

[ninjaboy]

Hi all, there was an issue mentioned earlier about the EventStore.ClientAPI.Embedded https://github.com/EventStore/EventStore/issues/1715 and this was supposed to be clarified in this thread.

So, may I ask what is the actual plan with regards to that lib? Will it be migrated? Shall we be looking for some other ways to tackle this?
We are using this library in some tests to avoid having fully functioning evenstore entity on the build agents, but the fact it only targets full framework prevents us from using it in tests targeting netcoreapp2.2.

[jageall]

@ninjaboy
if you can run docker on the build agents, then this works:
https://github.com/linedata/reactive-domain/blob/c2aa6b8ee6527406a8458f4093af4e61dd268749/src/ReactiveDomain.Testing/EmbeddedEventStoreFixture.cs

unfortunately there is quite a lot of work to be done to get the server running on .net core still and until it does there is no easy way of running the server embedded in a .net core project

We are planning on making the server able to run on core, but it will be a little while yet.

[jen20]

@ninjaboy Yes, as @jageall says, the easiest approach right now if using for tests is to use containers. That said, we do plan to port EmbeddedClient when we get the server over to .NET Core.

[jen20]

As a more general status update for all involved: Event Store 5.0.0 (currently at RC2, with one more planned before a release) makes good progress in getting to .NET Core. While it is still based on Mono and the .NET Framework, it's got us to the following point:

• The client library now targets .NET Framework 4.6 and .NET Standard 2.0 in the main Nuget package, obviating the need for a second client package.
• Dramatically simplified build process (now buildable in most circumstances using dotnet build).
• Late versions of all dependencies, and a more common dependency management system (dotnet restore).
• We target the latest version of Mono, which has improved TLS performance (in particular) on UNIX platforms and addressed idle CPU usage.
• Unified codebase between UNIX and Windows, with the exception of native code around V8. This is a huge deal, as differences between platforms are now dealt with at runtime instead of build time, and the same build artefacts can be used cross platform. Formerly conditional compilation was used for areas where Mono and .NET diverged.
• Synchronisation primitives are now the same between platforms. Earlier versions of Mono used AutoResetEvent instead of ManualResetEventSlim for signaling between threads, because of performance differences between Mono and the .NET Framework. This has now been eliminated, and the same synchronisation primitives are used everywhere following benchmarks.

There is still more work to do in order to get the server fully over to .NET Core, most of which centers around the HTTP APIs. This is the team's next focus following the imminent 5.0.0 release.

[Mnuzz]

Is there a broken down list of incremental issues necessary to complete this? I'd love to see if there's anything I can help with but I'm not sure what needs to be done. I noticed not much updates in this thread despite it being set as a priority.

Is the project supported by community contributions primarily or is there a full time team, or both?

[jen20]

Hi @Mnuzz - there isn’t a breakdown anywhere public right now. Although we’ve had the server running on Core CLR somewhat, the biggest change necessary to ship is a complete rewrite of everything related to HTTP to use the newer ASP .NET Core bits, since many of the pieces we currently depend on are not a part of .NET Core at all.

Following this, extensive performance regression testing is required - each port so far (the first one happened before .NET Core was called that, and we’re up to five or six attempts now!) has shown much spikier performance than with Mono or the full-fat .NET Framework.

The first pieces of this work - updating all dependencies to .NET Core-compatible versions, changing the project file format to the newer MSBuild format producing an officially supported .NET Standard Client API - all shipped in major version 5. We expect the server to require .NET Core in major version 6 which is under development now.

There is a full time team working on Event Store, which has gone through a considerable growth period over the last few months. There are also many community contributions - though usually we encourage discussion to be opened before a major amount of work is done. @jageall and I have been discussing publishing a public roadmap for which features will map to which versions for the open source project.

[Mnuzz]

@jen20 Thank you for the fast reply. I'd love to see a more incremental splitting of this task into more manageable subtasks, and I think that in general the approach helps get stuff moving along faster.

I actually have some experience in performance tuning (with full Framework but interested in getting some Core experience) and would love to take a swing at it sometime. I can't promise any results as it sounds like it would be a real challenge given the number of attempts... Do you have any documentation you are able to share on the progress and outstanding issues with it and past failed attempts? Thanks!

[jen20]

Hi @Mnuzz, as this gets attacked issues will start to appear - @jageall will keep it up to date.

[jen20]

This is progressing well - there are two important branches that @thefringeninja has been working on here and here with a new gRPC-based client API (as per #1596, also part of this milestone).

[thefringeninja]

Just so everyone knows, the grpc branch has all the work from the pestrel branch.

[jen20]

Thanks to @thefringeninja, @pgermishuys and @hayley-jean's work, we can now close this issue since .NET Core support (actually: requirement) is now on v6-master, from which releases of V6 will be cut, starting with a preview release over the next couple of weeks. There are still undoubtedly outstanding issues around .NET Core support (not least #2020) but each of these should be covered by it's own tracking issue in the v6 RC milestone.