Efcore: Couchbase provider discussion

[divega]

Creating this discussion issue to continue our conversation with people from Couchbase and the Couchbase open-source community about an EF Core provider.

cc @mgroves, @jeffrymorris, @brantburnett.

[jeffrymorris]

Hey Folks -

We have has some discussion internally regarding short and long term development goals for Couchbase EF and have decided to:

• Target EF Core 2.0 and porting over the Linq layer from Linq2Couchbase. Assuming this is feasible and is doable within a realistic time frame we would release a v1 Couchbase EF provider based on EF Core 2.0 sometime this year, prefferably in the early summer.
• Once we getting a published v1.0 Couchbase EF provider, switch over to v2 by targeted EF Core 3.0 - this should conincide with end of the dev cycle for EF Core 3.0 before production and should leave us with a more stable platform to base v2 Couchbase EF on.

We feel this would meet our short term goals to offer an EF provider sooner than later (even if its not 100%) and then work torwards supporting the future of EF, 3.0.

-Jeff

[smitpatel]

I would suggest using EF Core 2.1 (rather than 2.0)

[ajcvickers]

@jeffrymorris Sounds reasonable. A couple of thoughts:

• As @smitpatel said, target 2.1 (or 2.2) rather than 2.0.
  
  
  
  • 2.0 is already out-of-support.
  
  
  • 2.1 is LTS, so it will be in-support for a while yet
  
  
  • 2.2 has the most features and seems to be the most stable, but will be out of support before 2.1.
  
  
• We won't be able to change much in the 2.x codebase if things you need are missing/hard.
• We had discussed following along with the Cosmos provider, which will be harder to do since it is targeting 3.0.

That being said, the plan sounds reasonable.

[jeffrymorris]

I probably should have said _"Target EF Core 2.X"_...based upon LTS support, 2.1 seems best - unless we need to pivot somewhere. Thanks for the feedback!

[mgroves]

Is "Microsoft.EntityFrameworkCore.Relational.Specification.Tests" the functional test package we discussed? Which release of that should we start with. I'm assuming 2.2.1? Does the version number there roughly correspond with the EF Core version?

EDIT: Or, since Couchbase is not relational, should we look at a different package of specification tests?

[ajcvickers]

@mgroves Microsoft.EntityFrameworkCore.Specification.Tests is the subset of tests for all providers, including non-relational.

[ajcvickers]

And yes, the versions correspond with the other EF packages. We ship the specification tests every time we ship a major or minor version, including previews.

[mgroves]

@ajcvickers Excellent, thank you. Can you recommend any resources on how to get started with Microsoft.EntityFrameworkCore.Specification.Tests (other than Postgres repository)? My current thought is that these tests will help to drive the code that we write (a TDD approach).

[ajcvickers]

@mgroves We don't have any good docs on this, but the general idea is that the package contains test classes that you derive from in your project to run with your provider. A TDD approach might be hard at first with these tests they all require some level of basic functionality--they aren't unit tests as such, so you may want some of those of your own as appropriate. That being said, getting to a point where all these tests pass is a great goal and should cover all the most important functionality.

[mgroves]

I've taken a crack at getting some of the basic infrastructure created.

I used the "InMemory" provider from the release/2.1 branch as a guideline - https://github.com/aspnet/EntityFrameworkCore/tree/release/2.1

My efforts are in my own fork right now. It's basically just a bunch of skeleton classes. I can "add" entities, and then a "notimplementedexception" is thrown when I attempt to SaveChanges. If I want to proceed trying to bring in Linq2Couchbase, can you recommend a good place to start? I'm thinking CouchbaseQueryModelVisitorFactory and/or CouchbaseEntityQueryableExpressionVisitorFactory?

[ajcvickers]

@smitpatel Any thoughts on @mgroves's question?

[smitpatel]

@mgroves - There are 2 different paths depending how you want to go.
You can just override QueryCompiler and intercept Expression tree. There is parameterization step for EF in it but if you take care of it then you can just send the Expression tree to linq2couchbase. This may require processing for certain EF features like changetracking, query filters and EF's custom queryable operators.
If you want to combine the query model being used in linq2couchbase with EF's then there would be more services to be overridden. QueryModelVisitorFactory is main visitor of QM. EntityQueryableEV is basically to deal with query roots. DbSets in EF are query roots (corresponds to table/collection on database side) and they are entityQueryables in expression tree. A lot of dependent services should just fall in place. Though you may run into a lot of invariants inside the framework which creeped into from relational but could cause issue for non-relational provider.
While exact difficulty is hard to determine, path 1 should be better given no reliance on existing pipeline (lesser work) and if you some assumptions are invalid then we cannot change 2.2 pipeline.

[mgroves]

You all have been great helps so far, thank you!

To override QueryCompiler, I just TryAdd<IQueryCompiler,CouchbaseQueryCompiler> in the AddEntityFrameworkCouchbase extension method? TryAddCoreServices won't override it?

Kinda related, I'm a little unclear about how/where to accept and store config (like ip address, credentials, bucket name). I'm assuming the DbContextOptionsBuilder extensions method (UseCouchbase) should accept those settings, but what do I do with them at that point?

[ajcvickers]

@mgroves Look at IOptionsExtension. For example, there is one for InMemory that handles the database name.

[ajcvickers]

@mgroves Also, yes, TryAddCoreServices won't override it. The infrastructure ensures that provider services are registered first, and then EF will only use the default implementations if the provider has not registered an implementation.

[brantburnett]

What's the status of the 3.0 work? Is it ready to start developing a provider yet, or is it still at a high rate of flux?

[AndriySvyryd]

@brantburnett Sorry for the late response, but 3.0 is stable now :smile: We do have some breaking changes planned for the next release, but they should mostly only affect the relational providers.