Efcore: Implement ObjectMaterialized event

[ajcvickers]

Splitting this out as a separate issue as required on #626 so we can track it independently of other life-cycle hooks.

The ObjectMaterialized event will fire after an entity instance has been created and all it's non-navigation properties have been set.

[SidShetye]

@ajcvickers thanks for breaking this down into even smaller work items. Wasn't sure what granularity @divega was looking for.

Anyway, could this be promoted to 3.0.0 milestone just like https://github.com/aspnet/EntityFrameworkCore/issues/15910 ? The reason being we need hooks at both paths to and from the database - to preserve symmetry and avoid data corruption during data transformations. Having both ObjectMaterialized and SavingChanges is the bare minimum for us to unblock our EF Core customers who've been patiently asking us to support EF Core for a very long time. At our end, we can spotlight the 3.0.0 release as "fully supported" when it eventually comes out.

Thanks

[ajcvickers]

@SidShetye Can you give some details on how you intend to use ObjectMaterialized?

[SidShetye]

@ajcvickers : Sure. We basically use integration points in any ORM to enforce our data security pipeline.

Specifically, for data about to be written to the database we'd use something like SavingChanges in EF that results in encrypted data being sent to the database.

So on the reverse path, we again need our security pipeline to kick in that results in plaintext data being presented to the EF application. ObjectMaterialized in EF 6.x allows us that attach point and we'd like to use it similarly in EF Core too. As you can imagine, without anything like ObjectMaterialized on the reverse path, the data remains in an unusable state. For a visual, check out the very first figure on page 1.

[SidShetye]

@ajcvickers - will this be marked for the 3.0 release? Did my response address your question on usage? Thanks

[ajcvickers]

@SidShetye It's pretty unlikely to go into 3.0 at this point.

[SidShetye]

@ajcvickers Could we please have this for 3.1?

[optiks]

@SidShetye Have you considered using value conversions instead? Presumably you have a mechanism for flagging which properties need to be encrypted?

You could also consider introducing an Encrypted (potentially with an implicit operator) and expose that in your domain.

https://docs.microsoft.com/en-us/ef/core/modeling/value-conversions

[ajcvickers]

@SidShetye We haven't decided what will be in 3.1 yet.

[nphmuller]

With this event we could implement child entity ordering in a way that could simplify our code-base a lot.

We used to have a (pretty unoptimized) implementation that relied on internal EFCore classes. This solution broke when transitioning to EFCore 3.0 (as expected, but it was a risk we where willing to take at the time). For now we will probably order manually per entity type that needs it. But it would be great if we configure this globally for the entity type, which this event would allow.

[EMooreMAC]

We need this feature because our entities track their own changes independently of the DbContext, and our framework needs to know the difference between a property being set on initialization vs later. In EF6, we can set an IsInitializing flag to false after the object is materialized, and our tracking code (in property setters) can ignore property changes that happen during initialization.

I'm open to ideas as to how else we could achieve similar functionality.

[optiks]

I'm open to ideas as to how else we could achieve similar functionality.

@EMooreMAC, I’ve solved this in the past. Give me a few days and I’ll try and get some code for you.

[StevenRasmussen]

Ok.... posting this for anyone else that would like a workaround in the short term until this is implemented. This isn't fully tested so use at your own risk... and with the caveat that the underlying code may change at any time since it is relying on some EF Core internals. Perhaps someone from the EF team could comment on how stable/usable this might be for the short term. That being said... I'm using the code with great success.

Declare an interface for methods to be invoked

interface IMaterialize
{
    void OnMaterializing();
    void OnMaterialized();
}

Implement the interface on your model:

class YourModel : IMaterialize
{
    public void OnMaterializing()
    {
        // Called after the constructor has been called and object created
    }
    public void OnMaterialized()
    {
        // Called after all properties have been populated
    }
}

Create a new implementation of the EntityMaterializerSource class which inherits from the EFCore implementation but overrides one of the methods:

class MyEntityMaterializerSource : Microsoft.EntityFrameworkCore.Query.EntityMaterializerSource
{
    public MyEntityMaterializerSource(EntityMaterializerSourceDependencies dependencies)
        : base(dependencies)
    {
    }

    public override Expression CreateMaterializeExpression(IEntityType entityType, string entityInstanceName, Expression materializationContextExpression)
    {
        var baseExpression = base.CreateMaterializeExpression(entityType, entityInstanceName, materializationContextExpression);
        if (entityType.ClrType.GetInterfaces().FirstOrDefault(i => i == typeof(IMaterialize)) != null)
        {
            var onMaterializingMethod = entityType.ClrType.GetMethod(nameof(IMaterialize.OnMaterializing));
            var onMaterializedMethod = entityType.ClrType.GetMethod(nameof(IMaterialize.OnMaterialized));

            var blockExpressions = new List<Expression>(((BlockExpression)baseExpression).Expressions);
            var instanceVariable = blockExpressions.Last() as ParameterExpression;

            var onMaterializingExpression = Expression.Call(instanceVariable, onMaterializingMethod);
            var onMaterializedExpression = Expression.Call(instanceVariable, onMaterializedMethod);

            blockExpressions.Insert(1, onMaterializingExpression);
            blockExpressions.Insert(blockExpressions.Count - 1, onMaterializedExpression);

            return Expression.Block(new[] { instanceVariable }, blockExpressions);
        }

        return baseExpression;
    }
}

Now simply replace the service while adding the DbContext to the service collection or in the Dbcontext.OnConfiguring:

services.AddDbContext<TestContext>(options =>
{
    options.ReplaceService<IEntityMaterializerSource, MyEntityMaterializerSource>();
}...

// Or...
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
    optionsBuilder.ReplaceService<IEntityMaterializerSource, MyEntityMaterializerSource>();
}

[aktxyz]

wow ... MyEntityMaterializerSource works great :) an official approach will be great but for now this is perfect

[titobf]

@StevenRasmussen thanks so much for that workaround! I have just tried it and it works!

[haiplk]

Hi all,,
Just curious about the override EntityMaterializerSource cannot work with nested select

dbContext.Table1.Select(f=> new CustomModel{ DateTimeField = f.DateTime }).ToListAsync()

Do you all have any idea on it

[EMooreMAC]

@StevenRasmussen just wanted to check back and say that this solution is working for us as well. Thanks a ton, you helped clear a blockage for our large scale migration to EF Core.

[smitpatel]

@haiplk - Your query does not materialize any entity.