Efcore: Support lazy and explicit loading from entities queried with NoTracking behavior

[JoesGab]

Explicit loading of a Reference does not populate the navigation property when QueryTrackingBehavior.NoTracking is set and AsTracking() is used as an override.

Explicit loading of a Collection does work as expected though.

Steps to reproduce

Minimalistic example for various combinations included. The issue arises only in the last assert statement of the second using block.

Program.cs

```c#
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using Microsoft.EntityFrameworkCore;

namespace NoTrackingBug
{
public class BloggingContext : DbContext
{
public DbSet Blogs { get; set; }
public DbSet Posts { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlite("Data Source=blogging.db");
    }
}

public class Blog
{
    public int BlogId { get; set; }
    public string Url { get; set; }

    public List<Post> Posts { get; set; }

    public int OwnerId { get; set; }
    public Person Owner { get; set; }
}

public class Post
{
    public int PostId { get; set; }
    public string Title { get; set; }

    public int BlogId { get; set; }
    public Blog Blog { get; set; }
}

public class Person
{
    public int PersonId { get; set; }
    public string Name { get; set; }

    public List<Blog> OwnedBlogs { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        // Setup the example
        using (var db = new BloggingContext())
        {
            if (!db.Blogs.Any())
            {
                db.Blogs.Add(new Blog
                {
                    Url = "http://blogs.msdn.com/adonet",
                    Posts = new List<Post>{
                        new Post { Title = "SomeTitle" }
                    },
                    Owner = new Person
                    {
                        Name = "Kobert"
                    }
                });

                db.SaveChanges();
            }
        }

        // Explicit load and .NoTracking => does not work for References
        using (var db = new BloggingContext())
        {
            db.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

            var blog = db.Blogs
                .AsTracking()
                .Single();

            db.Entry(blog)
                .Collection(b => b.Posts)
                .Load();

            db.Entry(blog)
                .Reference(b => b.Owner)
                .Load();

            Debug.Assert(blog.Posts != null);
            Debug.Assert(blog.Owner != null); // <== This will fail!
        }

        // Eager load and .NoTracking => does work
        using (var db = new BloggingContext())
        {
            db.ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;

            var blog = db.Blogs
                .Include(b => b.Posts)
                .Include(b => b.Owner)
                .AsTracking()
                .Single();

            Debug.Assert (blog.Posts != null);
            Debug.Assert (blog.Owner != null);
        }

        // Explicit load and .TrackAll => does work
        using (var db = new BloggingContext())
        {
            var blog = db.Blogs
                .Single();

            db.Entry(blog)
                .Collection(b => b.Posts)
                .Load();

            db.Entry(blog)
                .Reference(b => b.Owner)
                .Load();

            Debug.Assert(blog.Posts != null);
            Debug.Assert(blog.Owner != null);
        }
    }
}

}
```

Further technical details

EF Core version: 1.1.0 and 2.0.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer and Micrisoft.EntityFrameworkCore.Sqlite
Operating system: Windows 7 Enterprise
IDE: Visual Studio 2017 Enterprise and Visual Studio Code

[ajcvickers]

Note for triage: behavior is to throw in all cases, as it was in 2.0. Implementation is a bit tricky because it requires running a no-tracking query where the root entity is already materialized but not tracked. Proposing we push this to post 2.1.

[ajcvickers]

Triage: Leaving this in the backlog for now. Please vote for this feature if you need it.

[Suriman]

Both the Lazy Loading feature and the AsNoTraking feature are fundamental for implementing applications that use the WebRule tool from https://codeeffects.com/.

This tool allows to evaluate expressions on graphs of objects where these expressions are defined in a declarative way, which provides a lot of power to the users of an application. You can see examples in the section:
https://codeeffects.com/Business-Rule-Demo

The tool allows navigating through the graph of objects accessing properties and collections of objects and it is precisely in this feature that the EF Core and the support of Lazy Loading come into play. For WebRule it is transparent to access a property of an object that is itself a collection of other objects thanks to the Lazy Loading mechanism of EF Core. Since WebRule only requires access to the properties to read and evaluate them, it is not necessary for the data to be tracked, which greatly increases the performance if the Lazy Loading combined with the AsNoTraking is allowed.

Please consider giving priority to this issue as it is essential in many scenarios, including this one that we particularly use.

[Starkie]

@divega told us that if we wanted to try and solve it we should contact you on the issue, as it's important to our use case.
Could you please guide us a bit on how could it be done?

Thanks in advance

[ajcvickers]

@Starkie Can you be more explicit about what you are asking?

[divega]

@ajcvickers I met @Starkie and @Suriman recently. They expressed a lot of interest on creating a PR to address this issue. They are asking for guidance and details on what makes the issue tricky to fix.

[Starkie]

Sorry for being so ambiguous @ajcvickers 😅

As Diego said, we'd like to try to solve this issue. He commented that you may have some ideas on how could this be done, and might be able to offer us some guidance.

Sorry again, and thanks for your time.

[ajcvickers]

@Starkie The difficult aspect of this is changing the query pipeline so it can handle performing a no-tracking query with fixup when the root entity has already been materialized. This is the tricky bit that will need input from query people like @smitpatel and @maumar. I can't really give any kind of deep guidance on this without doing more digging and experimenting.

[smitpatel]

Filed #12208 to have better support to do fixup in QueryBuffer so that it can be reused for scenario like above.

[divega]

@Starkie, @Suriman, just to close the loop on this: we identified #12208, which would be a pre-requisite for #10042. All in all, this issue is too complicated for a first PR. I would recommend trying something simpler first.

[Suriman]

@divega, thank you for the interest shown. We will try to think of a shortcut.

[ajcvickers]

See also #12780

[smitpatel]

We need to re-evaluate this. Query does not do any fixup other than eager loading.

[rosenbjerg]

Separating lazy-loading from change-tracking, perhaps still using a Castle proxy but a more lightweight one, could be useful for CQRS.
This could help gain some performance, if the change-tracking part of the proxying is costly, but I suspect the overhead is almost exclusively the creation of the proxy instance?

[seekingtheoptimal]

Maybe I'm missing something, but it seems to me the issue _("attempt was made to lazy-load navigation property on detached entity")_ can exists also when not using AsNoTracking() at all. When throw is turned off for this, then the lazy loading just does not work, with or without Include() the navigation properties are null.
Any idea when and why this could happen?

The current query where I noticed this, is way too complicated to start digging into that and couldn't find a consistent pattern (some corporate hackathon abomination I inherited sadly), but would be nice to know if there are any specific cases when such thing could happen. Because for now for some dbsets it works, and for some it doesn't, and I'm stuck on where to look.

[ajcvickers]

@seekingtheoptimal I'm not aware of any specific bugs we know of in this area.

[klepeis]

Any update on this?

We are designing a service implementing CQRS and I am leveraging Entity on our read side. I am leveraging an object that wraps my DbContext for my queries as shown below.

private readonly DbContext _dbContext;

public ReadOnlyContext(DbContext dbContext)
{
   _dbContext = dbContext;
}

public IQueryable<TEntity> Set<TEntity>() where TEntity : class
{
   return _dbContext.Set<TEntity>().AsNoTracking();
}

I'd like to be able to lazy load associated objects returned from my queries as needed.

Thanks!

[ajcvickers]

@klepeis This issue is in the Backlog milestone. This means that it is not planned for the next release (EF Core 5.0). We will re-assess the backlog following the this release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources.

[klepeis]

Thanks @ajcvickers!