Efcore: Guide on memory leak warning ef core 3.0 preview9

[dotnetshadow]

Currently using the following project:
https://github.com/KalikoCMS/KalikoCMS.NextGen/blob/netcore3.0-experimental/KalikoCMS.Legacy/Data/Repositories/LegacyRepositoryBase.cs

Using ef core 3.0 preview8 the following code would work:
```c#

public virtual IQueryable GetAll() {
return _context.Set().Select(Map()).AsNoTracking();
}

public override Expression> Map() {
return x => new ContentTypeEntity {
ContentTypeId = ToGuid(x.PageTypeId),
DefaultChildSortDirection = x.DefaultChildSortDirection,
ContentProviderId = Guid.Empty,
DefaultChildSortOrder = x.DefaultChildSortOrder,
DisplayName = x.DisplayName,
Name = x.Name,
ShowInAdmin = true
};
}

But in ef core 3.0 preview9 the following error occurs

Exception message:
Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository. This could potentially cause memory leak."}
Data: {System.Collections.ListDictionaryInternal}
HResult: -2146233079
HelpLink: null
InnerException: null
Message: "Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository. This could potentially cause memory leak."
Source: "Microsoft.EntityFrameworkCore"
StackTrace: " at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)rn at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)rn at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)rn at System.Linq.Expressions.ExpressionVisitor.VisitTrn at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)rn at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)rn at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)rn at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)rn at System.Linq.Expressions.ExpressionVisitor.VisitLambdaTrn at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryTResultrn at System.Linq.Enumerable.ToListTSourcern at KalikoCMS.Services.Initialization.ContentTypeSynchronizer.SynchronizeContentTypes() in D:\Programming\Projects\Libraries\KalikoCMS.NextGen\KalikoCMS.NextGen\KalikoCMS\Services\Initialization\ContentTypeSynchronizer.cs:line 35rn at KalikoCMS.Services.Content.ContentTypeResolver.Initialize() in D:\Programming\Projects\Libraries\KalikoCMS.NextGen\KalikoCMS.NextGen\KalikoCMS\S
ervices\Content\ContentTypeResolver.cs:line 35rn at KalikoCMS.Services.Initialization.InitializationService.Initialize() in D:\Programming\Projects\Libraries\KalikoCMS.NextGen\KalikoCMS.NextGen\KalikoCMS\Services\Initialization\InitializationService.cs:line 44"
TargetSite: {System.Linq.Expressions.Expression VisitConstant(System.Linq.Expressions.ConstantExpression)}
```

I understand it could be related to
https://github.com/aspnet/EntityFrameworkCore/issues/13048
https://github.com/aspnet/EntityFrameworkCore/pull/17051

I'm just wondering if there is any guidance on how to solve the issue?

Further technical details

EF Core version: 3.0.0-preview9.19423.6
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system:
IDE: (e.g. Visual Studio 2019 16.3 preview 3.0)

[smitpatel]

https://github.com/KalikoCMS/KalikoCMS.NextGen/blob/e1c08ce28c1e88ef6e12c7d63047ac60c50a98c2/KalikoCMS.Legacy/Data/Repositories/LegacyRepositoryBase.cs#L54-L59

ToGuid method which is being used in projection is defined on Repository so expression tree contains reference to object of your repository. Since your repository has DbContext also inside a field, it is actually leaking memory. Easy way to fix this is to use static method so you are not putting your repository objects inside expression tree.

[dotnetshadow]

@smitpatel Thank you so much I was looking at the Map() method not realising the ToGuid() was probably the problem. Making it static worked :)

[schmitch]

@smitpatel looks like the error happens in the Microsoft.AspNetCore.DataProtection.EntityFrameworkCore Package of AspNetCore:

(Version: 3.0.0-preview9.19424.4)

fail: Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider[48]
      An error occurred while reading the key ring.
System.InvalidOperationException: Client projection contains reference to constant expression of type: Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository<Asvg.SalesTool.Web.Areas.Base.Services.Framework.DataProtectionDbContext>. This could potentially cause memory leak.
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)
   at System.Linq.Expressions.ConstantExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository`1.GetAllElements()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh)
info: Microsoft.AspNetCore.DataProtection.Internal.DataProtectionHostedService[0]
      Key ring failed to load during application startup.
System.InvalidOperationException: Client projection contains reference to constant expression of type: Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository<Asvg.SalesTool.Web.Areas.Base.Services.Framework.DataProtectionDbContext>. This could potentially cause memory leak.
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.ConstantVerifyingExpressionVisitor.VisitConstant(ConstantExpression constantExpression)
   at System.Linq.Expressions.ConstantExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block)
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node)
   at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.InjectEntityMaterializers(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalShapedQueryCompilingExpressionVisitor.VisitShapedQueryExpression(ShapedQueryExpression shapedQueryExpression)
   at Microsoft.EntityFrameworkCore.Query.ShapedQueryCompilingExpressionVisitor.VisitExtension(Expression extensionExpression)
   at System.Linq.Expressions.Expression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.AspNetCore.DataProtection.EntityFrameworkCore.EntityFrameworkCoreXmlRepository`1.GetAllElements()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.GetAllKeys()
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.CreateCacheableKeyRingCore(DateTimeOffset now, IKey keyJustAdded)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.Microsoft.AspNetCore.DataProtection.KeyManagement.Internal.ICacheableKeyRingProvider.GetCacheableKeyRing(DateTimeOffset now)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRingCore(DateTime utcNow, Boolean forceRefresh)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider.GetCurrentKeyRing()
   at Microsoft.AspNetCore.DataProtection.Internal.DataProtectionHostedService.StartAsync(CancellationToken token)

[smitpatel]

@schmitch - I filed issue https://github.com/aspnet/AspNetCore/issues/13696 on aspnetcore repo. Thanks for pointing us to the error.

[JanEggers]

@smitpatel is it possible to improve the message:

Client projection contains reference to constant expression of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository.

Client projection contains reference to constant expression ToGuid of type: KalikoCMS.Legacy.Data.Repositories.LegacyContentTypeRepository.

to better point to the expression that is causing the issue

[smitpatel]

@JanEggers - In expression tree we-encounter the constant, but it may not always possible to understand why the constant there. I will look into it, and try to improve exception message to be as useful as it can be.

[ajcvickers]

Improve the exception message and add a fwlink.

[smitpatel]

@divega - To generate fwlink.

[divega]

@smitpatel link is https://go.microsoft.com/fwlink/?linkid=2103067, for now pointing to query overview.

Do we need a docs issue to add the contents? I am thinking we need to go back and review all the fwlinks created for this release in search for gaps.

[MCFHTAGENTS]

OK- I have come across this with code like this:
` Chart decimalData = new Chart();
var bookingsBySource = this.context
.BookingView
.Include(a => a.Booking)
.ThenInclude(a => a.Source)
.Where(a => a.BookingTypeId != BookingType.Cancelled && a.StartDate.Year == Year && a.PropertyId == PropertyId)
.GroupBy(a => new { a.StartDate.Year, a.Booking.Source.SourceId, a.Booking.Source.Name })
.Select(
a => new { a.Key.Year, a.Key.SourceId, Source = a.Key.Name, Bookings = a.Count(), Total = a.Sum(b => b.Total) });

            var chartData = new BarChartDataSet<decimal>
            {
                Label = "Booking Value",
                BorderWidth = 1,
                Data = bookingsBySource.Select(d => d.Total).ToList(),
                BorderColor = bookingsBySource.Select(b => borderColour).ToList(),
                BackgroundColor = bookingsBySource.Select(c => backgroundColour).ToList(),
            };

`
The exception is being thrown on the BorderColor = bookingsBySource line.
borderColor is a static class member. That I need to clone n times to create a list the same length as 'Data'.

I don't see what is wrong with the above code and why the warning exception is thrown.

[smitpatel]

@MCFHTAGENTS - Please file a new issue and share full repro code. It would be hard to guess what the expression tree would be based on info you posted above.