Efcore: Memory leak in ServiceProviderCache?

[gallivantor]

I have an application which has a timer, and every minute it does this:

```C#
using (var context = new MyDbContext())
{
// do some work
context.SaveChanges();
}

Gradually over time the memory usage of the process has been going up, and eventually it crashed with an OutOfMemoryException.

So I ran it again, took a memory dump, and investigated where the memory was going:

![image](https://user-images.githubusercontent.com/24268360/32373014-245a695c-c08f-11e7-9e64-7de8c5874311.png)

The ServiceProviderCache seems to gradually be growing with a new ServiceProvider every time it creates a context, and doesn't seem to get freed on the Dispose.


### Steps to reproduce

```c#
for (int i = 0; i < 1000; i++)
{
  using (var context = new MyDbContext())
  {
    context.SomeSet.Add(new SomeEntity());
    context.SaveChanges();
  }
}

then check memory usage

Further technical details

EF Core version: 2.0.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10
IDE: VS 2017

[ajcvickers]

@gallivantor There are a few pathological things that can be done when configuring the context that will cause this to happen. Can you share the code used to configure your context? (i.e the code in OnConfiguring or AddDbContext.)

Also, you should see a warning in the logs for this--did you see that?

[gallivantor]

I do see this warning logged:
_More than twenty 'IServiceProvider' instances have been created for internal use by Entity Framework. Consider reviewing calls on 'DbContextOptionsBuilder' that may require new service providers to be built._

The DbContext is as follows (I've changed some names as per company policy). I don't think I'm doing anything particularly edge-case here; the documentation isn't very forthcoming about what the possible causes of this warning might be?

    public class MyDbContext : DbContext
    {
        public DbSet<MyType1> Table1 { get; set; }
        public DbSet<MyType2> Table2 { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer(/* CONN STRING */, opt =>
            {
                opt.EnableRetryOnFailure();
                opt.CommandTimeout(60);
            });
            optionsBuilder.UseLoggerFactory(new EfConsoleLoggerFactory());

            optionsBuilder.ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning));

        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            ConfigureTable<MyType1>(modelBuilder, "Table1");
            ConfigureTable<MyType2>(modelBuilder, "Table2");
        }

        private void ConfigureTable<T>(ModelBuilder modelBuilder, string tableName) where T : MyBaseTable
        {
            modelBuilder.Entity<T>().ToTable(tableName).HasKey(e => e.Id);
        }
    }

[ajcvickers]

@gallivantor It is the creation of a new ILoggerFactory instance for each context instance that is causing the problem. Instead you should have one shared ILoggerFactory, or at least a very small number, shared by all your context instances. I agree that this is pretty unrecoverable--we will discuss what we can do to mitigate the issue.

[gallivantor]

Great thanks, yep that seems to have fixed it.

I think as this issue isn't very discoverable -- and is the sort of problem that someone could easily put into Production causing a slow-burning memory leak -- it would be good if a way could be found to change this behaviour; or at least update the docs to make clear what causes this problem, and perhaps re-word that warning message to be explicit about the fact that it could be creating a memory leak.
For example I think I copied that UseLoggerFactory line from a StackOverflow answer somewhere and if other people are doing the same they could be unwittingly introducing a similar bug.

On a side-note, is it possible to use the DBContext Pooling with other DI frameworks? Using the pooling would also work around this problem, but as far as I can tell it's tied into the Microsoft.Extensions.DependencyInjection DI framework (services.AddDbContextPool) and I have an existing app which is built around using Unity directly

By the way, good work with EF Core. It's great to be able to interact with the team here on Github, it gives us much more confidence to use the product knowing that we can get answers easily if problems come up.

[ajcvickers]

@gallivantor Yep, I started work on updating the docs yesterday. Hope to send them out for review today.

[ajcvickers]

Filed "Add fwlnk" and "Make warning error by default" and also submitted docs PR to update the link.

[codematrix]

Awesome! I was having the same issue as well. I was creating the LoggerFactory on every DbContext instance request.

[voroninp]

Hm... I have similar problem with 3.1.5

DbContext is disposed but not released.

[voroninp]

@ajcvickers , here is the repo to reproduce: https://github.com/voroninp/EFCoreMemoryLeak

dotMemory shows me disposed yet not collected instances of AppDbContext, and Entity instances, which stay alive even after several forced GC runs.

[voroninp]

Also in tests with NServiceBus I have

ObjectDisposedException: Cannot access a disposed object. Object name: 'IServiceProvider'.

thrown by the NOT YET disposed context (_disposed: fale) when I try to access ChangeTracker.

at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException() in /_/src/DependencyInjection/DI/src/ServiceLookup/ThrowHelper.cs:line 14
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) in /_/src/DependencyInjection/DI/src/ServiceLookup/ServiceProviderEngineScope.cs:line 34
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetServiceT
at Microsoft.EntityFrameworkCore.Internal.ScopedLoggerFactory.Create(IServiceProvider internalServiceProvider, IDbContextOptions contextOptions)
at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.b__7_0(IServiceProvider p)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope) in /_/src/DependencyInjection/DI/src/ServiceLookup/ServiceProviderEngine.cs:line 88
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) in /_/src/DependencyInjection/DI/src/ServiceLookup/ServiceProviderEngineScope.cs:line 37
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredServiceT
at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
at Microsoft.EntityFrameworkCore.DbContext.get_InternalServiceProvider()
at Microsoft.EntityFrameworkCore.DbContext.get_ChangeTracker()
at CarNext.Hydra.Data.Repositories.HydraDataUnitOfWork.Dispose(

I suspect this code in DI container may be the reason for the bug:

private List<object> BeginDispose()
{
    List<object> toDispose;
    lock (ResolvedServices)
    {
        if (_disposed)
        {
            return null;
        }

        _disposed = true;
        toDispose = _disposables;
        _disposables = null;

        // Not clearing ResolvedServices here because there might be a compilation running in background
        // trying to get a cached singleton service instance and if it won't find 
        // it it will try to create a new one tripping the Debug.Assert in CaptureDisposable
        // and leaking a Disposable object in Release mode
    }

    return toDispose;
}

First it sets itself as disposed and then returns services to dispose.

[ajcvickers]

@voroninp Disposing and garbage collection are not tightly coupled. That being said, if you believe there is a problem with D.I., then please file an issue at https://github.com/dotnet/runtime.

[voroninp]

@ajcvickers Look at the project I referenced, the leak is reproducible.

As to Disposing that's probably another issue somehow caused by how NSB works with DI. For me it's a mystery how internal service provider gets disposed.