Efcore: Discussion for announcement: EF Core 2.0: design-time DbContext discovery changes

[divega]

This is the discussion thread for https://github.com/aspnet/Announcements/issues/258.

[thiennn]

In 2.0 Preview 1. I implemented IDbContextFactory because I don't want Add-Migration, Update-Database,... to run my Startup.cs. It worked as expected.

In Preview 2, I changed to IDesignTimeDbContextFactory, but when I run Add-Migration, I see it run both ConfigureServices and Configure in Startup.cs before CreateDbContext of IDesignTimeDbContextFactory. Is this a expected new behavior?

[divega]

In Preview 2, I changed to IDesignTimeDbContextFactory, but when I run Add-Migration, I see it run both ConfigureServices and Configure in Startup.cs before CreateDbContext of IDesignTimeDbContextFactory. Is this a expected new behavior?

I suspect we changed the order in which things are discovered. I can't recall if we made an explicit decision that this was ok. @bricelam?

[bricelam]

Nothing has changed. As part of discovering DbContext types, we have always tried to construct the application service provider. IDesignTimeDbContextFactory is only a hook for overriding the construction of DbContext types (not the discovery).

[ajcvickers]

@bricelam Do we need the service provider to discover context types?

[bricelam]

We always look in three places:

1. Referenced by IDesignTimeDbContextFactory implementations in the assembly or startup assembly
2. Registered in the application's service provider
3. DbContext derived types in the assembly or startup assembly

After we've discovered them all, we pick one to use.

I've considered having a way to bypass the type discovery (e.g. specify an assembly-qualified name on the command) and then discover a construction mechanism (factory first, service provider second, default constructor last), but this would require a completely separate (new) code path, and the value always seemed minimal.

[bricelam]

We could also look into ways of short-circuiting the type discovery so Startup isn't run if we find the specified type from its factory. (note, not specifying -Context would still need to look in the service provider to ensure there's still only one)

[ajcvickers]

@bricelam I think it's very useful to have an escape hatch that doesn't involve trying to build the application's service provider since doing so could have side effects that need to be avoided. I had assumed that having a factory was the way to do that. Maybe the short circuiting would be the way to enable this.

Moving temporarily out of Discussions so we can triage this.

[thiennn]

I hadn't deep dived into the internal implementation of the type discovery. But I have code for both 2.0 preview 1 work as expected https://github.com/simplcommerce/SimplCommerce/tree/master and 2.0 preview 2 not work as expected https://github.com/simplcommerce/SimplCommerce/tree/netcore2preview2

[ajcvickers]

@thiennn When you say, "not work as expected" can you be more specific? Does it throw an exception? If so, what is the exception message and stack trace? If it doesn't throw an exception, then other than it running code when you don't expect it to, what is not working?

[thiennn]

@ajcvickers like my earlier comment. In 2.0 preview1: Add-Migration, Update-Database command doesn't invoke Startup.cs. But in 2.0 preview 2, it run both ConfigureServices and Configure of Startup.cs

[ajcvickers]

@thiennn Yes, but then what happens? Does the command complete successfully? Do you get an error?

[thiennn]

@ajcvickers In 2.0 preview 2, the command run into my Startup.cs which have some dependencies that not ready at design time. If I remove these dependencies, the command complete successfully.

What I expected is the command should not invoke my Startup.cs because I already implemented IDesignTimeDbContextFactory

[ajcvickers]

@thiennn If you don't remove those dependencies, what happens? Do you get an error? If so, what's the error message/output/stack trace?

[thiennn]

@ajcvickers I don't show the error or the stack trace here because I think it's not relevant, and it's specific to my project (I built a custom config provider and it read the value from database during the Startup. At design time the database haven't created so I got error)

Again, What I expected is that the migration commands (Add-Migration, Update-Database,..) should not invoke my Startup.cs when I have IDesignTimeDbContextFactory implemented.

[thiennn]

PM> Add-Migration Test
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.SqlClient.SqlException: Cannot open database "SimplCommerceDb" requested by the login. The login failed.
Login failed for user 'THIENPC\nlqth'.
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource1 retry) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open(Boolean errorsExpected) at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.BufferlessMoveNext(Boolean buffer)
at Microsoft.EntityFrameworkCore.Storage.Internal.SqlServerExecutionStrategy.ExecuteTState,TResult
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.MoveNext() at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.<_TrackEntities>d__172.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor1.EnumeratorExceptionInterceptor.MoveNext() at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer) at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector)
at SimplCommerce.Module.Core.Extensions.EFConfigProvider.Load() in C:\DATAProjects\SimplCommerce\SimplCommerce\src\Modules\SimplCommerce.Module.Core\Extensions\EFConfigProvider.cs:line 24
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList1 providers) at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build() at SimplCommerce.WebHost.Startup..ctor(IHostingEnvironment env) in C:\DATA\Projects\SimplCommerce\SimplCommerce\src\SimplCommerce.WebHost\Startup.cs:line 40 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters) at Microsoft.Extensions.Internal.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type) at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetServiceOrCreateInstance(IServiceProvider provider, Type type) at Microsoft.AspNetCore.Hosting.Internal.StartupLoader.LoadMethods(IServiceProvider hostingServiceProvider, Type startupType, String environmentName) at Microsoft.AspNetCore.Hosting.WebHostBuilderExtensions.<>c__DisplayClass1_0.<UseStartup>b__1(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactoryService(FactoryService factoryService, ServiceProvider provider) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProvider provider)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitSingleton(SingletonCallSite singletonCallSite, ServiceProvider provider)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor2.VisitCallSite(IServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass17_0.<RealizeService>b__0(ServiceProvider provider) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider) at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureStartup() at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices() at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication() at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build() at SimplCommerce.WebHost.Program.BuildWebHost(String[] args) in C:\DATA\Projects\SimplCommerce\SimplCommerce\src\SimplCommerce.WebHost\Program.cs:line 21 --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) at Microsoft.EntityFrameworkCore.Design.Internal.AppServiceProviderFactory.CreateFromBuildWebHost(String[] args) at Microsoft.EntityFrameworkCore.Design.Internal.AppServiceProviderFactory.Create(String[] args) at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.FindContextTypes() at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.FindContextType(String name) at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.CreateContext(String contextType) at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.AddMigration(String name, String outputDir, String contextType) at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigrationImpl(String name, String outputDir, String contextType) at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigration.<>c__DisplayClass0_1.<.ctor>b__0() at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_01.b__0()
at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
Exception has been thrown by the target of an invocation.

[divega]

@bricelam I think it is reasonable to expect that we will ignore any exceptions caused by building the service provider and/or trying to resolve DbContext from it. Is that something we can fix for 2.0?

@ajcvickers recalls we used to do that.

[bricelam]

Yes, we should ignore errors. This may have regressed.

[divega]

@bricelam @ajcvickers ok, I have created https://github.com/aspnet/EntityFramework/issues/9073 and assigned to @bricelam in 2.0.0 about ignoring errors. I will re-add this issue to the discussion milestone.

[ajcvickers]

Also filed #9076 about the escape hatch.

[bricelam]

Note, to prevent EF from calling Program.BuildWebHost(), you can simply make it private rename it.

[thiennn]

@bricelam , I made Program.BuildWebHost() private but EF still calling it. But after changed the method name to BuildWebHost2(), it works

public class Program
    {
        public static void Main(string[] args)
        {
            BuildWebHost2(args).Run();
        }

        private static IWebHost BuildWebHost2(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .UseStartup<Startup>()
                .Build();
    }

[bricelam]

Oh weird, in that case, the documentation on TypeInfo.GetDeclaredMethod lies!

[aghe]

Quick fix for add migrations errors:
(Just comment only when you need to generate/update migrations.)

To be able to add migration I just commented what is in Configure() from Startup. It seems that tools also call Startup.Configure when migration is generated and not only the ConfigureServices.

[smithaitufe]

Please I hope you don't mind. If I want to find the namespace where a particular class lies in DotnetCore, where can I get it?

For instance, what is the namespace forIdentityCookieOptions

[smitpatel]

@smithaitufe https://apisof.net/

[smithaitufe]

@smitpatel I have checked the links you provided. But it seems the information there have not been updated. For instance, I saw that IdentityCookieOption is listed under Microsoft.AspNetCore.Identity, but it is not recognized in core 2 preview2.

[smitpatel]

@smithaitufe - IdentityCookieOption was renamed to IdentityConstants in preview2. Therefore it is missing. For more info on changes in Identity, I suggest file a bug on https://github.com/aspnet/Identity repo.

[smithaitufe]

@smitpatel Thanks. I have another problem. Can you assist? This might not be the right place. I can't run migrations now. It keeps throwing error that Relation Roles does not exist. I am using PostgreSql

[smitpatel]

@smithaitufe - Please check https://github.com/aspnet/Identity There can be existing issue like yours there. If not file a new issue. It would have more chances of getting eyes from the people who are maintaining identity.

[TsengSR]

Why isn't the main issue (https://github.com/aspnet/Announcements/issues/258) assigned to a milestone? Or is it some kind of "pending" announcement which may or may not make it to the final version?

[smitpatel]

@TsengSR - Thanks for info. I put it in 2.0 milestone. IT will be applicable to 2.0.0-preview2 & 2.0.0 release

[markrendle]

I have a call in Startup.Configure that calls a helper function that runs dbContext.Database.Migrate() (in a try/catch) and a seed when the application is starting up.

It's _very_ annoying that loop: trying to add a migration from the command line runs the migration, which doesn't exist yet because I can't add it because goto loop;

[bricelam]

@glennc Have we followed up with Hosting about not calling Configure when all we need are the services?

[bricelam]

@markrendle After discussing with Hosting, it sounds like Startup.Configure() should only be used to configure the request pipeline. Application startup code should go inside Program.Main() instead.

public static void Main(string[] args)
{
    var webHost = BuildWebHost(args);

    using (var scope = webHost.Services.CreateScope())
    {
        var services = scope.ServiceProvider;

        try
        {
            var db = services.GetRequiredService<ApplicationDbContext>();
            db.Database.Migrate();
        }
        catch (Exception ex)
        {
            var logger = services.GetRequiredService<ILogger<Program>>();
            logger.LogError(ex, "An error occurred while migrating the database.");
        }
    }

    webHost.Run();
}

/cc @shawnwildermuth who was using Startup to seed the database

[markrendle]

@bricelam Thank you for the clarification, I shall use that pattern.

[bricelam]

@Tratcher Had a cool idea to make this an extension method.

public static IWebHost MigrateDatabase(this IWebHost webHost)
{
    using (var scope = webHost.Services.CreateScope())
    {
        var services = scope.ServiceProvider;

        try
        {
            var db = services.GetRequiredService<ApplicationDbContext>();
            db.Database.Migrate();
        }
        catch (Exception ex)
        {
            var logger = services.GetRequiredService<ILogger<Program>>();
            logger.LogError(ex, "An error occurred while migrating the database.");
        }
    }

    return webHost;
}

public static void Main(string[] args)
{
    BuildWebHost(args)
        .MigrateDatabase()
        .Run();
}

[ivankarpey]

@bricelam It's not clear for me what is the purpose of implicit call of BuildWebHost method? Why it called on every operation?

[bricelam]

You want #9076

[memark]

@bricelam Thanks for this solution! It makes perfect sense to me to put pipeline code in Startup.Configure(), and application code in Program.Main().

However, with that definition in place, why should EF Tools then ever want to run the code to "set up the pipeline"? (In addition to ConfigureServices().)

[Tratcher]

@memark EF doesn't 'want' to run Startup.Configure(), it's just a side effect of the new BuildWebHost pattern. It runs all of Startup before it returns the services that EF needs.

[pcbjorklund]

Hi,
I've got a question about the IDesignTimeDbContextFactory.
Previously i had to implement IDesignTimeDbContextFactory to get migrations running, e.g: PM > Add-Migration Initial PM > Update-Database

If not, the console threw an error and led me here: https://docs.microsoft.com/en-us/ef/core/miscellaneous/configuring-dbcontext#use-idesigntimedbcontextfactory.

So i did what it suggested and got migrations running. After that i have created new projects, and i didn't have to implement the IDesignTimeDbContextFactory. Migrations worked anyway. How is this possible? Same .NET Core version (2.0) on all of the projects.

Do we always need to create a IDesignTimeDbContextFactory if we want to use migrations from the Package Manager Console, or is it just in certain situations?

The below did't work at first, but once implementing the IDesignTimeDbContextFactory in one project, all the following new projects worked without it. Coincidence or did something in the background got updated?

Startups ConfigureServices:
services.AddDbContext<ApplicationDbContext>(options => options.UseSqlServer("the_connection_string"));

DbContext class
public class ApplicationDbContext : DbContext { public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options) : base(options) {} public DbSet<Location> Locations { get; set; } }

Thanks!

[bricelam]

If you have a default constructor in your DbContext or are using the Program.BuildWebHost() pattern established in the ASP.NET Core 2.0 project templates, you typically won't need an IDesignTimeDbContextFactory implementation.

[pcbjorklund]

Thanks for the response @bricelam
I'm using the Program.BuildWebHost() (as it shipped with the project template for ASP.NET Core 2.0) and my constructor in my DbContext class is passing the DbContextOptions to the base (DbContext).
It all works well now, i'm just a bit confused why it all of a sudden started to work woithout the IDesignTimeDbContextFactory for new projects. When you say "typically", are there rare occasions when the IDesignTimeDbContextFactory could be needed even though running 2.0 and the BuildWebHost()?

[bricelam]

In 2.0.0-preview1 Program.BuildWebHost() wasn't used and you needed a design-time factory.

[pcbjorklund]

Okay, thank you for the information @bricelam

[DavidNorena]

Hi @bricelam, is there a way to resolve serivces configured inside a IDesignTimeDbContextFactory Implementation ?

[bricelam]

Resolve them where? You may be able to use the dbContext.GetService<T>() extension method.

[leopignataro]

@bricelam , this "Application startup code should go inside Program.Main()" recommendation is a huge deal. If this is indeed coming from Microsoft, this should definitely be clearly stated on MS docs. Think of how many projects there are out there calling db.Database.Migrate() (among many other things) on Configure!

[bricelam]

There are improvements coming in version 2.1. Startup.Configure() won't be called when things like EF are just trying to access the IServiceProvider. (see aspnet/Hosting#1263)

[memark]

@leopignataro @bricelam Indeed! Apart from this (rather obscure) thread I have never seen this mentioned before.

[leopignataro]

Found an official recommendation:

https://docs.microsoft.com/en-us/aspnet/core/migration/1x-to-2x/#move-database-initialization-code