Efcore: 5.0 RC1 Problems in migrations

[mhosman]

Upgrading from 3.1 to 5.0 RC1.
Migrations just not work. It's working ok in 3.1 and previous versions.

gsql.PostgresException (0x80004005): 42P01: relation «companies» does not exist
   at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.ExecuteReaderAsync(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery()
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQuery(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Migrations.MigrationCommand.ExecuteNonQuery(IRelationalConnection connection, IReadOnlyDictionary`2 parameterValues)
   at Microsoft.EntityFrameworkCore.Migrations.Internal.MigrationCommandExecutor.ExecuteNonQuery(IEnumerable`1 migrationCommands, IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.Migrate(String targetMigration)
   at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.UpdateDatabase(String targetMigration, String connectionString, String contextType)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.UpdateDatabaseImpl(String targetMigration, String connectionString, String contextType)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.UpdateDatabase.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
  Exception data:
    Severity: ERROR
    SqlState: 42P01
    MessageText: relation «companies» does not exist
    File: d:\pginstaller_12.auto\postgres.windows-x64\src\backend\catalog\namespace.c
    Line: 426
    Routine: RangeVarGetRelidExtended
42P01: relation «companies» does not exist

[ajcvickers]

/cc @bricelam @roji

@mhosman Can you post more details? Preferably an application where we can reproduce the error.

[mhosman]

Thanks @ajcvickers The problem is that is generating tables with relations before creating the "parent" table. Example, is generating a "users" table with a "companyId" before creating the Companies table. In the migration file, if I manually change the "companies" table creation to the top, it works. I'll try to reproduce in a clean repo.

[tinchoel10]

Woow I got exactly the same issue

[mhosman]

I'm trying to reproduce it in a clean repo, but I must say... Don't know why, but sometimes, versions <= .NET Core 3.1 are not creating foreign keys while 5.0 is creating all foreign keys, but the thing is.... is trying to insert this relation in the users table:

table.ForeignKey(
        name: "fk_users_companies_company_id",
        column: x => x.company_id,
        principalTable: "companies",
        principalColumn: "id",
        onDelete: ReferentialAction.Restrict);

Before creating the companies table....
So... migration fails

[mhosman]

@roji @ajcvickers @bricelam
Finally I found the issue. This is only happening when I try to use a Collation using this code:

modelBuilder.HasCollation("my_collation", locale: "en-u-ks-primary", provider: "icu", deterministic: false);
modelBuilder.UseDefaultColumnCollation("my_collation");

If I remove those lines, I don't get any issues and the migration succeeds.
I don't know if it is a .NET Core or Npgsql problem.

[programmer1718]

I too upgraded from 3.1 to 5.0 RC1, and am also having issues with adding migrations...
I wasn't getting this issue in 3.1, though I did make some changes to my models with many-to-many relationships.
Previously I had a bunch of models that acted as join tables, but removed them on the upgrade to 5.0 RC1 because I wanted to test its ability to handle that.
Also, I am using Npgsql 5.0 RC1.

I can't quite figure it out, or determine if it's related, but I get this:

Build started...
Build succeeded.
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.ForeignKeyPropertyDiscoveryConvention.DiscoverProperties(IConventionForeignKeyBuilder relationshipBuilder, IConventionContext context)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.ForeignKeyPropertyDiscoveryConvention.ProcessSkipNavigationForeignKeyChanged(IConventionSkipNavigationBuilder skipNavigationBuilder, IConventionForeignKey foreignKey, IConventionForeignKey oldForeignKey, IConventionContext`1 context)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnSkipNavigationForeignKeyChanged(IConventionSkipNavigationBuilder navigationBuilder, IConventionForeignKey foreignKey, IConventionForeignKey oldForeignKey)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnSkipNavigationForeignKeyChangedNode.Run(ConventionDispatcher dispatcher)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.DelayedConventionScope.Run(ConventionDispatcher dispatcher)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Run()
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Dispose()
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnModelInitialized(IConventionModelBuilder modelBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnModelInitialized(IConventionModelBuilder modelBuilder)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.Model..ctor(ConventionSet conventions, ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.ModelBuilder..ctor(ConventionSet conventions, ModelDependencies modelDependencies, Boolean _)
   at Microsoft.EntityFrameworkCore.ModelBuilder..ctor(ConventionSet conventions, ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.CreateModel(DbContext context, IConventionSetBuilder conventionSetBuilder, ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.GetModel(DbContext context, IConventionSetBuilder conventionSetBuilder, ModelDependencies modelDependencies)
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel()
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
   at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__7_3(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
   at Microsoft.EntityFrameworkCore.DbContext.get_InternalServiceProvider()
   at Microsoft.EntityFrameworkCore.DbContext.Microsoft.EntityFrameworkCore.Infrastructure.IInfrastructure<System.IServiceProvider>.get_Instance()
   at Microsoft.EntityFrameworkCore.Infrastructure.Internal.InfrastructureExtensions.GetService[TService](IInfrastructure`1 accessor)
   at Microsoft.EntityFrameworkCore.Infrastructure.AccessorExtensions.GetService[TService](IInfrastructure`1 accessor)
   at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.CreateContext(Func`1 factory)
   at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.CreateContext(String contextType)
   at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.AddMigration(String name, String outputDir, String contextType, String namespace)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigrationImpl(String name, String outputDir, String contextType, String namespace)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigration.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
Object reference not set to an instance of an object.

[roji]

@programmer1718 your error doesn't seem related to this issue in any way - this issue is about relation «...» does not exist, not Object reference not set to an instance of an object. Can you please open a new issue, and include a runnable code sample that we can use to reproduce the problem?

[roji]

@mhosman it's difficult to piece together what's going on from the bits and pieces... From what I can understand above, you've tried to add a new dependent with a foreign key to an existing principal, and are having issues; I've tried with the below scenario and couldn't succeed. Could you please look at the below and try to provide a similar complete code sample which would allow us to reproduce the problem?

I tried reproducing by:

• Having a minimal model with a Companies table and a default collation, and creating an initial migration from that.
• Adding a dependent Users, and creating a second migration from that.
• Applying the migrations by executing dotnet ef database drop && dotnet ef database update

The database was created and the migrations were applied without any issues.

(we're getting close to the 5.0, so it's important we move fast if we want to fix anything at this point)

Program and model

```c#
class Program
{
static void Main(string[] args) {}
}

public class BlogContext : DbContext
{
public DbSet Companies { get; set; }

static ILoggerFactory ContextLoggerFactory
    => LoggerFactory.Create(b => b.AddConsole().AddFilter("", LogLevel.Information));

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    => optionsBuilder
        .UseNpgsql(@"Host=localhost;Username=test;Password=test")
        .EnableSensitiveDataLogging()
        .UseLoggerFactory(ContextLoggerFactory);

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.HasCollation("my_collation", locale: "en-u-ks-primary", provider: "icu", deterministic: false);
    modelBuilder.UseDefaultColumnCollation("my_collation");
}

}

public class Company
{
public int Id { get; set; }
public string Name { get; set; }
// public ICollection Users { get; set; } // Enabled only after creating the first migration
}

// Enabled only after creating the first migration
// public class User
// {
// public int Id { get; set; }
// public string Login { get; set; }
// }

</details>

<details>
<summary>Output from dotnet ef database update</summary>

```c#
warn: Microsoft.EntityFrameworkCore.Model.Validation[10400]
      Sensitive data logging is enabled. Log entries and exception messages may include sensitive application data, this mode should only be enabled during development.
info: Microsoft.EntityFrameworkCore.Infrastructure[10403]
      Entity Framework Core 5.0.0-rc.1.20451.13 initialized 'BlogContext' using provider 'Npgsql.EntityFrameworkCore.PostgreSQL' with options: SensitiveDataLoggingEnabled 
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (49ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE DATABASE test;
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (3ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE TABLE "__EFMigrationsHistory" (
          "MigrationId" character varying(150) NOT NULL,
          "ProductVersion" character varying(32) NOT NULL,
          CONSTRAINT "PK___EFMigrationsHistory" PRIMARY KEY ("MigrationId")
      );
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (7ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid=c.relnamespace WHERE c.relname='__EFMigrationsHistory');
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (1ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      SELECT "MigrationId", "ProductVersion"
      FROM "__EFMigrationsHistory"
      ORDER BY "MigrationId";
info: Microsoft.EntityFrameworkCore.Migrations[20402]
      Applying migration '20200919154847_initial'.
Applying migration '20200919154847_initial'.
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (7ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE COLLATION my_collation (LC_COLLATE = 'en-u-ks-primary',
          LC_CTYPE = 'en-u-ks-primary',
          PROVIDER = icu,
          DETERMINISTIC = False
      );
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (3ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE TABLE "Companies" (
          "Id" integer NOT NULL GENERATED BY DEFAULT AS IDENTITY,
          "Name" text COLLATE my_collation NULL,
          CONSTRAINT "PK_Companies" PRIMARY KEY ("Id")
      );
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (0ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      INSERT INTO "__EFMigrationsHistory" ("MigrationId", "ProductVersion")
      VALUES ('20200919154847_initial', '5.0.0-rc.1.20451.13');
Applying migration '20200919154906_user'.
info: Microsoft.EntityFrameworkCore.Migrations[20402]
      Applying migration '20200919154906_user'.
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (4ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE TABLE "User" (
          "Id" integer NOT NULL GENERATED BY DEFAULT AS IDENTITY,
          "Login" text COLLATE my_collation NULL,
          "CompanyId" integer NULL,
          CONSTRAINT "PK_User" PRIMARY KEY ("Id"),
          CONSTRAINT "FK_User_Companies_CompanyId" FOREIGN KEY ("CompanyId") REFERENCES "Companies" ("Id") ON DELETE RESTRICT
      );
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (1ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      CREATE INDEX "IX_User_CompanyId" ON "User" ("CompanyId");
info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (0ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      INSERT INTO "__EFMigrationsHistory" ("MigrationId", "ProductVersion")
      VALUES ('20200919154906_user', '5.0.0-rc.1.20451.13');
Done.

[programmer1718]

@roji Sure, I will create a new issue and provide more details.

[mhosman]

Hey @roji @ajcvickers @bricelam

First of all my apologies since the problem was not related to the collations.
Here I was able to reproduce the problem in a small project attached below.

Thank you!
TestAPI.zip

[roji]

Thanks, this reproduced for me in RC1, but not in the latest RC2 daily builds (nor in 3.1). Can you please try using the latest RC2 as per these instructions, and confirm that everything works fine?

Maybe @AndriySvyryd or @ajcvickers know to dedup this based on the code sample below.

Minimal repro code

```c#
public class Program
{
public static async Task Main(string[] args)
{
await using var ctx = new DataContext();
await ctx.Database.EnsureDeletedAsync();
await ctx.Database.EnsureCreatedAsync();
}
}

public class DataContext : DbContext
{
static ILoggerFactory ContextLoggerFactory
=> LoggerFactory.Create(b => b.AddConsole().AddFilter("", LogLevel.Information));

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    => optionsBuilder
        .UseSqlServer(...)
        .EnableSensitiveDataLogging()
        .UseLoggerFactory(ContextLoggerFactory);

public DbSet<Order> Orders { get; set; }
public DbSet<OrderCode> OrderCodes { get; set; }
public DbSet<User> Users { get; set; }
public DbSet<Role> Roles { get; set; }

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<OrderCode>()
        .HasOne(c => c.CreatedByUser)
        .WithMany()
        .HasForeignKey(c => c.CreatedByUserId)
        .OnDelete(DeleteBehavior.Restrict);

    modelBuilder.Entity<Order>(builder =>
    {
        builder.HasOne(c => c.CreatedByUser).WithMany().HasForeignKey(c => c.CreatedByUserId).OnDelete(DeleteBehavior.Restrict);
        builder.HasOne(c => c.OrderCodeA).WithMany().HasForeignKey(c => c.OrderCodeAId).OnDelete(DeleteBehavior.Restrict);
        builder.HasOne(c => c.OrderCodeB).WithMany().HasForeignKey(c => c.OrderCodeBId).OnDelete(DeleteBehavior.Restrict);
    });

    modelBuilder.Entity<Role>()
        .HasOne(c => c.CreatedByUser)
        .WithMany()
        .HasForeignKey(c => c.CreatedByUserId)
        .OnDelete(DeleteBehavior.Restrict);

    modelBuilder.Entity<User>(builder =>
    {
        builder.HasOne(c => c.Role).WithMany().HasForeignKey(c => c.RoleId).OnDelete(DeleteBehavior.Restrict);
        builder.HasOne(c => c.CreatedByUser).WithMany().HasForeignKey(c => c.CreatedByUserId).OnDelete(DeleteBehavior.Restrict);
    });
}

}

public class Order
{
public int Id { get; set; }
public int? CreatedByUserId { get; set; }
public User CreatedByUser { get; set; }
public int? OrderCodeAId { get; set; }
public OrderCode OrderCodeA { get; set; }
public int? OrderCodeBId { get; set; }
public OrderCode OrderCodeB { get; set; }
}

public class OrderCode
{
public int Id { get; set; }
public int? CreatedByUserId { get; set; }
public User CreatedByUser { get; set; }
}

public class User
{
public int Id { get; set; }
public int RoleId { get; set; }
public Role Role { get; set; }
public int? CreatedByUserId { get; set; }
public User CreatedByUser { get; set; }
}

public class Role
{
public int Id { get; set; }
public int? CreatedByUserId { get; set; }
public User CreatedByUser { get; set; }
}
```

[mhosman]

Hey @roji thank you!! If you try in 3.1 the error is not happening because is not generating the foreign key. I'm gonna try this with RC2 an I'll tell you later. Thanks!!!!

[mhosman]

OK in RC2 that small project is working okay (don't know what changed), but my big project still fails so I'll need to start reducing code again to get the minimal code to reproduce the issue.... I'll start doing that.

[sandord]

I ran into the same problem with rc1. After upgrading to 5.0.0-rc.2.20472.13, the problem went away for me as well.

[mhosman]

I confirm is working on latest RC2 5.0.0-rc.2.20474.5. Great job!!! Closing

[roji]

Thanks for confirming!