Efcore: Commands: A way to see current Migrations status

[bricelam]

We should enable the following scenarios:

• What migration have been applied to the database?
• Which ones are pending?
• Do I have unscaffolded model changes?

[Psypher9]

@bricelam I am actually working on a prototype for the first two scenarios. Would this still be a helpful enhancement? I may be able to have a PR ready next week for this. The third scenario is going to take more research, but I have some ideas.

[bricelam]

Yeah, I'd be happy to help get this feature in. The diagnostics page does a check for pending model changes here.

[bricelam]

Can we review your design before you send a PR? Just a comment here about what command(s) you run and what the output will look like should be enough for us to discuss.

[Psypher9]

Yeah definitely. I would look forward to that discussion. I have been studying the work that you guys have done, both in the PSM and in ef.dll, so that I can follow the methodology closely.

[Psypher9]

Yeah, I'd be happy to help get this feature in. The diagnostics page does a check for pending model changes here.

This is helpful to know! Thanks for pointing this out.

[ErikEJ]

EF Core Power Tools also has some code to get Migrations status...

[Psypher9]

Still working on this feature. I thought I'd be done by now, but I'm not quite ready yet.

One thing I'm thinking through is, on the check for unscaffolded changes, what would be most helpful? Should that command return a message saying "yes, there are unscaffolded changes", a list of the class names where the changes occurred, or a list of the DbSet names? I'm not sure which of these options would be best.

I'm also thinking about having a follow up question for the user to ask "Would you like to add a migration now? Y/N", then use Add-Migration if Y.

[bricelam]

My initial two cents on the design:

We could update dotnet ef migrations list to show whether a migration is applied or not and warn if there are pending model changes.

> dotnet ef migrations list --prefix-output
20181018102100_InitialCreate
20181018102101_SomeAppliedMigration
20181018102102_SomeUnappliedMigration (Not applied)

There are pending model changes. Add a new migration.

We'll need a corresponding Get-Migration command in PMC

We should also add the pending model changes warning to dotnet ef database update and Update-Database.

For programmability, we'd need to enhance the JSON:

> dotnet ef migrations list --json
[
    { "id": "20181018102100_InitialCreate", "applied": true },
    { "id": "20181018102101_SomeAppliedMigration", "applied": true },
    { "id": "20181018102102_SomeUnappliedMigration", "applied": false }
]

To get the pending changes information, we could add it to dotnet ef dbcontext info and Get-DbContext.

> dotnet ef dbcontext info --json
{
    "pendingChanges": true
}

[Psypher9]

Okay cool, I see what you mean now. That makes sense to me. Luckily, the Get-Migrations command was actually the first things I started working on in the PMC.

[Psypher9]

Would this be best as separate PRs for each piece of the puzzle?

[bricelam]

@Psypher9 It's up to you, but before you send the PR(s), I should discuss the design with the team to see if they have any feedback.

[Psypher9]

@bricelam Okay, no problem. I'll let you know

[Psypher9]

@bricelam Okay, I think I have the Get-DbContext changes made on my fork's master branch.

Do you want to take a look at it?

[bricelam]

I reviewed the design with the team today and they didn't have any objections (except the name Get-Migration but that's easy to change later).

I haven't looked at your branch, but sending it as a PR would be a good way to start iterating on the implementation.

[Psypher9]

Okay, so here's where I'm at on this issue. I've been trying to get the List-Migrations command to run on my test project using a local package of EFCore.Tools but, after I install the local package, powershell doesn't recognize my new command. I think that maybe I'm overlooking something, but would anyone have any tips on working with the packages locally? @bricelam, any ideas?

I updated the description in EfCore.Tools/tools/about_EntityFrameworkCore.help.txt , and can see my new List-Migrations there when I open NuGet Package Manger in VS , like this.
image

but, when I go to run the command in my test project PowerShell can't find it.

It seems like my local feed shows the package metadata, but that I'm actually getting an older version of the underlying package contents.

Which is what this seems to suggest:

But, I'm still not sure how I can fix this. I have even gone so far to manually remove all NuGet feeds from VS except my local feed and removing the default feed from the .csproj file and only leaving $(RestoreSources) or even hardcoding the path to my local feed; though, still to the same effect.

I just haven't been able to get my changes tested properly. What can I do to make sure I'm getting the right packages? This is where I got stuck back in February and keeping hitting a wall.

I am grateful for any insights!

[Psypher9]

Here's an update, I found that, for whatever reason, when I used nuget init to create a controlled feed, the underlying NuGet packages seem to be coming from another location. However I redirected to the default local feed after I noticed that the 3.0.0-dev packages were there too. I can get the right version now, but it looks like I need to update the PowerShell module manifest.

Is that something that you guys have a tool you'd like me to use for standardization?

[bricelam]

To debug, I usually...

• Run build -pack in the EF Repo
• Add C:\Path\to\EntityFrameworkCore\artifacts\packages\Debug\Shipping as a NuGet package source
• Install-Package Microsoft.EntityFrameworkCore.Tools -Version 3.0.0-dev

You can add a call to Debugger.Launch() inside the Program.Main of ef.exe or the constructor of OperationExecutor to get a debugger attached when running commands.

[Psypher9]

Okay great! I will give that a shot. Admittedly, I should have thought about that earlier.

[br3nt]

These will be very useful commands!

I hope you don't mind if I suggest two additional commands for the following scenarios:

• Check that all applied migrations exist in the assembly
• Check that pending migrations will be applied chronologically after applied migrations

An applied migration may be missing from an assembly if a migration from one git branch is applied to the database, and then the user switches to another branch without rolling-back the migration from the other branch. This can happen for example when deploying feature branches to testing environments.
It would be nice to be able have CI tools test for this scenario and fail before performing a migration.

Likewise, a pending migration with a timestamp earlier than the timestamp of one or more applied migrations will occur when one git branch contains migrations that have been applied is then merged into another git branch containing pending migrations with earlier timestamps. This also seems to happen when deploying feature branches to testing environments.

Both of these scenarios can be frustrating to fix, so a command to prevent it from occurring in the first place would be very welcome.

Both of these scenarios can be checked programatically:

```C#
// Check that all applied migrations exist in the assembly
var assemblyMigrations = context.Database.GetMigrations();
var appliedMigrations = context.Database.GetAppliedMigrations();
var pendingMigrations = context.Database.GetPendingMigrations();
if (appliedMigrations.Any(a => !assemblyMigrations.Contains(a)))
throw new Exception("There are applied migrations that do not exist in this assembly. All applied migrations must exist in the assembly. Aborting the migration.");

```C#
// Check that pending migrations will be applied chronologically after applied migrations
var appliedTimestamps = appliedMigrations.Select(m => Convert.ToUInt64(m.Substring(0, 14)));
var pendingTimestamps = pendingMigrations.Select(m => Convert.ToUInt64(m.Substring(0, 14)));
if (appliedTimestamps.Any(a => pendingTimestamps.Any(p => a > p)))
    throw new Exception("There are pending migrations with a timestamp less than the timestamps of one or more applied migrations. Migrations must be applied chronologically. Aborting the migration.");

[bricelam]

Splitting out the pending model changes feature into #22105