Efcore: Remove-Migration does not remove last migration and does not restore ModelSnapshot

[MichaelPruefer]

I have an asp.net core web api with 2 Contexts

when running (Package Manager Console)
add-migration Test -context AppIdentityDbContext
and then
remove-migration -context AppIdentityDbContext

the generated migration file still exists

The verbose output from remove-migration ends with:

Executing 456 update commands as a batch.
A manual migration deletion was detected.
Reverting model snapshot.
Done.
'AppIdentityDbContext' disposed.

When there are real model changes and the modelsnapshot is changed, remove-migration does not revert those changes.

Changes that are generated (every time) are e.g.:
c# migrationBuilder.UpdateData( table: "AspNetRoles", keyColumn: "Id", keyValue: new Guid("5de8d15a-02c5-4042-b7ed-a4a77f5a8f3a"), columns: new[] { "CreatedAtUtc", "ModifiedAtUtc" }, values: new object[] { new DateTime(2019, 6, 1, 10, 0, 0, 0, DateTimeKind.Utc), new DateTime(2019, 6, 1, 10, 0, 0, 0, DateTimeKind.Utc) });
CreatedAtUtc, ModifiedAtUtc are additional properties on our custom Role class which is derived from IdentityRole<Guid>.
These values are generated from DateTime.Parse(...) and then spcifing DateTimeKind.Utc

And everytime i add a new migration i get the same changes to be applied

Further technical details

dotnet core skd: 3.1.300
EF Core version: 3.1.4
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: .NET Core 3.1)
Operating system: Windows 10
IDE: Visual Studio 2019 16.6.1

[ajcvickers]

Related to #20666 and #18943 and many closed duplicates of these, but non seem to be exactly the same case.

[MichaelPruefer]

Postpone it to a 5.0.0 Milestone is a real bad thing for me.
At the moment a empty migration will generate 10k lines of rubbish.

If i have a real migration with updated columns, data etc. it will be horrible to find these real changes an double check if they are correct.

So why will it fixed that late?

[ajcvickers]

@MichaelPruefer See the release planning process.

[MichaelPruefer]

Updated some other project from EF Core 2.2.6 to EF Core 3.1.5
First Add-Migration produces same changes as mentioned above, including some of

migrationBuilder.AlterColumn<DateTime>(
                name: "ModifiedAtUtc",
                table: "Regions",
                nullable: false,
                oldClrType: typeof(DateTime),
                oldDefaultValueSql: "GETUTCDATE()");

A second Add-Migration produces same changes like the migration before but without the AlterColumn statements.

In this image you can see default values being updated. These update statements are now in every migration i create.
It's made of the second migration, directly run after the first run. 12k lines, 6k lines up, 6k down, pulluting our versioning.

Both migrations can't be undone with remove-migration.

Running remove-migration with -verbose I'm getting following output:

A manual migration deletion was detected.

But I have not deleted any migration manually.

The Column definition for ModifiedAtUtc is like

    builder.Property(p => p.ModifiedAtUtc).IsRequired().HasDefaultValueSql("GETUTCDATE()");

The Column definiiton for IsExportAllowed is like

    builder.Property(p => p.IsExportAllowed).HasDefaultValue(false).IsRequired();

[MichaelPruefer]

Can confirm that in EF Core 3.0.3 everything works as expected.
With 3.1.0 i have the problems mentioned above.

[MichaelPruefer]

@ajcvickers I have a big problem:
because of #21146 (this bug) i can't use 3.1.x
because of #17422 i can't use 3.0.x
and no defnitive decision whether #21146 will be fixed in 3.1.x although 3.1 is a LTS release

[stephan-bauer-hetras]

The problem seems to be even worse. If there are multiple migrations in a project and the last one is removed as mentioned above the migration still exists AND in model context snapshot the changes from second last migration are reverted. Everytime the snapshot model context needs to be modified manually which is a potential risk for errors.
@ajcvickers: To the discussion above I think fixing a bug should be part of a patch release. We are talking here about functionality which isn't working as intended!

[AndriySvyryd]

The redundant UpdateData is fixed in https://github.com/dotnet/efcore/commit/a8f0f4c4dfc94975bd318a6b5a5d3a627df1d5ba

[bricelam]

@MichaelPruefer Could you share a repro project so I can investigate the remaining issues you mention?

[MichaelPruefer]

@bricelam I will do that in the next days

[TadGhostal66]

Postponing any bug fix until 5.0 milestone should not be an option - 3.1 is LTS, and that means bug fixes.

[bricelam]

A manual migration deletion was detected.
Reverting model snapshot.

The redundant UpdateData was also confusing Remove-Migration. Both issues should be fixed now.

[tamirzamo]

After upgrading to latest .net core nuget packages to 3.1LTS, still having this issue.

When I Remove-Migration -Verbose, I get the "A manual migration deletion was detected." message,
And the migration is not removed from the snapshot + the migration file is not deleted.

Will appreciate any direction @bricelam

[TadGhostal66]

I assume this fix will be ported into 3.1, correct?

[ajcvickers]

@TadGhostal66 We currently have no plans to port this fix to 3.1. See release planning for more details.

[TadGhostal66]

Looking at the LTS policy vs Release Planning, it appears that there is little (if any) reason for designating a version to be LTS. What does the LTS designation provide that a non-LTS version doesn't? It can't be stability. LTS versions are designated as such when they are released, right? That means they are just as likely to be buggy as a non-LTS version.

With a non-LTS verson, critical bugs may be patched, and less-than-critical bugs might be fixed, but in a newer version. As I understand it, isn't that basically the same as an LTS version?

What exactly is the "support" in a Long-Term Support version?

Conversely, for open-source software like .NET core, what does EOL really mean anyway? How is .NET Core 2.2 different (as far as support goes) from the LTS 3.1? Both have known bugs that were decided to be fixed in later versions, meaning that for certain bugs, the only way to get the fixes is to upgrade.

According to the release planning mentioned above, "The bar to make a change in a patch release is very high.", and "This bar gradually rises through the lifetime of a long-term support (LTS) release." Doesn't that just mean that the older an LTS version gets, the less likely it is to be patched? That doesn't really sound like support (long-term or otherwise), but rather the opposite.

From what I see, an LTS version is actually _worse_ than a non-LTS version, because it only makes you _think_ staying on that version is good, rather than upgrading.

If your policy around fixing bugs in LTS versions isn't going to change, I'd recommend dropping the concept of LTS versions completely, as it seems to do little more than confuse developers who are familiar with what LTS typically means.

[bricelam]

What does the LTS designation provide that a non-LTS version doesn't?

Security updates for three years.

What does EOL really mean

No more security updates.

From what I see, an LTS version is actually worse than a non-LTS version

If it's features and bug fixes you're looking for, I would have to agree with you. Heck, use the daily builds if you can live with a bit of churn around new features as we develop them.

I'd recommend dropping the concept of LTS versions completely, as it seems to do little more than confuse developers

In the past, we've asked the .NET leadership for stronger and more consistent semantics around what LTS means across .NET products. We'll continue to advocate for our users and forward feedback like this along to them.

[bricelam]

BTW, there was an interesting discussion around this on the Entity Framework Community Standup last month.

[TadGhostal66]

Thanks for the info and the link. Good stuff all around.

[yakeer]

Also suffering from this bug,
If it was a minor bug, I would understand why it won't be ported to 3.1.
But that's a quite serious one,
From what I read here, my alternative is to upgrade to 5.0RC.. which seems to be very buggy, in order to be able to continue coding using Migrations.

I would really appreciate if this bugfix could be ported to 3.1. It was only broken in one of the latest minor builds, since it was working fine until not so long ago.

[ajcvickers]

@yakeer If you haven't already, then we'd really appreciate it if you could file the bugs you found using RC1 so we can fix them.

We will discuss again porting this to 3.1.

[yakeer]

Hey.
Thanks for the reopen.

I haven't yet updated to v5, as I'm developing enterprise systems integrated with our hardware, so we have enough bugs already ;)

Once v5 goes stable, I will start porting to it.

Thanks for the great work!

[ajcvickers]

@yakeer In that case can you provide more details on your statement, "which seems to be very buggy?" What is making you believe that?

[yakeer]

I take my words back on the "very buggy" part, just was noticing some issues after rc1 release that could definitely affect my systems.

[TadGhostal66]

Thanks for reopening. We made a large, concerted effort to upgrade from 2.x to 3.1, largely for the LTS status, and largely driven by myself. Unfortunately, this bug got noticed by a few devs about a month in. I'm _really_ not looking forward to saying, "it'll be fixed once we move to 5".

[ajcvickers]

Note from triage: @AndriySvyryd to prepare a 3.1 patch to take for approval.