Ef6: Configuration seed method aware of migrations applied.

[JoseCMRocha]

Possibility to have the knowledge of the migrations applied during an update on the method seed of the class configuration.

Right know there is no easy way to know what migrations where applied when running an upgrade, so we can use that information to run only the seed/s for the target migration.

The most common non-invasive solutions you see for this are creating a DataBaseInitializer that is useless running the upgrade from the package manager console, or creating a class from DbMigrator on the class Configurator, what in my opinion is simple wrong since the DbMigratoralready exists and it will bring a lot of other problems with the context.

So what I'm proposing a new method override on the class DBMigrationsConfiguration, that has as parameter the context and a list with the id's of the migrations applied, very simple and noninvasive solution protected virtual void Seed(TContext context, IEnumerable<string> migrationsApplied) as a new signature that if it is not overrided by the user class it calls the previous protected virtual void Seed(TContext context).

This will change the way we use the seeder\s today, making them more useful, or just stick with the way it were before, that's up to the user choice.

A simple note is that it this a nice to have would be a extension similar with the AddOrUpdate called RemoveIfExist the name is self explanatory.

Ps. This is a feature "request" discussion, so I can decide the usefulness of this changes for others, and make pull a request of a fork already with this changes.

[colltoaction]

Hi Jose,

I'm currently working on a seed data proposal in EF Core: https://github.com/aspnet/EntityFramework/issues/629#issuecomment-282368487. I'm not suggesting we should do this in EF6, but anyway I'd like to know if this addresses your use case. The main idea is to move the data seeding code to the migration itself.

Thanks!

[JoseCMRocha]

Hi tinchou,

I have read trough your proposal, and the its a good proposal but in a different context. In EF6 there is already a way of calling seeders on upgrade migrations from the method Seed on the class Configuration, what I was proposing here was a more simple way to have an overload of that method and instead of have a method that just receives the context, an overload to receive the context and a list with the ids of the migrations applied.

Your proposal is great but it implies a major development.

[colltoaction]

Thank you for taking a look! I'm implementing that proposal on EF Core, and since it's different from EF6 data seeding I want to be sure I'm covering the same and new scenarios.

[smithaitufe]

@tinchou Thanks so much for your interest in seeding. I love the ideas you proposed.

I will like to know the timeline for this and if there is a way I can contribute.

[colltoaction]

Hey Samuel, thanks for your interest.

I just posted a demo on the EF Core thread. Feel free to test it and report any bugs you find!

One interesting thing would be to add support for Navigations, and I don't think the team will be doing that any time soon. I'll make sure we create a bug tracking it and put an up-for-grabs label, in case you or anyone would like to tackle it.

[ajcvickers]

@JoseCMRocha We have been discussing this today but have been having trouble recognizing the value that it adds. Can you elaborate a bit on how you will use the migration IDs? In might help to explain if you could show some example code of how using the IDs will improve your Seed method.

[JoseCMRocha]

Of course @ajcvickers, going to try explain it by example.
Lets assume we have 4 migrations, A B C and D respectively, and a seeder file called seederA that was created on the migration A and it is called on the Configurator class like

protected override void Seed(EfDbContext context){
    new SeederA().Seed(context);
}

Inside this seederA there is an AddOrUpdate of the class foo that contains three fields (Id, Name, Description) and I'm inserting a row with (Name = "Boo", Description = "BooDesc").
Now I'm running the application and at this point I only have the migration A, and an edition is made on the row description to "BooDescTest". I stop the application and run it again now with the migration B, the seederA will run again making the row with the description "BooDescTest" goes to "BooDesc". In the case that the name is the field that was edited, and the identifier expression of the method AddorUpdate on the seederA is the field name it will insert a new row with the name "Boo" instead of editing the row that already exists .
Now if the code was aware of the applied migration during the update the seeders could be called like:

protected override void Seed(EfDbContext context, IEnumerable<string> migrationsApplied)
{
    foreach(var migrationId in migrationsApplied){
          switch(migrationId){
               case 'A':
                     new SeederA().Seed(context);
               case 'B':
                     new SeederB().Seed(context);
               ...
          }
    }
}

So instead of having to remove the seederA or change it i can still have the seederA but only call it when that migration A is being applied.

Other use case could be that for systems that are using the migration C I have a table that contains 5 rows but this 5 rows are not necessary anymore on systems running the migration D so I want to clean this 5 rows when i migrate to D but add them when i migrate to C.

Other use case is simply if the method seed is called with an empty migrationsApplied I know that i should not run any seeder preventing this way reversion of the data, because on each start of the application the initializer calls the seed method.

This way i can Migrate not only the structure but the content to, so it can evolve with the structure.

[mbaaz]

I can see a great use of this feature.
In many projects I work on, I want to make content changes to the database as well as structure changes. Migrations are good on dealing with structure changes, while the Seed-method is great for content.

Some content has to be modified when a migration is applied. It could be that when a column is added to a table, I want to calculate the default value to insert for each row currently in the database. That does not mean that I want this calculation to happen every time I start the application (thus overriding any manual changes).

Today I have to use SQL-scripts added to the migrations to be able to modify the content. It would be preferred to be able to use the context, so I can use my managed code to perform these updates. But the context is only available in the Seed-method. So having the Seed-method knowing about which migrations has been applied would greatly benefit

[ajcvickers]

Thanks for the more detailed information. We have discussed this again and we are concerned that what is being described won't really work in practice. There are two main reasons for this:

• The CLR types and the underlying model are always going to be for the latest/current model. This means trying to perform operations related to previous iterations of the model will be at best confusing and at worst impossible because the appropriate mappings will not be available.
• Seed is only ever called when migrating to the last migration, so when, for example, going down to C, the seed method will not be called.

All that being said, it seems that what you are asking for here is the ability to have migrations manipulate data as well as the schema. This is totally reasonable, but in order to make it a good experience the data manipulation needs to be part of the migration. It also needs to be not bound directly to the model CLR types, since these will change, causing previously created migrations to break. For EF6, this means that in reality the data manipulation should be added to the migrations and should be implemented in SQL. For EF Core, the work referred to above by @tinchou creates a better experience in that C# calls can be used and these are translated into statements in the migrations themselves, so there is no need to write the SQL directly. However, this is not yet completed for EF Core, and we don't plan to port this work to EF6.

[JoseCMRocha]

Hello and thanks for your time, yes I do to believe that the work referred by @tinchou would be better than my proposal.

In relation to the 2 points refereed by you, I agree with the first one that if one choose to use the seeder logic that I explained, it may need to rework the old seeders to remove columns that were dropped or types of the columns that may change, but, it is any different from the way it is now? If one uses the seeder in configuration as it is now, and there is a dropped column in the table or a change in the column type he still will have to change the seeders as entity framework 6 is at the moment.
The second point I might be wrong but, as EF6 it is now, every time I start/deploy/run the application the method seed will always run and not only when upgrading to last migration. Going down will not call the seeder , but using the package manager console I can downgrade 2 migrations and upgrade 1 migration using the example A, B, C and D going from D to B and after upgrade to C this will call the seeder of the migration C.
My suggestions, at least for the 2 points that you presented here does not interfere in any way as the work one have to do right now in the EF6 if he changes something in the classes that construct the table because that work as to be done anyway. My suggestion only improves a little bit the possibilities in how to use the seeder functionality, because every one can still use the old way.

Just to finish I would to emphasize that this is something not only me have the need. I searched for a possible solution for this and there are other people with this problem, and the solutions that they run in or it is to create a custom DataBaseInitializer so they can create their on instance of the DbMigrator and this way being able to get the pending migrations, but loosing control of the options in the package manager control. or on the class Configuration creating a new instance of the class DbMigrator using the class configuration it self in other words calling var migrator = new DbMigrator(this); and contolling the stackoverflow with static flags not to create the class again in a infinite loop. I only decide to work this out because none of the solution is acceptable for me.

I respect the decision. But by the two points given, I still believe that my proposal is an enhancement.

[mbaaz]

I have to say I agree with both of you, @JoseCMRocha and @ajcvickers.

In most cases, it is a bad idea to rely on CLR objects for the seed data, since that might mean you need to re-work old seeders to be working with the current structure of the objects. However, as @JoseCMRocha points out, this is no different than what we need to do in the Seed-method today. This addon will just make sure we get some more data to determine what seeders to run.

I still think this proposed addon is beneficial for my use cases, even though I technically can understand that sql scripts is the better way to go. But the lazy programmer in me likes to use the clr objects and the already built repositories for making sure data is entered in a uniform state.

[smithaitufe]

I really don't know why this is an issue for us in. Net

In Elixir/Phoenix, seed is as simple as mix run /priv/migrations/seed.exs whether in development or production environment.

I am not really following the debate that has been for months but I know this is a need.