Efcore: Improve user experience for running migrations after deployment

[natemcmaster]

Problem
Running migrations on deployment typically means you only have compiled output, not the project system. In theory, ef.exe/ef.dll should work well in this scenario, but figuring out all the arguments requires deep knowledge of how these tools work and is opaque to most users. Furthermore, ef.exe does not end up in compiled output, which requires users to do some NuGet cache spelunking to get the file.

Possible solutions

• generate response files at design-time that ef.exe can parse on deployment
• generate bash/cmd files that contain all the arguments for running an ef.exe command on compiled output
• add msbuild targets to optionally put ef.exe/ef.dll in build output on publish

[divega]

cc @rowanmiller we move this to backlog for now but we can discuss pulling it in 1.1.0 or whether we should have better documentation on how to do it manually.

[Allann]

Any progress on this? I am keen to use this in our application as we are ready to migrate to new environments. I notice that ef.exe is now part of preview 3 and 4 and whilst the --help for the console app is somewhat helpful, a more detailed explanation of the options and commands would be helpful.

[bricelam]

I've helped this scenario a little bit by logging our call to dotnet exec ef.dll in the PMC and CLI tools when -v (verbose) is specified. This is the same command you'd need to run on the server.

dotnet exec --depsfile C:\Projects\ConsoleApp1\bin\Debug\netcoreapp1.0\ConsoleApp1.deps.json --runtimeconfig C:\Projects\ConsoleApp1\bin\Debug\netcoreapp1.0\ConsoleApp1.runtimeconfig.json C:\Users\bricelam.nuget\packages\microsoft.entityframeworkcore.tools.dotnet\1.0.0-msbuild3-final\tools\netcoreapp1.0ef.dll database update --assembly C:\Projects\ConsoleApp1\bin\Debug\netcoreapp1.0\ConsoleApp1.dll

It's a bit less dramatic on .NET Framework:

C:\Users\bricelam.nuget\packages\microsoft.entityframeworkcore.tools.dotnet\1.0.0-msbuild3-final\tools\net451ef.exe database update --assembly C:\Projects\ConsoleApp1\bin\Debug\net451\ConsoleApp1.exe

[benday]

@bricelam -- Any updates on how to get this to work with the current version of EFCore? As a little background, I'm trying to do an assembly-based migration (not code-based) from TFS Release Management.

Thanks,
-Ben

[bricelam]

Look at the output of dotnet ef database update --vervbose to see how it calls ef.exe (or ef.dll)

[benday]

Thanks for the tip, @bricelam. To make it easier to call from an automated build or TFS Release Management, I turned the calls into a batch script. https://www.benday.com/2017/03/17/deploy-entity-framework-core-migrations-from-a-dll/

[pdesterlich]

what about an app published on linux, where ef.dll is nowhere to be found?

[benday]

Hi @pdesterlich --

It works almost exactly the same but the paths are slightly different. I've just updated my blog post (https://www.benday.com/2017/03/17/deploy-entity-framework-core-migrations-from-a-dll/) to include a bash shell script for doing the deployment plus a sample of how to call it.

Here's a direct link to the bash script: https://www.benday.com/wp-content/uploads/2017/03/deploy-ef-migrations.sh_.txt

[pdesterlich]

thank you @benday for the reply, but it seems to me i'm missing something.
what if i don't have a .nuget directory in my $HOME folder?

maybe a bit of information about my test setup can help you to help me

i have an asp.net core project, developed both on windows and linux; now i'm trying to deploy on a linux server; for my test i'm using a local virtual machine with ubuntu server 16.04 (i'm following more or less a tutorial found here: https://www.junian.net/2017/01/running-your-first-aspnet-core-web-app-on-ubuntu-16-04.html )
I installed dotnet core with .deb package (instructions here: https://www.microsoft.com/net/core#linuxubuntu), then i copied my published app (dotnet publish -c Release) in /var/www/app-folder/

When i run my app (with dotnet app-name.dll, or with service as in tutorial above) for some reason the first migration is automatically run, but any subsequent migration is ignored (the same happen in my dev machine if i try to run on a non existent database).

I have no .nuget dir anywhere on the server, and i have no ef.dll file also.

I could easily generate a migration script and manually apply it to the database, but i'd like to use migration on the deploy machine.

What am i missing?

[benday]

Hmm. If you can build your dotnet core project on that Ubuntu machine ("dotnet build"), then you definitely have a .nuget folder somewhere. I know that sometimes I get confused when I need to find something in a hidden folder in Linux -- any chance you aren't seeing your $HOME/.nuget directory because that folder is a hidden folder?

What happens if you run the following lines?:
cd $HOME
ls -la

Do you see a .nuget folder now?

What about if you run "cd $HOME/.nuget"?

[pdesterlich]

Well my ubuntu dev machine is a different machine from my ubuntu server.

I already tried "ls -la" on home dir in ubuntu server but as said there isn't a .nuget dir.

Keep in mind that everything i'm doing is on an ubuntu server wich is only a deploy server, no development is done on that server

[benday]

oh...that's tricky. :)

I think you're going to have to find a way to include ef.dll in your build output. It should be pretty easy -- just copy the file from $HOME/.nuget on your dev box and include it in your output files.

[SergiiKliebanov]

Hi guys, I've also faced this problem. When I execute script from https://www.benday.com/wp-content/uploads/2017/03/deploy-ef-migrations.bat_.txt I receive
dotnet exec needs a managed .dll or .exe extension
My input arg is dll like it was described in the article. When I run dotnet ef database update from project directory everything works fine. You can find csproj below.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup Label="Configuration" Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <OutputType>exe</OutputType>
    <NuGetPackageRoot> $(USERPROFILE)\.nuget\packages\</NuGetPackageRoot>
  </PropertyGroup>
  <PropertyGroup>
    <TargetFramework>netcoreapp1.1</TargetFramework>
    <GenerateAssemblyConfigurationAttribute>false</GenerateAssemblyConfigurationAttribute>
    <GenerateAssemblyCompanyAttribute>false</GenerateAssemblyCompanyAttribute>
    <GenerateAssemblyProductAttribute>false</GenerateAssemblyProductAttribute>
    <ApplicationIcon />
    <OutputTypeEx>exe</OutputTypeEx>
    <StartupObject />
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Abp.EntityFrameworkCore" Version="2.1.3" />
    <PackageReference Include="Abp.ZeroCore.EntityFrameworkCore" Version="2.1.3" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="1.1.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="1.1.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer.Design" Version="1.1.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="1.1.1">
      <PrivateAssets>All</PrivateAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.Extensions.Configuration" Version="1.1.2" />
  </ItemGroup>
  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.EntityFrameworkCore.Tools.DotNet" Version="1.0.0" />
  </ItemGroup>
  <ItemGroup>
    <Folder Include="Migrations\" />
  </ItemGroup>
</Project>

dotnet --info returns

.NET Command Line Tools (2.0.0-preview1-005977)
Product Information:
 Version:            2.0.0-preview1-005977
 Commit SHA-1 hash:  414cab8a0b
Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.0-preview1-005977\
Microsoft .NET Core Shared Framework Host
  Version  : 2.0.0-preview1-002111-00
  Build    : 1ff021936263d492539399688f46fd3827169983

[bricelam]

Version 1.0.0 of the Microsoft.EntityFrameworkCore.Tools.DotNet package isn't compatible with the .NET Core SDK 2.0. Add the following global.json to your solution.

{
  "sdk": {
    "version": "1.0.4"
  }
}

[Nogbit]

For .NET Core 2.0 the script by @benday (a huge help) needs modifying. The runtimeconfig arg was needed, a few other args were not. Of course the earlier suggestion by using -v during a working migration helps. Looking forward to a better solution when running EF commands outside of the csproj.

EfMigrationsNamespace=$1
EfMigrationsDllName=$1.dll
EfMigrationsDllDepsJson=$1.deps.json
DllDir=$PWD
EfMigrationsDllDepsJsonPath=$PWD/$EfMigrationsDllDepsJson
PathToRuntimeConfig=$PWD/$1.runtimeconfig.json
PathToNuGetPackages=$HOME/.nuget/packages
PathToEfDll=$PWD/ef.dll

dotnet exec --depsfile ./$EfMigrationsDllDepsJson --additionalprobingpath $PathToNuGetPackages --runtimeconfig $PathToRuntimeConfig $PathToEfDll database update --assembly ./$EfMigrationsDllName --startup-assembly ./$EfMigrationsDllName --project-dir . --verbose --root-namespace $EfMigrationsNamespace

[benday]

Hey @Nogbit --

I've been meaning to post an update to the script. I've got a batch script working but (for whatever reason) it seems like there's some inconsistency about where that ef.dll is located for some people's machines.

If you're trying to deploy from a TFS build or a TFS release, I've got a better solution that hides the existence of the ef.dll location and skips the deployment script. You can download/install this TFS extension and use the Deploy EF Core Migrations task directly in your build. https://marketplace.visualstudio.com/items?itemName=bendayconsulting.build-task

I do a demo of how to use that task in a DevOps video I did for Channel 9. The ef core task part of the demo starts at about 48 min and 45 secs. https://channel9.msdn.com/Events/dotnetConf/2017/T219

If TFS isn't an option, here's the updated batch script:

set EfMigrationsNamespace=%1
set EfMigrationsDllName=%1.dll
set EfMigrationsDllDepsJson=%1.deps.json
set DllDir=%cd%
set PathToNuGetPackages=%USERPROFILE%\.nuget\packages
set NuGetFallbackFolder="C:\Program Files\dotnet\sdk\NuGetFallbackFolder"
set PathToEfDll="C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.entityframeworkcore.tools.dotnet\2.0.0\tools\netcoreapp2.0\ef.dll"

dotnet exec --depsfile .\%EfMigrationsDllDepsJson% --additionalprobingpath %PathToNuGetPackages% --additionalprobingpath %NuGetFallbackFolder% %PathToEfDll% database update --assembly .\%EfMigrationsDllName% --startup-assembly .\%EfMigrationsDllName% --project-dir . --data-dir %DllDir% --verbose --root-namespace %EfMigrationsNamespace%

-Ben

[Nogbit]

I'm building and deploying on and to Linux. I have to manually copy that ef.dll from my nuget dir to my /bin/Release as part of a separate deploy bash script. After the deploy runs it then runs the script I posted above. Works great, even with zero nuget folders on the server. Thanks for your help on this, it was your blog post I found when googling.

[benday]

@Nogbit --

Managing the location of that ef.dll is about 75% of the problem, huh? :)

FYI, my TFS build task is cross-plat and works on Linux and on Windows build agents.

[Diaskhan]

Hie guys!

Is there some progress creating tool that could do migration after deploying assembly ???

[bricelam]

The tool is available. Getting it on the server and invoking it are the hard parts (see comments above).

[Diaskhan]

Maybe some One Provides A tutorial? I Have searched this solution about week !

[bricelam]

@benday has some good guidance and automation tips: https://www.benday.com/2017/03/17/deploy-entity-framework-core-migrations-from-a-dll/

[ajcvickers]

Closing in favor of https://github.com/aspnet/EntityFramework.Docs/issues/691