Runtime: JIT Compiler encountered an internal limitation. EF Core Migrations.
On an ASP.NET Core with EntityFramework Core I am creating an EF Migration:
dotnet ef migrations add InitialCommit
The migration includes seeding 280 objects in one of the entities using HasData.
I am able to seed 1 and 20 objects but I get an exception when seeding 280 objects.
Exception
Exception message: JIT Compiler encountered an internal limitation.
Stack trace:
System.InvalidProgramException: JIT Compiler encountered an internal limitation.
at Proj.Configurations.CompositionConfiguration.Configure(EntityTypeBuilder`1 builder)
at Microsoft.EntityFrameworkCore.ModelBuilder.ApplyConfiguration[TEntity](IEntityTypeConfiguration`1 configuration)
at Proj.Context.<>c.<OnModelCreating>b__209_0(ModelBuilder x) in Proj/Context.cs:line 80
at Proj.Context.OnModelCreating(ModelBuilder builder) in Proj/Context.cs:line 72
at Microsoft.EntityFrameworkCore.Infrastructure.ModelCustomizer.Customize(ModelBuilder modelBuilder, DbContext context)
at Microsoft.EntityFrameworkCore.Infrastructure.RelationalModelCustomizer.Customize(ModelBuilder modelBuilder, DbContext context)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.CreateModel(DbContext context, IConventionSetBuilder conventionSetBuilder, IModelValidator validator)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.<>c__DisplayClass5_0.<GetModel>b__1()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
at System.Lazy`1.CreateValue()
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.GetModel(DbContext context, IConventionSetBuilder conventionSetBuilder, IModelValidator validator)
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel()
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__7_1(IServiceProvider p)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScoped(ScopedCallSite scopedCallSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(IServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
at Microsoft.EntityFrameworkCore.DbContext.get_InternalServiceProvider()
at Microsoft.EntityFrameworkCore.DbContext.Microsoft.EntityFrameworkCore.Infrastructure.IInfrastructure<System.IServiceProvider>.get_Instance()
at Microsoft.EntityFrameworkCore.Internal.InternalAccessorExtensions.GetService[TService](IInfrastructure`1 accessor)
at Microsoft.EntityFrameworkCore.Infrastructure.AccessorExtensions.GetService[TService](IInfrastructure`1 accessor)
at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.CreateContext(Func`1 factory)
at Microsoft.EntityFrameworkCore.Design.Internal.DbContextOperations.CreateContext(String contextType)
at Microsoft.EntityFrameworkCore.Design.Internal.MigrationsOperations.AddMigration(String name, String outputDir, String contextType)
at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigrationImpl(String name, String outputDir, String contextType)
at Microsoft.EntityFrameworkCore.Design.OperationExecutor.AddMigration.<>c__DisplayClass0_1.<.ctor>b__0()
at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
JIT Compiler encountered an internal limitation.
Steps to reproduce
The entity Composition is as follows:
public class Composition {
public Int32 ProfuctId { get; set; }
public Decimal? P01 { get; set; }
public Decimal P02 { get; set; }
public Decimal? P03 { get; set; }
public Decimal P04 { get; set; }
public Decimal? P05 { get; set; }
public Decimal? P06 { get; set; }
public Decimal? P07 { get; set; }
public Decimal? P08 { get; set; }
public Decimal? P09 { get; set; }
public Decimal? P10 { get; set; }
public Decimal? P11 { get; set; }
public Decimal? P12 { get; set; }
public Decimal? P13 { get; set; }
public Decimal? P14 { get; set; }
public Decimal P15 { get; set; }
public Decimal? P16 { get; set; }
public Decimal? P17 { get; set; }
public Decimal? P18 { get; set; }
public Decimal? P19 { get; set; }
public Decimal? P20 { get; set; }
public Decimal P21 { get; set; }
public Decimal? P22 { get; set; }
public Decimal? P23 { get; set; }
public Decimal? P24 { get; set; }
public Decimal? P25 { get; set; }
public Decimal? P26 { get; set; }
public Decimal? P27 { get; set; }
public Decimal? P28 { get; set; }
public Decimal? P29 { get; set; }
public Decimal? P30 { get; set; }
public Decimal? P31 { get; set; }
public Decimal? P32 { get; set; }
public virtual Product { get; set; }
}
And this is the Composition configuration:
builder.ToTable("Compositions");
builder.HasKey(y => y.ProductId);
builder.Property(x => x.P01).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P02).IsRequired(true).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P03).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P04).IsRequired(true).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P05).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P06).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P07).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P08).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P09).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P10).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P11).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P12).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P13).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P14).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P15).IsRequired(true).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P16).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P17).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P18).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P19).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P20).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P21).IsRequired(true).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P22).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P23).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P24).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P25).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P26).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P27).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P28).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P29).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P30).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P31).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.Property(x => x.P32).IsRequired(false).HasColumnType(Sql.Decimal(8, 2));
builder.HasOne(y => y.Product).WithOne(y => y.Composition).HasForeignKey<Composition>(y => y.ProductId).IsRequired(true).OnDelete(DeleteBehavior.Cascade);
builder.HasData(
new {
ProductId = 1,
P01 = (Decimal?) 110.0,
P02 = (Decimal) 60030.0,
P03 = (Decimal?) 0.0,
P04 = (Decimal) 336.0,
P05 = (Decimal?) 10600.0,
P06 = (Decimal?) 0.0,
P07 = (Decimal?) 0.633,
P08 = (Decimal?) 8.27,
P09 = (Decimal?) 184.0,
P10 = (Decimal?) 106.0,
P11 = (Decimal?) 2.075,
P12 = (Decimal?) 424.0,
P13 = (Decimal?) 542.0,
P14 = (Decimal?) 1112.0,
P15 = (Decimal) 23520.0,
P16 = (Decimal?) 0.226,
P17 = (Decimal?) 331.0,
P18 = (Decimal?) 16.0,
P19 = (Decimal?) 6900.0,
P20 = (Decimal?) 0.853,
P21 = (Decimal) 1260.0,
P22 = (Decimal?) 0.633,
P23 = (Decimal?) 0.0,
P24 = (Decimal?) 0.003,
P25 = (Decimal?) 0.0,
P26 = (Decimal?) 0.357,
P27 = (Decimal?) 1.5,
P28 = (Decimal?) 0.0,
P29 = (Decimal?) 0.39,
P30 = (Decimal?) 0.005,
P31 = (Decimal?) 11950.0,
P32 = (Decimal?) 3.37
});
In the code example I just posted HasData is seeding only one Composition ...
In reality I am seeding 280 compositions and I also tested seeding only 20 ...
Further technical details
EF Core version: 2.1.3
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: macOS Mojave
IDE: Visual Studio Code 1.29.1
NET Core SDK: 2.1.403
mdmoura
All 24 comments
Any chance of a self contained repro? I've been trying to put together something using the bits and pieces of code you provided but so far I haven't been able to reproduce the error.
In reality I am seeding 280 compositions and I also tested seeding only 20 …
Does that mean that you have a HasData call with 280 parameters? Could you try building the params array manually (allocate an array, initialize its elements one by one, pass it to HasData) or is this some kind of autogenerated code?
mikedn
on 2 Dec 2018
Does that mean that you have a HasData call with 280 parameters?
No, HasData has 280 instances each one with 32 parameters ...
@mikedn I just created a repository with it: https://github.com/mdmoura/JITCompilerEFCore
You just need to run the command:
dotnet ef migrations add InitialCommit
With that command, and with the code as it is, you will get the error:
JIT Compiler encountered an internal limitation.
Note: I am inserting 262 records to get the error ... But you can get the error with much less.
If you comment all lines, in HasData, leaving only one then the command dotnet ef migrations add InitialCommit will create the migration without any error ...
If you then want to create the database for that migration just run:
dotnet ef database update
The SQL Database configuration is in ContextFactory. I am on a Mac so using a Docker container.
But the error happens when adding the migration. Any idea what might be wrong?
mdmoura
on 3 Dec 2018
Thank you for the sample!
I can reproduce the error using netcoreapp2.1 but I can't seem to reproduce it when using netcoreapp3.0.
@AndyAyersMS Was there any improvement done for cases where a large number of temporary variables are created due to newobj? AFAIR there was an issue like this and you made some improvements in the importer? Method Context.OnModelCreating generates tens of thousands of lclvars...
mikedn
on 3 Dec 2018
Hmm, this might have been fixed by @briansull in dotnet/coreclr#17793
mikedn
on 3 Dec 2018
@mikedn Is the fix included in Net Core 2.2 or only in Net Core 3.0?
mdmoura
on 3 Dec 2018
@mdmoura As far as I can tell no, the fix is not in 2.2.
mikedn
on 4 Dec 2018
I also checked and dotnet/coreclr#17793 is NOT in .Core 2.2 sources
briansull
on 4 Dec 2018
@briansull Is there any way to solve this problem in the meanwhile?
mdmoura
on 4 Dec 2018
Do you really need to use nullable decimal in the initialization data (apart from those couple of values that are really null)? I would guess that even if the column is nullable EF should be able to convert from decimal to nullable decimal when inserting the data. Or perhaps use double instead of decimal, assuming that you don't end up with precision issues and that EF can convert from double to decimal.
Other options:
- split the data into multiple methods (each returns an array containing a part of the data, concatenate all arrays and pass the result to
HasData) - deserialize the array from some kind of file (JSON, XML, CSV etc.) instead of hardcoding it
I'd do the later, it seems preferable to keep this kind of data in a CSV file rather than in code. Even VS seems to having some problems dealing with this amount of code, it's rather sluggish when I open the file. And I wouldn't be surprised to discover that it is faster to read the file rather than to JIT-compile such code. The JIT has to generate around one megabyte of code!
To try to explain a bit the tehnical issue: this code, while perhaps it doesn't look like much - a bunch of numbers, generates a very large number of local variables in the JIT. Both decimal and Nullable<T> are structs and they need memory locations, unlike primitive types that can be hold in registers. So the JIT keeps creating variables - one for a decimal, another one for a Nullable<decimal>, another one for the next decimal and so on, until it hits an internal limit.
mikedn
on 4 Dec 2018
Do you really need to use nullable decimal in the initialization data (apart from those couple of values that are really null)? I would guess that even if the column is nullable EF should be able to convert from decimal to nullable decimal when inserting the data.
I am casting to Decimal? because I got an error when not using it ...
Deserialize the array from some kind of file (JSON, XML, CSV etc.) instead of hardcoding it
Yes, I am going for this one ...
I was hardcoding because that would avoid to have files embedded on the class library.
mdmoura
on 4 Dec 2018
Yes, I am going for this one …
I wonder if that actually helps. I'm not exactly familiar with EF but I was looking around and I see that the migrations command generated some .cs files in a directory. And those .cs files also contain similar code. I wonder if it won't do the same thing even if the data is loaded from a file.
mikedn
on 4 Dec 2018
If this is an ingrained pattern produced by EF tooling, we can certainly argue that Brian's fix should be ported to 2.2. So please keep us updated.
I have some prototype changes lying around where the jit will reuse temps in cases like this and so avoid hitting these limitations and also emit much cleaner and smaller code, but don't recall offhand how close they were to being usable.
AndyAyersMS
on 4 Dec 2018
the jit will reuse temps
Now I'm curious, how do you know when it's safe to reuse such a temp?
mikedn
on 4 Dec 2018
However this is Fixed in .Net 4.8
VSO 604943
briansull
on 4 Dec 2018
Idea was to do something similar to what we do for the box temp: https://github.com/AndyAyersMS/coreclr/tree/OpportunisticReuseNewObjTemp
AndyAyersMS
on 4 Dec 2018
https://github.com/AndyAyersMS/coreclr/tree/OpportunisticReuseNewObjTemp
Hmm, I don't think that handles the (hopefully very rare) corner case of a struct constructor that escapes this somehow, does it? Come to think of it, that would be extremely bizarre, especially in the context of newobj.
mikedn
on 5 Dec 2018
@AndyAyersMS @briansull This is a pretty common pattern in EF. Can we bring this fix to shiproom for a 2.2.x patch?
/cc @divega @Eilon @DamianEdwards
ajcvickers
on 11 Dec 2018
Seems reasonable -- @briansull do you want to set this up, it since it's your change?
cc @RussKeldorph
AndyAyersMS
on 11 Dec 2018
It would be great to have a patch, not in a far future, to solve this problem.
mdmoura
on 11 Dec 2018
Fix is in PR dotnet/coreclr#17793 and PR dotnet/coreclr#19065
briansull
on 11 Dec 2018
Should have been closed by dotnet/coreclr#21493?
RussKeldorph
on 16 Jan 2019
Any ETA for this fix? It seems it wasn't released with Asp.Net Core 2.2.1 patch.
mdmoura
on 22 Jan 2019
It is only shipping in 2.2.2
@vivmishra any ETA for 2.2.2 ?
briansull
on 22 Jan 2019
@briansull - should be in early-to-mid Feb, so very soon.
Eilon
on 27 Jan 2019
Related issues
sahithreddyk
·
3Comments
GitAntoinee
·
3Comments
iCodeWebApps
·
3Comments
chunseoklee
·
3Comments
omariom
·
3Comments