Efcore: Add option to reverse engineer to string-based model builder calls

[dapowers87]

Large numbers of lambda expressions can cause issues with stack use. Adding an option to use the string-based API instead would help with this, and potentially also be faster in model building.

---

Original issue

I am running dotnet version 3.1.301 on Mac OS Catalina. There are a couple hundred indices on one DbSet that was created when scaffolded. I found more than one HasIndex that caused the issue. Here is one:

entity.HasIndex(e => new { e.Partially, e.InvAmount, e.PaidAmt, e.SalesRep, e.Po, e.Estdate, e.UserId, e.DelivMeth, e.ShipCode, e.Reference, e.DocAlias, e.Custchar1, e.Custchar2, e.Custchar3, e.Custchar4, e.Custdate1, e.Custdate2, e.Custdate3, e.Custdate4, e.Custlog1, e.Custlog2, e.Custlog3, e.Custlog4, e.Custnum1, e.Custnum2, e.Custnum3, e.Custnum4, e.Weight, e.Currcode, e.QuoteNo, e.OrderDate, e.Terms, e.IsIncludedInReport, e.HandOff, e.RecurringDocId, e.ShipAmt, e.IsHistorical, e.Custchar5, e.Custchar6, e.Custchar7, e.Custchar8, e.IsPos, e.TrDiscFex, e.InvoicesId, e.BillCode, e.CustCode, e.StageId, e.Status, e.Shipped, e.IsManual, e.DocNo, e.DocNoSort })
                    .HasName("_dta_index_INVOICES_7_542624976__K1_K33_K8_K89_K2_K13_K82_K3_K161_5_7_11_14_19_20_26_28_29_30_31_34_40_51_52_53_54_55_56_57_58_");

The issue does not show itself when running the same call in Windows 10 or Docker's dotnet/core/aspnet:3.1-alpine.

Removing all HasIndex's is a workaround for now.

[ajcvickers]

@dapowers87 At what point in your application does the stack overflow occur?

[dapowers87]

During the ToList, ToListAsync, FirstOrDefault, etc... (or anything like it) call.

[ajcvickers]

@dapowers87 Is it possible to attach code that reproduces the issue?

[dapowers87]

It would simply be something like:

context.Xinvoic.ToList()

It appears that the index that I originally posted is the reason behind the SO as it went away when I removed it.

[smitpatel]

@dapowers87 - Can you share the entity class on which that index is defined so I can create a repro code?

[dapowers87]

Here is the Invoice class. One more bit of information, I found the stack overflow occurs on any DbSet in the context, as they all call that OnModelCreating function it seems.

public partial class Invoices
    {
        public int InvoicesId { get; set; }
        public byte Status { get; set; }
        public string DocNo { get; set; }
        public string OrderNo { get; set; }
        public string QuoteNo { get; set; }
        public DateTime? DateFld { get; set; }
        public DateTime? OrderDate { get; set; }
        public string CustCode { get; set; }
        public string Dep { get; set; }
        public int? ContCode { get; set; }
        public string Terms { get; set; }
        public string RefNo { get; set; }
        public string Shipped { get; set; }
        public string Partially { get; set; }
        public string CRegister { get; set; }
        public decimal? Taxable { get; set; }
        public decimal? TaxAmount { get; set; }
        public decimal? Exempt { get; set; }
        public decimal? InvAmount { get; set; }
        public decimal? PaidAmt { get; set; }
        public decimal? PaidSofar { get; set; }
        public decimal? TrDisc { get; set; }
        public DateTime? Shipdate { get; set; }
        public string Paid { get; set; }
        public string Oncepaid { get; set; }
        public string SalesRep { get; set; }
        public DateTime? InvDueOn { get; set; }
        public string Po { get; set; }
        public DateTime? Estdate { get; set; }
        public string UserId { get; set; }
        public string DelivMeth { get; set; }
        public string Memos { get; set; }
        public int? BillCode { get; set; }
        public int? ShipCode { get; set; }
        public decimal? Cog { get; set; }
        public decimal? MiscCog { get; set; }
        public decimal? CCog { get; set; }
        public decimal? CMiscCog { get; set; }
        public string RegNo { get; set; }
        public string Reference { get; set; }
        public string Fob { get; set; }
        public DateTime? PoDate { get; set; }
        public string Tracking { get; set; }
        public string RevAcnt { get; set; }
        public string RetAcnt { get; set; }
        public string ExpAcnt { get; set; }
        public string Processed { get; set; }
        public string Juriscode { get; set; }
        public string WebOrder { get; set; }
        public byte? Copies { get; set; }
        public string DocAlias { get; set; }
        public string Custchar1 { get; set; }
        public string Custchar2 { get; set; }
        public string Custchar3 { get; set; }
        public string Custchar4 { get; set; }
        public DateTime? Custdate1 { get; set; }
        public DateTime? Custdate2 { get; set; }
        public DateTime? Custdate3 { get; set; }
        public DateTime? Custdate4 { get; set; }
        public string Custlog1 { get; set; }
        public string Custlog2 { get; set; }
        public string Custlog3 { get; set; }
        public string Custlog4 { get; set; }
        public decimal? Custnum1 { get; set; }
        public decimal? Custnum2 { get; set; }
        public decimal? Custnum3 { get; set; }
        public decimal? Custnum4 { get; set; }
        public decimal? Weight { get; set; }
        public string SsFrInfo { get; set; }
        public decimal? SsFrAmt { get; set; }
        public string WebSite { get; set; }
        public string Currcode { get; set; }
        public decimal? TaxableFex { get; set; }
        public decimal? TaxAmtFex { get; set; }
        public decimal? ExemptFex { get; set; }
        public decimal? InvAmtFex { get; set; }
        public decimal? Tranrate { get; set; }
        public decimal? PdAmtFex { get; set; }
        public decimal? PdSfarFex { get; set; }
        public string DiscAcnt { get; set; }
        public string Posted { get; set; }
        public string IsManual { get; set; }
        public string PolicyMemo { get; set; }
        public DateTime? ExpiryDate { get; set; }
        public decimal? TriangulationRate { get; set; }
        public string IsPos { get; set; }
        public decimal? TrDiscFex { get; set; }
        public byte? FreightDistMthd { get; set; }
        public int? StageId { get; set; }
        public decimal? Probability { get; set; }
        public string IsIncludedInReport { get; set; }
        public DateTime? OpeningDate { get; set; }
        public DateTime? ProbableCloseDate { get; set; }
        public byte? Priority { get; set; }
        public string IsOpportunity { get; set; }
        public int? Source { get; set; }
        public int? Grade { get; set; }
        public int? ReasonClosed { get; set; }
        public string IsOpportunityClosed { get; set; }
        public DateTime? OpActualCloseDate { get; set; }
        public int? LeaseId { get; set; }
        public int? RentId { get; set; }
        public string IsTaxInLeaseAmt { get; set; }
        public string IsTaxInRentAmt { get; set; }
        public DateTime? OpSourceDate { get; set; }
        public DateTime? OpGradeDate { get; set; }
        public DateTime? OpStageDate { get; set; }
        public byte? HandOff { get; set; }
        public string AllowBckOrd { get; set; }
        public int? RecurringDocId { get; set; }
        public string UseBatchDate { get; set; }
        public int? RecDocTaskId { get; set; }
        public byte? Picked { get; set; }
        public string UYahooOrder { get; set; }
        public string IsDistSale { get; set; }
        public decimal? EcTranRate { get; set; }
        public string ShipmentTrackingNote { get; set; }
        public byte? EverestModeCreatedin { get; set; }
        public string PromotionCode { get; set; }
        public string EfeInsured { get; set; }
        public string EfeZip { get; set; }
        public string EfeCountry { get; set; }
        public string EfeState { get; set; }
        public string EfeCity { get; set; }
        public byte? TaxCalcType { get; set; }
        public decimal? ShipAmt { get; set; }
        public decimal? ShipAmtFex { get; set; }
        public string FreightPaddingApplied { get; set; }
        public string DueDateOverride { get; set; }
        public byte? FreightPadding { get; set; }
        public decimal? FreightPaddingAmtPercent { get; set; }
        public int? ReasonClosedGroup { get; set; }
        public int? CustomerActionGroup { get; set; }
        public int? CustomerActionChoice { get; set; }
        public string IsHistorical { get; set; }
        public string Custmemo1 { get; set; }
        public string Custmemo2 { get; set; }
        public string Custmemo3 { get; set; }
        public string Custmemo4 { get; set; }
        public int? PayflowproInvnum { get; set; }
        public string PgSaleAcnt { get; set; }
        public string PgSaleRetAcnt { get; set; }
        public string PgLnDiscAcnt { get; set; }
        public string PgPymtDiscAcnt { get; set; }
        public string PgAcntRecv { get; set; }
        public string PgCustAdvAcnt { get; set; }
        public int? PgOverrideFlag { get; set; }
        public DateTime? DwUpdateDate { get; set; }
        public string DwUpdated { get; set; }
        public DateTime? DwDimensionUpdateDate { get; set; }
        public int SyncStatus { get; set; }
        public string OeOrderNo { get; set; }
        public string Custchar5 { get; set; }
        public string Custchar6 { get; set; }
        public string Custchar7 { get; set; }
        public string Custchar8 { get; set; }
        public string IsHeld { get; set; }
        public string IsPickLocPrinted { get; set; }
        public DateTime CreatedAt { get; set; }
        public string DocNoSort { get; set; }
    }

[smitpatel]

@dapowers87 - Stackoverflow must be happening during HasIndex call on OnModelCreating.
Accessing any DbSet in query will cause model to be built and which will end up in OnModelCreating codepath.

[dapowers87]

Right - I've come to that conclusion as well. Is there any idea why it is happening, though?

[smitpatel]

Likely this https://github.com/dotnet/efcore/issues/14702#issuecomment-465787678

[ErikEJ]

I think you would have been "saved" by this fix https://github.com/dotnet/efcore/issues/7665 (which was in 2.0, but not in 2.1 to 3.1 - now back in 5.0)

[AndriySvyryd]

Creating a lambda takes space on the stack. Replacing the calls with the HasIndex(params string[]) overload should also mitigate this.

[ajcvickers]

@ErikEJ Is there any way to tell for sure this was a hypothetical index?

[ErikEJ]

No - but it is likely, given the index name.

[ajcvickers]

@AndriySvyryd @bricelam I can't find an existing issue about reverse engineering using string args rather than lambdas. Should we consider it part of #4038, or should it have its own issue?

[dapowers87]

Creating a lambda takes space on the stack. Replacing the calls with the HasIndex(params string[]) overload should also mitigate this.

Is there anyway to accomplish using scaffolding? This is how all of this indexes were created in the first place.

[AndriySvyryd]

@ajcvickers Seems we only have one for the snapshot: https://github.com/dotnet/efcore/issues/18620. We could repurpose this one.

[ajcvickers]

@dapowers87 Not currently--putting this issue on the backlog to consider doing it in the future. However, it's likely that you are hitting #20705, which is fixed in the EF Core 5.0 preview.

[bricelam]

@dapowers87 Check out my EFCore.TextTemplating sample for a good starting point for customizing the generated code.

The EF Core Power Tools also let you use Handlebars templates to customize the code