Efcore: Query: inconsistent behavior around anonymous objects comparison

[maumar]

from g in context.Gears
from o in context.Gears.OfType<Officer>()
where new { Name = g.LeaderNickname, Squad = g.LeaderSquadId, Five = 5 } == new { Name = o.Nickname, Squad = o.SquadId, Five = 5 }
select g.Nickname;

produces the following SQL:

SELECT [g].[Nickname]
FROM [Gear] AS [g]
CROSS JOIN [Gear] AS [o]
WHERE [g].[Discriminator] IN (N'Officer', N'Gear') AND ((([g].[LeaderNickname] = [o].[Nickname]) AND ([g].[LeaderSquadId] = [o].[SquadId])) AND (5 = 5))

and yields some results, however uncorrelated:

from g in context.Gears
where new { Five = 5 } == new { Five = 5 }
select g.Nickname

produces:

SELECT [g].[Nickname]
FROM [Gear] AS [g]
WHERE 0 = 1

Which always returns false.
This is due to Relinq optimization, generalted QM already has the anonymous type comparison replaced with "false". Linq to objects always returns false when comparing two anonymous objects, however it does seem to be more useful to perform the SQL translation.

If we want to be consistent with L2O then we could collapse anonymous types comparison to false, unless they are part of Join operation - we need to perform proper comparison for joins, e.g. for the composite key join.

[smitpatel]

new { ... } == new { ... } is always false in L2o world. They are reference equality between 2 different instances (since anonymous types don't implement equals method). Therefore Re-Linq optimizes it. In first query, since the properties are correlated with outer query sources, it was not optimized which eventually got translated to server producing SQL. That is inconsistent. It is possible that if for some reason client eval is forced then we would generate different result.

Comparison of anonymous types behaves differently with equals operator for joins. It does property-based comparison for those cases. It is required because you can reference only 1 query source on each side of equals. But for queries like above, it is where predicate. There is no need to introduce anonymous type if the purpose is to do just property comparison since it is easy to write predicate with AND condition. We should detect such cases and either block or in the least case warn just to be more consistent. (or deterministic)

[divega]

@maumar @smitpatel @anpete the compiler actually generates an Equals() override for each anonymous type to perform property by property comparisons. The == operator is not overridden, hence it just performs reference comparisons as @smitpatel mentioned.

So, I guess the main question in this issue is whether we consider the "normal" semantics of the == operator undesirable in queries and therefore we want to compensate. I remember vaguely we have already discussed drifting from the "normal" behavior for byte[], which would be a precedent.

The second (kind of OT) question for me is whether there are any cases in our in-memory implementations in which currently we could be leveraging Equals() but we aren't. AFAIK, all standard LINQ to Objects operators that need to perform equality under the hood, e.g. Join(), GroupJoin(), GroupBy() rely on the default EqualityComparer<T> which will use the Equals() method.

[divega]

Clearing up milestone so that we can decide now what we want to do (we believe changing the behavior is a breaking change that we can make in 2.0)

[maumar]

additional info from the latest investigation:

var q1 = ctx.Factions.Where(f => new { Five = 5 } == new { Five = 5 }).ToList(); // returns nothing

var q2 = ctx.Factions.Where(f => new { Five = 5 }.Equals(new { Five = 5 })).ToList(); // returns all (Equals method is funcletized)

var q3 = ctx.Factions.Where(f => Equals(new { Five = 5 }, new { Five = 5 })).ToList(); // same as q3 

var q4 = ctx.Factions.Where(f => new { Five = f.Name } == new { Five = f.Name }).ToList(); // returns all, translates predicate to SQL query

var q5 = ctx.Factions.Where(f => new { Five = f.Name }.Equals(new { Five = f.Name })).ToList(); // throws

var q6 = ctx.Factions.Where(f => Equals(new { Five = f.Name }, new { Five = f.Name })).ToList(); // same as q5

We should fix the case 5, currently we throw the following:

No mapping to a relational type can be found for the CLR type 'Expression[]'.
at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSourceExtensions.GetMapping(IRelationalTypeMappingSource typeMappingSource, Type clrType) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Storage\RelationalTypeMappingSourceExtensions.cs:line 80
at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSourceExtensions.GetMappingForValue(IRelationalTypeMappingSource typeMappingSource, Object value) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Storage\RelationalTypeMappingSourceExtensions.cs:line 27
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.GenerateSqlLiteral(Object value) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 828
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.VisitConstant(ConstantExpression constantExpression) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 1669
at System.Linq.Expressions.ConstantExpression.Accept(ExpressionVisitor visitor)
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.VisitBinary(BinaryExpression binaryExpression) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 1313
at Microsoft.EntityFrameworkCore.SqlServer.Query.Sql.Internal.SqlServerQuerySqlGenerator.VisitBinary(BinaryExpression binaryExpression) in D:\git\EntityFrameworkCore\src\EFCore.SqlServer\Query\Sql\Internal\SqlServerQuerySqlGenerator.cs:line 59
at System.Linq.Expressions.BinaryExpression.Accept(ExpressionVisitor visitor)
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.GeneratePredicate(Expression predicate) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 549
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.VisitSelect(SelectExpression selectExpression) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 264
at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.Accept(ExpressionVisitor visitor) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Expressions\SelectExpression.cs:line 1191
at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.GenerateSql(IReadOnlyDictionary2 parameterValues) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Sql\DefaultQuerySqlGenerator.cs:line 139 at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.GetRelationalCommand(IReadOnlyDictionary2 parameters) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Internal\ShaperCommandContext.cs:line 129
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer) in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Internal\QueryingEnumerable.cs:line 122 at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func3 operation, Func3 verifySucceeded) in D:\git\EntityFrameworkCore\src\EFCore.SqlServer\Storage\Internal\SqlServerExecutionStrategy.cs:line 47 at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable1.Enumerator.MoveNext() in D:\git\EntityFrameworkCore\src\EFCore.Relational\Query\Internal\QueryingEnumerable.cs:line 83
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.<_TrackEntities>d__172.MoveNext() in D:\git\EntityFrameworkCore\src\EFCore\Query\Internal\LinqOperatorProvider.cs:line 185 at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor1.EnumeratorExceptionInterceptor.MoveNext() in D:\git\EntityFrameworkCore\src\EFCore\Query\Internal\LinqOperatorProvider.cs:line 143

[maumar]

Problem is that for q4 (== operator), SqlTranslatingExpressionVisitor performs UnfoldStructuralComparison on binary expressions, which is supposed to convert Expression[] comparison into comparing their constituents.

In case of Equals method we go thru MethodCallTranslator, which uses EqualsTranslator, which resolves to a Equal BinaryExpression, but UnfoldStructuralComparison is not performed on the result, hence the error.

[ajcvickers]

Notes from triage:

• Fix the exception case
• Decide on a consistent behavior

[smitpatel]

@maumar Something to investigate again in new pipeline.

[NetMage]

Yes, this appears to be a problem again in EF Core 3.1 where it works with == on simple value types but fails when encapsulated in anonymous types (and fails when using Equals as well), at least in the context of Where(... Any(... new { } == new { })). Even worse, replacing Where(a => Any(b => a.fld1 == b.fld1)) with Where(a => Any(b => new { a.fld1 } == new { b.fld1 })) gives a translation error.

[ajcvickers]

Discuss with @smitpatel

[smitpatel]

Where(a => Any(b => new { a.fld1 } == new { b.fld1 })) gives a translation error.

It was client eval before 3.0. We haven't regressed anything here.

[ajcvickers]

We may consider translating some more anonymous object queries depending on how they are generated--we need to follow up with OData on this.

@NetMage Did you write this manually, or was the LINQ query generated?