Efcore: This overload is currently not supported.
I try to create an extension expression that has no impact on the generated SQL result.
I only want to react in my custom QueryCompiler method public override TResult Execute<TResult>(Expression query).
But compiling the query failed. How do I "register" my extension or how to tell to ignore it at the compilation or do I have to rewrite the query befor compilation and remove my custom part?
Exception message:
Could not parse expression 'value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[Model.AzureSQL.Core.Entities.Inverter]).*****.Where(d => (d.ID == __expectedResult_ID_0)).Cacheable(00:00:05)': This overload of the method 'EntityFrameworkQueryableExtensions.Cacheable' is currently not supported.
Steps to reproduce
I created a custom Expression close to the AsNotTracking expression.
```c#
public class CacheableExpressionNode : ResultOperatorExpressionNodeBase
{
[...]
public class CacheableResultOperator : SequenceTypePreservingResultOperatorBase, IQueryAnnotation
{
[...]
public static class EntityFrameworkQueryableExtensions
{
internal static readonly MethodInfo CacheablehMethodInfo
= typeof(EntityFrameworkQueryableExtensions)
.GetTypeInfo().GetDeclaredMethod(nameof(Cacheable));
public static IQueryable<T> Cacheable<T>(this IQueryable<T> source, [NotParameterized] TimeSpan timeToLive)
{
Check.NotNull(source, nameof(source));
Check.NotNull(timeToLive, nameof(timeToLive));
return
source.Provider is EntityQueryProvider
? source.Provider.CreateQuery<T>(
Expression.Call(
instance: null,
method: CacheablehMethodInfo.MakeGenericMethod(typeof(T)),
arg0: source.Expression,
arg1: Expression.Constant(timeToLive)))
: source;
}
}
```
Further technical details
Further technical details
EF Core version: 2.2.0
Database Provider: any
SteffenMangold
All 27 comments
If you decide to add custom node in expression tree which you want to be available at compile time then
- Add custom method with ResultOperator like above.
- Add your custom method in the list here https://github.com/aspnet/EntityFrameworkCore/blob/master/src/EFCore/Query/Internal/MethodInfoBasedNodeTypeRegistryFactory.cs so that Relinq can parse it and add it to QueryModel
- Override ResultOperatorHandler if you want to handle it as result operator. If not, then override some other part which would strip off the result operator (like how AsTracking/AsNoTracking) is deal with.
smitpatel
on 23 Dec 2018
@smitpatel thanks!
By "add" you mean override and ReplaceService on DbContextOptionsBuilder?
SteffenMangold
on 23 Dec 2018
Override the service and add additional line like this
https://github.com/aspnet/EntityFrameworkCore/blob/359424adda850025d55e912684df4fa065e23948/src/EFCore/Query/Internal/MethodInfoBasedNodeTypeRegistryFactory.cs#L40-L41
smitpatel
on 23 Dec 2018
Just finished it before a few seconds! It works fantastic now. Thank you very much for your help so long.
SteffenMangold
on 23 Dec 2018
I also made a Git project out if it. If someone wants to see complete source.
SteffenMangold
on 24 Dec 2018
Duplication of effort? https://github.com/VahidN/EFSecondLevelCache.Core/
ErikEJ
on 24 Dec 2018
Duplication of effort? https://github.com/VahidN/EFSecondLevelCache.Core/
No. Same usage but way faster and based on more EF Core like implementation.
EFSecondLevelCache.Core uses a ToSql implementation to identify queries, that is extremely slow if you hit the cache on a heavy workload. I do not use a reflective monster to identify the query, I intercept the query compiler.
SteffenMangold
on 24 Dec 2018
Good work, Steffen!
ErikEJ
on 24 Dec 2018
Good work, Steffen!
I had to work a little bit more on it to close all TODOs, but after I would love to see it under https://docs.microsoft.com/de-de/ef/core/extensions/
I think it can be really helpful for other developers.
SteffenMangold
on 24 Dec 2018
I plan to update the extension list soon!
ErikEJ
on 24 Dec 2018
@SteffenMangold, just checked your implementation. It is small but possible chance to hit wrong cache item. Maybe it is better to make key as an object which contain whole expression tree and parameters with cutom equality members. I assume that this can be paranoid check, but who knows ;)
sdanyliv
on 24 Dec 2018
@sdanyliv I think the possibility is as high as that MSSQL server is selecting a false query plan. They also using 64bit Hash values. Also if it really happens, I think the result type would not match and throw an exception. But you are right there is indeed a really tiny chance. I'm not sure if it needs to be handled.
SteffenMangold
on 24 Dec 2018
@SteffenMangold - I looked at CustomQueryCompiler of yours briefly. I would suggest not tying it with Relinq and adding result operators. Instead you can write a custom ExpressionVisitor, which traverse the query expression and strip out Cacheable (or the custom method you are providing), and give you back result of caching or not. That way, you don't need to parse the query expression when its cache hit which could help perf
smitpatel
on 25 Dec 2018
@smitpatel thanks for the hint, I edited to logic! In a future EF version, I may inherit from ParameterExtractingExpressionVisitor. The internal logic changes a lot in EF 3.0.
Is there a possibility of preventing that a query can contain two calls of my expression?
SteffenMangold
on 25 Dec 2018
Just strip out that method call. In your method call, first arg would be inner queryable & 2nd arg is time to live. So when you encounter that particular method, to strip out your method just call return Visit(node.Arguments[0]). That will remove your method from tree.
I may inherit from ParameterExtractingExpressionVisitor
It may not be good idea since different providers interact with it so it can mess up a lot of things.
Is there a possibility of preventing that a query can contain two calls of my expression?
Sadly, expression tree does not allow such checks. So only way to deal with it, (since you are using time to live parameter) is "Last one wins". In one way, that is what ends up happening when you call OrderBy multiple times.
smitpatel
on 26 Dec 2018
Thank you! Removing query part works perfect.
So for future versions, would it be better to inherit from MethodCallExpressionNodeBase or even just build an extension method like this one? If I understand you correctly there will be no more ResultOperatorExpressionNodeBase on next major release.
SteffenMangold
on 26 Dec 2018
Just build extension methods on Queryable. And they will be in expression tree as MethodCallExpression. At that point they are same as having Queryable.Where, so there shouldn't be any need to inherit from anything (unless I am misunderstanding something).
smitpatel
on 26 Dec 2018
Perfect! Thanks for all you help, it was perfect for a deeper understanding.
SteffenMangold
on 26 Dec 2018
@smitpatel Sandly I detected a major bug that breaks the complete function.
In this line the query is only compiled but not executed. So if I put it into the cache and later returning it, it is calling the database again.
So I think I have to detect TResult is an IEnumerable or a direct type of TResult and invoke it. But I couldn't find similar code in EF Core. Do you have any idea?
SteffenMangold
on 28 Dec 2018
This weird code takes care of that. Currently EF Core's query compiler has unwritten contract that result type is always going to IEnumerable so in front end we have such code to call first to get TResult rather than IEnumerable. (Don't worry, we are changing it in 3.0)
https://github.com/aspnet/EntityFrameworkCore/blob/ec3f6f714de2206e0900ea8b482a174a64910f1f/src/EFCore/Query/Internal/QueryCompiler.cs#L118-L163
smitpatel
on 28 Dec 2018
But I call CompileQueryCore method that already is calling this ComipleQueryCore method.
Should it not be executed after that and the result should be the "after-db-call" Enumerable?
SteffenMangold
on 28 Dec 2018
@smitpatel what every I do (even with crazy reflection code) I can't get the result out of EnumeratorExceptionInterceptor[MyTyp]. This underlying type makes it impossible. :(
Update:
Thats the only way I found and really ugly:
```c#
private TResult GetCompiledQueryResult
{
var queryModel = queryModelGenerator.ParseQuery(query);
var resultItemType = (queryModel.GetOutputDataInfo() as StreamedSequenceInfo)?.ResultItemType ?? typeof(TResult);
if (resultItemType == typeof(TResult))
{
return queryResult;
}
var genericToListMethod = ToListMethod.MakeGenericMethod(new Type[] { resultItemType });
var result = genericToListMethod.Invoke(queryResult, new object[] { queryResult });
return (TResult)result;
}
```
But I think there is no other way right now.
SteffenMangold
on 28 Dec 2018
https://github.com/SteffenMangold/EntityFrameworkCore.Cacheable/blob/b726648b72fbd33e0685fe9f9c1922832c92ff54/EntityFrameworkCore.Cacheable/CustomQueryCompiler.cs#L91
That lines looks spooky. You should not be calling CreateCompiledQuery method. If is method which works with EF.CompileQuery and not for normal query compilation phase. The correct method to use is CompileQueryCore directly. (Notice the difference in Extract parameter args)
I think that would fix the issue of query not being executed. Let me know if you still face any issue.
smitpatel
on 28 Dec 2018
@smitpatel I did not use this method because it is in private. I Just copied it, but it produces the same "before DB" result. It still returned an ExceptionInterceptor instance.
SteffenMangold
on 28 Dec 2018
OK I think I found the reason...
This part is not supposed to return the actual result. It returns an ExceptionInterceptor that is calling the DB only if you call .ToList(... for example on the underlying enumerator.
So I think I was right with workaround this to get the actual result.
For notice: for single result queries it work because of a .First(... call already happens.
I'm not quite sure if the way EF core works here right now is intended. Maybe this is something to work on. I would expect that public virtual TResult Execute<TResult>(Expression query) alway returns the DB query result. No matter if single or enumerable result type.
SteffenMangold
on 28 Dec 2018
So I ended up with this solution, which works fine for now.
SteffenMangold
on 28 Dec 2018
@SteffenMangold - I think I understood now what is happening. The issue is EF Core returns result type or IEnumerable. The result type would always execute query and gives back the single result. (the First call) but the IEnumerable will execute query only when it is enumerated. In other word, in order to store results of IEnumerable in cache, you would actually need to enumerate it. So calling ToList method on it would be one way to do it.
smitpatel
on 28 Dec 2018
Related issues
RehanSaeed
路
3Comments
divega
路
3Comments
ryanwinter
路
3Comments
ericvb
路
3Comments
spottedmahn
路
3Comments