Efcore: System.ArgumentOutOfRangeException after migration to 2.0

[denismaster]

Hello. I'm trying to write dynamic filtration and sorting functionality in my application. On the previous version, 1.1, there is no problems. But when I started using the .NET Core 2.0 and EF Core 2.0, the exception was thrown:

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. 
Parameter name: index 
at System.ThrowHelper.ThrowArgumentOutOfRange_IndexException() 
at System.Collections.Generic.List`1.get_Item(Int32 index) 
at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.BindSubqueryProjectionIndex(Int32 projectionIndex, IQuerySource querySource) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression) 
at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit(Expression expression) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.Visit(Expression expression) 
at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitOrderByClause(OrderByClause orderByClause, QueryModel queryModel, Int32 index) 
at Remotion.Linq.Clauses.OrderByClause.Accept(IQueryModelVisitor visitor, QueryModel queryModel, Int32 index) 
at Remotion.Linq.QueryModelVisitorBase.VisitBodyClauses(ObservableCollection`1 bodyClauses, QueryModel queryModel) 
at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalEntityQueryableExpressionVisitor.VisitSubQuery(SubQueryExpression expression) 
at Remotion.Linq.Clauses.Expressions.SubQueryExpression.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.ExpressionVisitorBase.VisitLambda[T](Expression`1 node) 
at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes) 
at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalEntityQueryableExpressionVisitor.VisitMethodCall(MethodCallExpression node) 
at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor visitor, BlockExpression block) 
at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression node) 
at System.Linq.Expressions.BlockExpression.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.ExpressionVisitorBase.VisitLambda[T](Expression`1 node) 
at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes) 
at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node) 
at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalEntityQueryableExpressionVisitor.VisitMethodCall(MethodCallExpression node) 
at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor) 
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node) 
at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.ReplaceClauseReferences(Expression expression, IQuerySource querySource, Boolean inProjection) 
at
Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitSelectClause(SelectClause selectClause, QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitSelectClause(SelectClause selectClause, QueryModel queryModel) 
at Remotion.Linq.Clauses.SelectClause.Accept(IQueryModelVisitor visitor, QueryModel queryModel) 
at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel(QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult](QueryModel queryModel) 
at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](QueryModel queryModel) 
-— End of stack trace from previous location where exception was thrown —- 
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](Expression query, INodeTypeProvider nodeTypeProvider, IDatabase database, IDiagnosticsLogger`1 logger, Type contextType) 
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass15_0`1.b__0() 
at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler) 
at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler) 
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query) 
at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression) 
at Remotion.Linq.QueryableBase`1.GetEnumerator() 

Steps to reproduce

Include a complete code listing (or project/solution) that we can run to reproduce the issue.
SortDirection.cs
```c#
public enum SortDirection
{
Ascending=0,
Descending
}

SortExpression.cs
```c#
public class SortExpression<TEntity> where TEntity : class
    {
        public SortExpression(Expression<Func<TEntity, object>> sortBy, SortDirection sortDirection)
        {
            SortBy = sortBy;
            SortDirection = sortDirection;
        }
        public Expression<Func<TEntity, object>> SortBy { get; set; }
        public SortDirection SortDirection { get; set; }
    }

Query.cs
```c#
public class Query where T : class, IEntity
{
public Expression> FilterExpression { get; set; }
public IList> SortExpressions { get; set; }
public int? Skip { get; set; }
public int? Take { get; set; }
}

Repository.cs method:
```c#
    public IEnumerable<T> Get(Query<T> request, IEnumerable<string> includes = null)
        {
            var result = new PagedEnumerable<T>();
            IQueryable<T> query = _context.Set<T>().AsNoTracking();

            if (request.FilterExpression != null)
            {
                query = query.Where(request.FilterExpression);
            }
            if (includes != null)
            {
                foreach (var include in includes)
                    query = query.Include(include);
            }
            var sortExpressions = request.SortExpressions;
            if (sortExpressions != null && sortExpressions.Any())
            {
                IOrderedQueryable<T> orderedQuery = null;
                for (var i = 0; i < sortExpressions.Count(); i++)
                {
                    if (i == 0)
                    {
                        if (sortExpressions[i].SortDirection == SortDirection.Ascending)
                        {
                            orderedQuery = query.OrderBy(sortExpressions[i].SortBy);
                        }
                        else
                        {
                            orderedQuery = query.OrderByDescending(sortExpressions[i].SortBy);
                        }
                    }
                    else
                    {
                        if (sortExpressions[i].SortDirection == SortDirection.Ascending)
                        {
                            orderedQuery = orderedQuery.ThenBy(sortExpressions[i].SortBy);
                        }
                        else
                        {
                            orderedQuery = orderedQuery.ThenByDescending(sortExpressions[i].SortBy);
                        }

                    }
                }
                query = orderedQuery;
            }
            if (request.Skip.HasValue)
            {
                query = query.Skip(request.Skip.Value);
            }
            if (request.Take != null)
            {
                query = query.Take(request.Take.Value);
            }
            return query.AsEnumerable();
        }

When the sorting part is commented, everything is fine. But the exceptions is thrown even when there is only one, default SortExpression.
By default, I use Query, where Skip is null and Take is 10, FilterExpression is null and SortExpressions contains one element, default Sorting ( for example, obj=>obj.Id)
IEntity is interface which contains long Id, nothing more.

Further technical details

EF Core version: 2.0.0
Database Provider: Npgsql.EntityFrameworkCore.PostgreSQL
Operating system: Windows 10 x64 Creators Update
IDE: VS 2017 Upd.3, VS Code.

[smitpatel]

@denismaster - Can you give example of a query? Do you have includes in your query?

[denismaster]

Yes, there is a simple example of query, copied from debug view in vscode, that cause the error:

request: Query<Teacher>
{
  FilterExpression [Expression]: null,
  Skip:0,
  SortExpressions [IList]: Count = 1 {
    [0]: [SortExpression<Teacher>]{
      SortBy [Expression] : teacher=>teacher.Id,
      SortDirection [SortDirection]: Ascending
  },
  Take: 10
},
includes [IEnumerable] : {string[3]} {
  [0] [string] "Position",
  [1] [string] "Department",
  [2] [string] "Students"
}

The goal of provided request is selecting first 10 teachers from database and sort them by their Id in ascending order. I want to include some extra information from Department, Position and Students properties.
I'd like to provide also Teacher class, which maybe can help you.

public class Teacher : IEntity
    {
        public int Id { get; set; }  //this is IEntity property
        public string FIO { get; set; }
        public int MaxWorkCount { get; set; }
        public int PositionId { get; set; }
        public TeacherPosition Position { get; set; }
        public int DepartmentId { get; set; }
        public Department Department { get; set; }
        public List<DiplomWork> DiplomWorks { get; set; }
        public List<Student> Students { get; set; }
        public List<TeacherComment> TeacherComments { get; set; }
    }

[smitpatel]

This seems to be error in anonymous object in related queries with dynamically created first query with ordering

[denismaster]

Is there any hotfix available?)

[ajcvickers]

@smitpatel To investigate how common this is and what the workarounds might be.

[smitpatel]

The issue is with ordering by OrderBy<Teacher, object> . If you don't introduce casting to object everything works fine. If you introduce casting to object but with string type things work but for int type it is failing. Specifically this query fails
var query = context.Set<Order>().Include(t => t.OrderDetails).OrderBy<Order, object>(c => c.OrderID).ToList();

It is uncommon to introduce such casting.

[smitpatel]

The issue is in Include compiler generating collection query model. It is failing to combine 2 order by which are same but having different types due to convert. When we translate them to SQL, it becomes 1 projection hence throwing error because of mismatch between size of expressions in anonymous object & projection list.

[ajcvickers]

Hi @denismaster. We are gathering information on the use of EF Core pre-release builds. You reported this issue shortly after the release of 2.0.0 RTM. It would be really helpful if you could let us know:

• Did you consider testing your code against the pre-release builds?
• Was there anything blocking you from using pre-release builds?
• What do you think could make it easier for you to use pre-release builds in the future?

Thanks in advance for any feedback. Hopefully this will help us to increase the value of pre-release builds going forward.