Roslyn: Extend LINQ syntax

[scalablecory]

I can't count the number of times I've written something like this:

var x = (from item in collection
         where item.foo
         select item.bar).Single()

Readability takes a nose dive, especially with complex or nested queries. There's a lot we can do to improve what is one of C#'s most powerful features. From low-hanging fruit that are compatible with existing infrastructure like ORMs:

from item in collection
select single item.foo;

from item in collection
select first item.foo or default;

from item in collection
select sum of item.foo;

from item in collection
skip 5
select top 3 item.foo;

from item in collection
left join item2 in collection2 on item.foo equals item2.foo
select new { item, item2 };

To bits which are currently only usable by LINQ to Objects:

from item in collection
group item.foo by item.bar into grp using StringComparer.InvariantCulture
select grp;

from item in collection by idx
select "item " + item + " at index " + idx;

[ErikSchierboom]

I really like this idea. Having to add the braces around the expression just to use the Single() call always seemed a bit unnecessary.

[svick]

I agree that something like this would be very useful, but I don't like that you're proposing 4 different syntaxes for 4 different, but very similar, operations.

What I would prefer is a single syntax that can be used to call any "finishing" method. VB.NET already has something like this, though I think its syntax is not ideal.

[eklam]

I like the idea as well, but your first query (and most of the others) could be made simpler just writing like this:
var x = collection.Single(item => item.foo);

[DustinCampbell]

How about this?

Aggregate item In collection
Where item.foo
Into Sum

:smile:

[giggio]

I like this idea a lot.

[scalablecory]

@svick @DustinCampbell On the surface I do like the idea of a more generic syntax. If we consider "from" to be a multi-result reduce, a single-result reduce keyword may be well anchored. I worry that readability will be sacrificed as it breaks out of the current Plain English syntax.

[jwooley]

While we're at it, go ahead and add Distinct, Skip and Take. Just steal the code from the VB code base.

[axel-habermaier]

@scalablecory: Agreed - this is a common thing to do and has always bothered me.So much, even, that I usually prefer the method-style syntax.

On the other hand, I like how F# supports most LINQ operations quite nicely. The underlying mechanism (computation expressions) even allows for extensibility, so in F# you can easily define your own LINQ variant with all the operators you could ever want. That's probably not necessary for C# as long as the most common operators are supported out-of-the-box.

[AdamSpeight2008]

Extend LINQ query syntax.
Simple Sequence

From y In 1 To 10
From x In 10 To 1 Step -1

Syntax is close to what is already used by VB's for loop.

It could be extended to include datetimes.

hours = From hr In #2015/01/01# To #2015/12/31# Step #01:00:00#

[metatron-the-chronicler]

I think that this could be solved in a much more comprehensive way if C# allowed functions to be used as infix operators. Rather than extending the Linq syntax, C#'s syntax should be extended to take care of this without it being limited to just what the compiler team has the time to provide.

[gafter]

@metatron-the-chronicler Do you have a specific proposal to offer?

[metatron-the-chronicler]

I actually wanted to make this a specific proposal but I haven't had the time to sit down and write it.

At any rate I think it would be a good idea to simply allow functions to be used in the infix position.

One might write something like

from c in collection
select somefield ToList()

the important thing is that ToList isn't some special syntax or expression. The compiler simply resolves to the method we all know and love based on the position (Though I can imagine that this might make parsing more complicated.) of ToList.

Of course the same infix notation would ideally be available outside the context of a linq expression. The main benefit of allowing any function to be used in the infix position other than resolving this particular issue would be allowing for cleaner dsls.

I've always found it to be a real shame that the query syntax wasn't implemented in this sort of way from the beginning. Though I suppose that doing so has other technical implications that I don't have the knowledge to dispute.

[gafter]

@metatron-the-chronicler I'd be interested in seeing your proposal if you ever get around to it.

[scalablecory]

@metatron-the-chronicler Maybe use attributes to define new linq keywords, so it doesn't have to break out of the Plain English look.

[GSPP]

select top 3 item.foo; really is not in the spirit of LINQ. The design principle is that you chain separate operations. select and top should not be mixed. SQL certainly is not a nice language and we do not strive to make LINQ more SQL like. LINQ is better.

What about:

from x in list
orderby x.Prop
skip 20
take 10
select x.Something

---

Regarding the materialization clause, let's make it support any collection at all:

from x in list
...
select x.SomeInt
as List<int>

or:

as int[]

or:

as HashSet<int>

Maybe we can shortcut to

as HashSet<>, List<>, []

We need some way to make the compiler pick up the appropriate conversion. Maybe look for a constructor with an IEnumerable signature.

---

Also, distinctness would be useful:

from x in list
distinct x by x.SomeKey
select x

We need new methods in the BCL: DistinctBy, ExceptBy, IntersectBy, UnionBy, etc.

---

Also, there is one helper method that comes up again and again on Stack Overflow: A method that can be used to partition a sequence into chunks of fixed length. This really should be added to the Enumerable class.

---

I work a lot with data. Often even big, nasty ETL-style queries. These features would be extremely handy. Especially the materialization feature because you often need to materialize for debuggability. This forces an ugly syntax: You need to wrap the query in braces and add this dangling .ToList() at the end. Code formatters have trouble with that.

[svick]

@GSPP

Also, there is one helper method that comes up again and again on Stack Overflow: A method that can be used to partition a sequence into chunks of fixed length. This really should be added to the Enumerable class.

You might want to propose that at the corefx repo, Enumerable is not part of Roslyn.

[GSPP]

@svick good point, https://github.com/dotnet/corefx/issues/2146.

[HaloFour]

@GSPP I'm sure you meant it as example syntax, but select <expression> as <type> is already valid LINQ syntax for attempting to type cast the result of the expression over each element in the sequence, e.g.:

var managers = from employee in employees
               where employee.IsManager
               select employee as Manager;

[jnm2]

Single, SingleOrDefault, FirstOrDefault or ToList is on just about every LINQ expression I write. This would be much appreciated.

Would Sum, Min and Max be able to be used this way?

Also, Entity Framework has SingleAsync, ToListAsync, etc. I don't suppose extension methods could be included? @metatron-the-chronicler's method would allow that.

[GSPP]

@HaloFour true. Suggestions?

from x in list
where x != null
select x into List<>

Maybe? Should be unambiguous. But the pattern doesn't work for Single etc. Probably this should not be based on type but on (extension) methods

[jwooley]

VB already supports the extra LINQ methods for Aggregate, Skip, Take, Distinct, Sum, Count. I have wondered what things would look like if you could an arbitrary extension method and have the compiler turn it into a LINQ operator. I suspect this is along the same lines as the infix recommendation. Consider the following:

<Extension>
Public Function PutMeInline(Of T)(
    items as IEnumerable(Of T), 
    predicate As Function(Of T, bool)) As IEnumerable(Of T)

    ‘ Do something and yield results
End Function

Public Sub Main
    Dim vals = {1, 2, 3}
    Dim test = 
        From num In nums
        PutMeInline num % 2 = 0
        Select num
        Distinct
        ToListAsync
End Sub

While this is an intriguing possibility, I’m sure the number of edge cases where the compiler would choke would make such a feature unwieldy.

[HaloFour]

@GSPP Unfortunately into is also valid there in projecting the partial result into a new named range variable:

from x in list
where x != null
select x into List
where List == null
select List

I'm not sure what would work best. While as and into both make sense they're both already used and they also imply how the query would be terminated which is not necessarily correct. Neither of those terms seem to make sense if you would want to terminate with Any or All.

:spaghetti:

I might start by borrowing the existing LINQ clauses from VB.NET, distinct, aggregate, skip and take which are probably all common use cases. Beyond that is it worthwhile to have a separate clause that would invoke a final method on the entire query?

from x in list
where x != null
then ToList();  // or apply, or terminate, or frob

Or a way to mark extension methods as extending the LINQ syntax with a well defined behavior as to how parameters are treated?

[LinqClause("tolist")]
public static List<T> ToList<T>(this IEnumerable<T> source) {
    // redundant much?
    return source.ToList();
}

from x in list
where x != null
tolist;

[GSPP]

@jwooley VB has the advantage that keywords are capitalized. In C# this would look awkward. LinqClause is an interesting idea but it's far fetched.

[AdamSpeight2008]

@GSPP VB.net is case-insensitive so it doesn't matter about the capitalization.

[metatron-the-chronicler]

@jwooley: After thinking about it my proposal for infix functions really only makes sense when you are able to provide implicit parameters, which you can via extension methods. Scala achieves this using a mechanism that is more or less identical to C#'s extension methods at the surface level at least.

@GSPP I would think that the best place to put any materialization function calls when using the query syntax is at the end after all where clauses and projections. In the case where someone wanted to put the ToList call on the same line the filter or select clause would need to be delimited with braces.

from c in collection  
select {c} ToList

Or perhaps we could treat everything to the left of ToList as one complete expression, requiring dot notation on anything we wanted to materialize while still constructing the expression.

[Indomitable]

I like the idea. My proposal is to add some new key words in the query syntax.

1. Single, SingleOrDefault, First, FirstOrDefault:

from x in collection
select x
single;

from x in collection
select x
firstOrDefault;

2.Take and Skip

from x in collection
skip 20
take 10
select x;

3.ToList and ToArray

from x in collection
select x
toArray

One more thing I miss in query syntax is the index in Where and Select functions.
If we add lambda expressions in query syntax things could become better:

from x in collection
select (x,i) => i;

from x in collection
where (x,i) => i % 2 == 0
select x;

[svick]

@Indomitable

One more thing I miss in query syntax is the index in Where and Select functions.
If we add lambda expressions in query syntax things could become better:

``` c#
from x in collection
select (x,i) => i;

``` c#
from x in collection
where (x,i) => i % 2 == 0
select x;

How would that work if you have more than one range variable? For example:

c# from x in collection1 from y in collection2 select (???,i) => ???;

[Indomitable]

@svick
What is the problem to have something like that:

from x in collection1
from y in collection2
where (x, i) => { return x != null && i % 2 == 0; } && (y, i) => { return i % 2 == 1; }  
select x;

Which will be translated to

int xi = 0;
foreach (var x in collection1)
{
  int yi = 0;  
  foreach(var y in collection2)
  {
    if ((x != null && xi % 2 == 0) && (yi % 2 == 1))
    {
      yield return x;
    }
    yi++;    
  }
  xi++;
}

[HaloFour]

@Indomitable LINQ query expressions are translated into equivalent extension method syntax, not unwrapped into foreach loops. It doesn't make sense to have a LINQ query clause accept lambda-like parameter lists for the range variables if the parameter lists can be restructured. It also doesn't make sense to try to include the index at the point of the where clause if the index would have to be determined much earlier on in order to be remotely accurate. If you were to use the Select or Where extension method overload accepting an index at that point in the query that index would increment per each element of the Cartesian product of x and y, not for the corresponding indexes of either range variable.

[svick]

@HaloFour That's a good point about computing the index too late. Which makes me wonder whether a syntax like the following wouldn't make sense:

``` c#
from x, xi in collection1
from y, yi in collection2
where // you can access all four variables here
select new { x, y }

Though if C# 7.0 adds good support for destructuring tuples, maybe something like this would work:

``` c#
// WithIndexes returns IEnumerable<(T, int)>
from (x, xi) in collection1.WithIndexes()
from (y, yi) in collection2.WithIndexes()
where // you can access all four variables here
select new { x, y }

[gordanr]

We can expect in LINQ queries (C# 7.0) something like this?

let (x,y) = something that makes tuple

[svick]

@gordanr I don't know. Maybe? The tuple proposal (#347), is pretty light on details (because they haven't been worked out yet). But it seems to me like a natural extension of the proposed tuple deconstruction syntax.

[leppie]

What about using a 'pseudo' property (similar to ROW_NUMBER() in SQL)? Eg:

from x in a
from y in b
where index(x) == 1 && index(y) == 5
select ...

or as extension method (but could be problematic)

from x in a
from y in b
where x.index() == 1 && y.index() == 5
select ...

[metatron-the-chronicler]

Has anyone thought about not requiring select (or group by) to end the query expression? It is entirely possible to not require a transformation function (projection) after filtering the source collection. It may prove easier to work with Take and Skip etc. As well as making query syntax more consistent with method syntax.

[GSPP]

The missing index is annoying sometimes. I solved that with an extension AsIndexed that converts the sequence into a Tuple to pass the indexes as normal data. Maybe we need such a framework method. Instead of a tuple this should be struct IndexedItem<T>.

[AdamSpeight2008]

@metatron-the-chronicler
the requirement for a select must be some specific to C# as VB doesn't require one to be present.

[AdamSpeight2008]

Zip would be a nice addition to the LINQ query style.

 From i In n
 From x In xs
 Zip i With x

[svick]

@AdamSpeight2008 I think that syntax is a bad idea, because it drastically changes what the second From means. And it wouldn't make sense if the second collection depended in the first value. For example, what would the following do?

vb.net From i In n From x In f(i) Zip i With x

[AdamSpeight2008]

@svick I ain't arguing about syntax more about the ability to pull from two enumerable at the same time.

Dim indexed = Enumerable.Zip( Iterator Function()
                                  Dim i = 0
                                  While True
                                      Yield i
                                      i += 1
                                  End While
                                End Function, 
                                someEnumerable )

Try doing that example using only query style LINQ operators.

[alrz]

Also iterate clause would be a good alternative to from:

F() iterate item select ...
// instead of
from item in F() select ...

[scalablecory]

@alrz what would the benefit be?

[alrz]

@scalablecory You can see the benefit in my example; if you want to chain some expressions together this provides a more readable code (just like match when you chain it multiple times):

obj match  ( ... ) match ( ... ) iterate item select ... 

Otherwise, this should be written like

from item in (obj match ( ... ) match ( ... )) select ...

I'm not sure if it's a good idea to introduce new syntax for existing constructs e.g. _query-expression_.

[HaloFour]

How about the following?

var dictionary = from employee in db.Employees
    where employee.City == "London"
    select employee into employees
    finally employees.ToDictionary(employee => employee.ID);

equivalent to:

var dictionary = (from employee in db.Employees
    where employee.City == "London"
    select employee).ToDictionary(e => e.ID);

I'm proposing a terminal clause to be permitted after a continuation clause which can be used to execute an expression using the continuation. I'm reusing the finally keyword as that clause but don't focus on that as much as the structure. It's a little verbose but it fits syntactically within LINQ and doesn't require strange rules for translating arbitrary methods or referencing the query itself. It has the added benefit of the range variables used within the query going out of scope so they could be reused as lambda arguments.

[jnm2]

@HaloFour I need this in my life.

On Sat, Nov 14, 2015, 9:48 AM HaloFour [email protected] wrote:

How about the following?

var dictionary = from employee in db.Employees
where employee.City == "London"
select employee into employees
finally employees.ToDictionary(employee => employee.ID);

equivalent to:

var dictionary = (from employee in db.Employees
where employee.City == "London"
select employee).ToDictionary(e => e.ID);

I'm proposing a terminal clause to be permitted after a continuation
clause which can be used to execute an expression using the continuation.
I'm reusing the finally keyword as that clause but don't focus on that as
much as the structure. It's a little verbose but it fits syntactically
within LINQ and doesn't require strange rules for translating arbitrary
methods or referencing the query itself. It has the added benefit of the
range variables used within the query going out of scope so they could be
reused as lambda arguments.

—
Reply to this email directly or view it on GitHub
https://github.com/dotnet/roslyn/issues/100#issuecomment-156706373.

[alrz]

@HaloFour At first I thought you are proposing exception-handling in query expressions (like Catch method in Rx, Ix)

With #5445 you should be able to write

var dictionary =
    from employee in db.Employees
    where employee.City == "London"
    select employee
    |> Enumerable.ToDictionary(employee => employee.ID);

[svick]

@HaloFour That's an interesting idea. Except that select employee into employees already has a meaning: it makes employees into a range variable. So I think this syntax would be confusing.

Maybe another keyword could be used instead of into?

[alrz]

Forward pipe might be not preferrable solution, I propose an optional then following _select-clause_

var dictionary = from employee in db.Employees
    where employee.City == "London"
    select employee then.ToDictionary(employee => employee.ID);

[HaloFour]

@svick That's exactly why I'm using it here. select into and group into both construct query continuations from which you can then use another from clause. At that point you have a named enumerable and a terminal clause referencing that named enumerable makes sense and doesn't require any syntax voodoo. I'm not really proposing a change to into itself.

This would involve the following changes to §7.16:

Modifying:

query-body:
    _query-body-clauses_<sub>opt</sub> _select-or-group-clause_ _query-continuation_<sub>opt</sub>
    _query-body-clauses_<sub>opt</sub> _select-or-group-clause_ _query-termination_<sub>opt</sub>

Adding:

query-termination:
    into _identifier_ finally _expression_

And amending §7.16.2.1 (of the C# 5.0 spec) adding that:

from ... into ... x finally expression

translates into something akin to (I'm not quite sure how to translate the expression):

var $x = (from ...); expression;

There's probably a much better way to describe that.

@alrz Possibly, but that involves bringing in a completely separate and non-LINQ feature into the equation. I'd rather something that stands on its own.

[alrz]

@HaloFour Yeah, I really like your proposed syntax since it would allow a generic _expression_ that then doesn't, so I presume it would translate to a sequence expression somehow?

var dictionary =
   (var employees = from employee in db.Employees
    where employee.City == "London"
    select employee; 
    employees.ToDictionary(employee => employee.ID));

But I think finally wouldn't be the right choice for that as it would be confused with exception-handling.

[svick]

@HaloFour Either I'm missing something, or you don't understand how into currently works. It does not give you a named enumerable, you can't use it in a following from (unless you originally had enumerable of enumerables). It describes a single item, just like any other range variable.

From your description, it sounds to me like you would expect the following to work:

``` c#
from i in new[] { 1,2,3 }
select i into a
from j in a
select j

But it does not, this is how you use `into`:

``` c#
from i in new[] { 1,2,3 }
select i into j
select j

[alrz]

@svick I think finally keyword causes to alter the range variable type to the type of the whole query, but if you're using _query-continuation_ rather than _query-termination_ you would get the regular range variables as before.

[svick]

@alrz That's how I understood @HaloFour 's proposal originally. And I think that would be incredibly confusing. Subsequent code shouldn't change the type of a variable that was declared previously and what it represents.

[HaloFour]

@svick You're right, it doesn't work with _query-continuation_. I misread the example in §7.16.2.1 and didn't bother testing select ... into x from x in x ....

For a _query-termination_ clause of the nature I've described it would need a new syntax to name the query as a whole which I think is something worth doing since it gives you a lot of flexibility in how that query is terminated, e.g. calling an extension method, passing to some other arbitrary method.

[alrz]

So the problem is just the into keyword being part of the _query-termination_? finally .. in .. looks good to me, if it doesn't imply iteration.

[HaloFour]

@alrz Yeah, that _query-continuation_ isn't similar enough to what I'd like of _query-termination_ in that it doesn't produce an identifier representing the query as a whole that could then be used as a part of an expression. Using random keyword placeholders I'm thinking of something like:

var dictionary = from employee in db.Employees
    where employee.City == "London"
    scooby employees doo employees.ToDictionary(employee => employee.Id);

[alrz]

@HaloFour eventually employees wouldbe employees.ToDictionary(employee => employee.Id);.

[alrz]

I believe that the query should not be "terminated" after that like @HaloFour suggested, I think the new clause (whatever it is) should be _a kind of_ query terminal.

_query-body:_
 _query-body-clauses_<sub>opt</sub> _select-or-group-clause_ _output-clause_<sub>opt</sub> _query-continuation_<sub>opt</sub>
 _query-body-clauses_ _output-clause_ _query-continuation_<sub>opt</sub>

_output-clause:_
 output _identifier_ in _expression_

So these would be possible:

// first form
from employee in db.Employees
select employee.City
output cities in cities.Distinct() into cities
from city in cities select city.ToUpper()

// second form
from employee in db.Employees
where employee.City == "London"
output employees in employees.ToDictionary(employee => employee.Id)

Since the expression cities.Distinct() could return any type, you should use _query-continuation_ to define a new variable which has a different scope from first cities hence it can use the same name.

PS: I don't see why the _select-clause_ isn't optional at the end of simple queries like VB?

[GSPP]

Any kind of "terminating clause" basically just adds convenience syntax for

(from x in y
 ...
 select ...)
.SomeMethodCallHere();

This would not work well with Skip and Take because those would benefit from range variables, like here:

from x in Enumerable.Range(0, 10)
let sq = x * x
skip 1
take 3
select sq //uses range variable

Skip and Take are the only operators that come to mind right now that could make use of range variables. So maybe we need to add those two operators to the language. Seems kind of arbitrary but it really is what a lot of people need. Also, these operations are "timeless" in the sense that 10 years from now we will still be calling these operations. They have little potential to become obsolete and replaced with something better.

We can solve many other cases with one of the "terminating clause" proposals from this thread. But for Skip and Take compiler-understood syntax would be helpful.

Or did I misunderstand the proposals? As far as I can tell none of them could support the 2nd query from this post.

[alrz]

@GSPP I'm strongly against supporting every single Enumerable method in query expressions. That doesn't make sense, F# does this well with generalizing query expressions into computation expressions which is not the direction that C# would take. (I don't remember the issue number, but it's now closed).

from x in Enumerable.Range(0, 10)
let sq = x * x
skip 1
take 3
select sq //uses range variable

I don't even understand this, why you should use let before skip and take? Of course you should call Skip and Take on the source collection.

If you want to use these methods (or any other methods) in the middle of the query, still output will be useful.

from employee in db.Employees
select employee.City
output cities in cities.Skip(1).Take(3) into cities
from city in cities select city.ToUpper()

[GSPP]

@alrz well, this is an example query and it is assumed that there is a reason for using let and paging late in the query. Here is a more realistic example:

x in Enumerable.Range(0, 10)
let expensiveValue = ExpensiveFunction(x)
where SomeComputation1(expensiveValue) || SomeComputation2(expensiveValue)
skip 1
take 3
select new { x, expensiveValue }

Here, we cannot page early and we need let to reuse this value.

Clearly, I can rewrite it to something else but I don't necessarily want to. The query is just right this way. This is a very practically relevant query structure.

I'm strongly against supporting every single Enumerable extension method in query expressions.

I am too and I did not mean to suggest doing anything like it. Would you be against adding a where clause as well if it did not already exist? The cut-off point is debatable. I think skip and take should make the cut and nothing else that I can think of.

Your output example drops the range variable employee which I might want to refer to later. This is the whole point of my previous post: "Custom clauses" drop range variables which is a regrettable property.

[alrz]

@GSPP I see. My suggestion is not about skip or take or where anyway. It has its own use cases.

[alrz]

@GSPP How about

from x in Enumerable.Range(0, 10)
let expensiveValue = ExpensiveFunction(x)
where SomeComputation1(expensiveValue) || SomeComputation2(expensiveValue)
select (x,expensiveValue)
output q in q.Skip(1).Take(3) into t
select t case(var x, var v) : new { x, v }

Although, I think you'd better to return the tuple right away. since tuples can have names I think they are superior to using anonymous types which are reference types. hence they would have better performance in loops.

In your example let itself causes to allocate an anonymous type anyway.

[TonyValenti]

I like the idea of simplifying the expression but I don't like how the functions are appearing at the end. What about moving them to the beginning?

Here is an idea:

var managers = FirstOrDefault()
               from employee in employees
               where employee.IsManager
               select employee as Manager;

var managers = ToList()
               from employee in employees
               where employee.IsManager
               select employee as Manager;

This also keeps the english pretty straight forward.

[HaloFour]

I think that expanding the LINQ clauses to common extension methods is reasonable. VB.NET has always had Skip, Skip While, Take, Take While, Distinct and Aggregate clauses. Cover most of the translation operators and I think a remaining terminal clause would cover nearly all use cases. I use the term "terminal" there but I don't necessarily intend that to mean that it must end the query, only that it allows referencing the query as a whole up until that point in order to use it in an expression. Further syntax to continue the query over the result of that expression would be welcome.

from employee in db.Employees
    let bonus = CalculateBonus(employee.Salary)
    orderby bonus descending
    skip 20
    take 10
    select distinct new { Id = employee.Id, Bonus = bonus }
    pink employees floyd employees.ToDictionary(employee => employee.Id, employee => employee.Bonus)
    from pair in employees  // doesn't quite feel right, maybe an into clause above?
    do Console.WriteLine($"{pair.Key}: {pair.Value}");

[HaloFour]

@TonyValenti

Syntax issues aside that would be pretty nasty with Intellisense. That's why LINQ put select at the end of the query, so that autocomplete would actually be useful because it would know the shape of the query up to that point.

[scalablecory]

@HaloFour I was thinking almost your exact same syntax earlier today.

If we can say that select only terminates a _context_ (variables/etc.) rather than the entire LINQ statement, a generalized map/reduce transform could be pretty powerful.

from employee in employees
where employee.IsManager
select employee
map results with results.Skip(1).Take(1) into result
select result;

from employee in employees
where employee.IsManager
select employee
reduce results to results.First();

[AdamSpeight2008]

That'll be Aggregate than.

[alrz]

What about a forward pipe operator specialized for LINQ:

Normally, select is just like map:

from item in list
select item.Foo
// equivalent to
list.Select(item => item.Foo)

If we want to change the projection operation,

from item in list
select item.Foo pipe Sum()
// equivalent to
list.Sum(item => item.Foo)

pipe behaves like a forward pipe operator (#5445) but in the LINQ context — it reuses the _selector_ item.Foo of the select clause in the following function. As a consequence we have:

from item in list
select item.Key pipe ToDictionary()
// equivalent to
list.ToDictionary(item => item.Key);

If we treat tuples like "functors":

from item in list
select (item.Key, item.Value) pipe ToDictionary()
// equivalent to
list.ToDictionary(item => item.Key, item => item.Value);

This behaves like ||> operator in F# but since in LINQ we have "range variables" it transforms values to functions (same happens when you use a regular select ).

Following the pipe operator rules, if this function hasn't an appropriate overload, we just apply Select:

from item list
where item.Foo
select item.Bar pipe ToList()
// equivalent to
list.Where(item => item.Foo).Select(item => item.Bar).ToList();

If _selector_ happen to be an _identity fucntion_ then we can just skip it:

from item list
where item.Foo
select item pipe ToList()
// equivalent to
list.Where(item => item.Foo).ToList();

This works with other functions, like

from item list
where item.Foo
select item.Bar pipe Skip(2).Take(3)
// equivalent to
list.Where(item => item.Foo).Skip(2).Take(3);

Now, assume that we use over instead:

from item in list
select item.Foo over Average()
// equivalent to
list.Average(item => item.Foo);

Because of this similarities, I think it would be a good idea to extend this to supportOVER-like behavior in SQL:

from item in list
select item.Foo over Average() partitionby item.Bar

We can mix this with, say, let, to project it altogether.

from item in list
let RunningTotal = item.Deposit over Sum() orderby item.Id
select new { item.Id, RunningTotal }

from item in db.SalesOrderDetail 
where item.SalesOrderID is 43659 or 43664
let Total = item.OrderQty over Sum() partitionby item.SalesOrderID
select new { item.SalesOrderID, item.ProductID, Total }

PS: Actually, over reiterates the collection, so does ToList(), etc. You see the pattern here.

[weitzhandler]

I strongly vote for this one.

I never use LINQ language queries and instead my coding guidelines is avoid using them and always use the extension methods directly, and that's because the ugly parentheses each query has to be wrapped in order to materialize it (ToArray, ToList, Sum, SingleOrDefault etc.).

Until this issue is addressed, language-build-in LINQ is merely useless.
I really hope to see this implemented soon. Maybe to a more limited extent (avoiding the introduction of a gazillion new language keywords.

I'd say the syntax should provide an operator that expects a shortcutted extension method available for IEnumerable<TElement>, for instance:

//parents is Parent[]
var parents = from student in students
                     where student.Age < 18
                     select student.Parent
                     call ToArray()

//student is Student
var student = from st in students
                      call SingleOrDefault(st => st.Id == id);

Asynchronous methods should also be supported:

   var student = from st in students
                         call await SingleOrDefaultAsync(st => st.Id == id);

Maybe there should be a verbose LINQ fashioned way to pass the arguments and completely avoid the use of parenthesis, but I personally don't see it as necessary.

Anyway this feature is crucial for the completeness of the LINQ syntax.

Some suggestions above proposed the ToList at the beginning of the query, but that's a no go, since we want to be able to process it after the selection, and we don't want to be limited to parameterless ex. methods only. What if we wanna call ToLookup with a key selector, bottom line we can't tie it up to the language, we just need to find a way to call any Enumerable ex. methods on the current query state and treat its result as the final type of the query.

[metatron-the-chronicler]

If the compiler does not do it already would it be impossible to just require the type to be given to the identifier being assigned to and have the compiler do the conversion itself?

[weitzhandler]

There are so many opened issues all addressing the same idea.
Wasn't sure which one is the actual discussion.
I'll happily delete all of them but one if I know there is a central active place.

[AdamSpeight2008]

What would be a good addition, is to also have the item index as well.

[alrz]

As @AdamSpeight2008 said, Aggregate in VB can already do this, but it's limited somehow, e.g. can't be used with ToDictionary overload with two params, It can be a great addition to C# though. I also suggest Aggregate to support Partition By and Order By for windowing.

[Eirenarch]

Adding terminating operators as keywords is probably not a good idea because there are too many of them. Single, SingleOrDefault, SingleAsync, SingleOrDefaultAsync, First, FirstOrDefault, FirstAsync, FirstOrDefaultAsync, Count, CountAsync, ToList, ToListAsync, ToDictionary, ToDictionaryAsync, ToArray, ToArrayAsync... where does it end?

On the other hand I think it is a very good idea to support non-terminating operators specifically Skip, Take, Distinct.

[MadsTorgersen]

We'll keep this on the backlog as a reminder to consider _something_ here.

The best proposal I've seen was

c# var x = from item in collection where item.foo select item.bar do Single();

(Or some other keyword). The idea is to add a query operator that is like a . but with different precedence.

[alrz]

Would like to see if it can handle something like ToDictionary,

var x = 
    from item in collection
    where item.foo
    do ToDictionary(item.Key, item.Value);

i.e. the range variable is available in the do clause.

[TonyValenti]

Grr... there was a minor typo....

I would say completely eliminate the variable declaration since I think the
goal is to make Linq a replacement for standard loops. Perhaps since that
is the goal, SELECT isn't even needed.

``````
from Parent in Parents
from Child in Parent.Children
let Age = (DateTime.Now - Child.DateOfBirth).Years
where Age <= 13
do {
//I should have access to all variables in scope here. Namely, Parent,
Child, and Age.
Console.WriteLine("Hi {0}, because you are {1} years old, you should
ask {2} for permission.", Child.Name, Child.Age, Parent.Name);
};

On Mon, Aug 15, 2016 at 4:26 PM, Tony Valenti [e-mail]
wrote:

I would say completely eliminate the variable declaration since I think
the goal is to make Linq a replacement for standard loops. Perhaps since
that is the goal, SELECT isn't even needed.

````
from Parent in Parents
from Child in x.Children
let Age = (DateTime.Now - Child.DateOfBirth).Years
where Age <= 13
do {
//I should have access to all variables in scope here. Namely,
Parent, Child, and Age.
Console.WriteLine("Hi {0}, because you are {1} years old, you should
ask {2} for permission.", Child.Name, Child.Age, Parent.Name);
};

````

On Mon, Aug 15, 2016 at 3:43 PM, Alireza Habibi [e-mail]
wrote:

Would like to see if it can handle something like ToDictionary,

var x =
from item in colleciton
where item.foo
do ToDictionary(item.Key, item.Value);

or not.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
http://t.sidekickopen55.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJN7t5XZsd0HHzW1pNnyC5vwmmjW8qSyyn56dD0mdSTSt602?t=https%3A%2F%2Fgithub.com%2Fdotnet%2Froslyn%2Fissues%2F100%23issuecomment-239922983&si=5089146699251712&pi=f59624b4-e545-4380-c3f9-c9fc17fb4058,
or mute the thread
http://t.sidekickopen55.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJN7t5XZsd0HHzW1pNnyC5vwmmjW8qSyyn56dD0mdSTSt602?t=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAM-qVl2RLlSUoNfdce9v70hAIZX5MS0Mks5qgM9zgaJpZM4DYJ5g&si=5089146699251712&pi=f59624b4-e545-4380-c3f9-c9fc17fb4058
.

--
Tony Valenti

--
Tony Valenti
``````

[alrz]

@TonyValenti That would be a "LINQ statement" which is already proposed in #1938.

[svick]

@TonyValenti

I think the goal is to make Linq a replacement for standard loops

No, that is not the goal of this proposal. The goal is to make queries that end with calls to methods like Single() or ToList() easier to write. I don't think your suggestion does that.

[scalablecory]

@TonyValenti the goal of this proposal is to make LINQ syntax align more with the standard feature set of LINQ methods, to reduce the need to fall back to that. It is not intended to replace loops.

[TonyValenti]

@MadsTorgersen I really like your proposal and it would be really nice if there was some flexibility on what follows the DO statement.

For example:
I should be able to write all of the following:

var items = new[]{1,2,3,4,5};

//This is a statement block that is essentially a "For Each" loop.
//For this loop, the statement has a void return time
from x in items
select x
do {
    Console.WriteLine(x.ToString());
}

//This is a statement block returns a List.
var items = 
from x in items
select x
do Skip(1).ToList();

//This might not be as useful but I figured I'd include it.
//This is a statement block that returns an iterator that does extra processing on each element..
//For this, the statement returns an IEnumerator<INT>
var Squares = 
from x in items
select x
do {
    yield return x * x;
}

[gafter]

We are now taking language feature discussion on https://github.com/dotnet/csharplang for C# specific issues, https://github.com/dotnet/vblang for VB-specific features, and https://github.com/dotnet/csharplang for features that affect both languages.

[scalablecory]

This is largely continued in csharplang #101.