Roslyn: Allow C# to use anonymous iterators.

[mburbea]

VB.net allows you to have a lambda return an iterator, it should hopefully be possible to use such functionality in c#. E.g.

Func<IEnumerable<int>> f = ()=>yield return 3;

While that's a terrible example it hopefully can demonstrate the point.

[HaloFour]

The one Devil's advocate argument against that is that you wouldn't be able to tell that the lambda was an iterator unless you read through the contents of the body. In VB.NET they dedicated a new keyword Iterator which is a required component of the method signature in order for that method to be an iterator. This is also why C# has the async keyword for denoting methods (and lambdas) as asynchronous and allowing the use of await.

[SLaks]

@HaloFour Actually, the reason that C# needs the async keyword is that await x; is already valid code if you have a class named await.
yield return is never valid syntax, so there is no need for an iterator keyword.

[MadsTorgersen]

I'm somewhat suspicious of anonymous iterators as a feature. Think about how you would use one to actually start getting values out:

``` c#
Func> iterator = () => { ... yield return some stuff ... };
var enumerable = iterator();
var enumerator = enumerable.GetEnumerator();
enumerator.MoveNext();
var actualElement = enumerator.Current;

I'm exaggerating a little here to draw out the number of steps it actually takes to unpack such an iterator lambda. That seems a little too far removed!

But think about it: you are rarely required to actually pass a `Func<IEnumerable<T>>` to anything. More likely you need to pass an `IEnumerable<T>`. So you would create an iterator delegate, execute it immediately and henceforth only use the result:

``` c#
Func<IEnumerable<int>> iterator = () => { ... yield return some stuff ... };
SomeMethod(iterator()); // need to invoke the iterator. The delegate object itself is ignored

Maybe local functions (as discussed in #14) would be a slightly better approach. Here you could define an actual named iterator function inside your method body, and use the result of calling that:

c# IEnumerable<int> iterator() { ... yield return some stuff ... } // local iterator function SomeMethod(iterator()); // still need to invoke the iterator, but there's no delegate object

Not perfect, but better than creating a delegate object just to immediately invoke and discard it.

[paulomorgado]

Other than the allocation and invocation of the delegate, what are the other benefits of local functions over lambdas or anonymous delegates?

As for the consuming source code, once you get the delegate, it's not much different than calling any method.

foreach(var item in (() => { ... yield return some stuff ... })())
{
    ...
}

I don't think I ever wanted to do that on production code. Usually it's some LINQ query over an existing enumerable. But for test code, that could be useful.

[SLaks]

Anonymous iterators would be more useful for SelectMany(); there have been a couple of times when I've needed to do that.

[MgSam]

I think the biggest benefit of anonymous iterators hasn't been mentioned yet, which is to combine the argument checking and iterator body into a single method.

``` C#
public IEnumerable Where(this IEnumerable enumerable, Func filter)
{
if(enumerable == null) throw new ArgumentNullException();
if(filter == null) throw new ArgumentNullException();

Func<IEnumerable<T>, Func<T, bool>, IEnumerable<T>> iter = (enumerable, filter) => 
{
      foreach(var item in enumerable)
          if(filter(item)) yield return item;
}
return iter(enumerable, filter);

}
```

Much nicer than having separate methods. A local named function would also work well.

[orthoxerox]

Yes, I agree they would be good for SelectMany. Sometimes you don't have the new sequence ready for you, and generating it using an external generator is complicated by the source being an anonymous type, so you can't really pass it outside.

For example, I had to generate a frequency dictionary of three-letter sequences out of a frequency dictionary of words, so I had a IEnumerable<{string Word, double Freq}> words after parsing the file. Of course there was a LINQy way: words.Where(w=>w.Word.Length>=3).SelectMany(w=>Enumerable.Range(0,w.Word.Length-2).Select(i=>new {Word = w.Word.Substring(i,3), Freq = w.Freq}));, and I could've passed only w.Word into the external generator and then projected the resulting sequence (that's actually what I've done), but my first impulse was to put a generator lambda with a for loop inside SelectMany, as that would be the clearest explanation of the algorithm.

[joelmartinez]

I realized this would be incredibly useful, after writing an anonymous delegate to flatten a hierarchy and finding that it wasn't supported. In the short term of course I've simply moved it into a separate method, but I'd like to lend a vote to this issue, while also hoping to stoke some implementation discussion.

The error message given by the compiler is CS1624, which states:

The body of 'accessor' cannot be an iterator block because 'type' is not an iterator interface type

So the yield return in the body of my lambda would be trying to return to the top level method's iterator (if there was one). So I think the trick here would be to define exactly how this would work ... you'd probably have to state that any yield return will return to the "most recent" iterator. So ...

IEnumerable<int> SomeMethod() {
    yield return 0;

    Func<IEnumerable<int>> anon = () => yield return 1;
    foreach(var i in anon()) yield return i;

    yield return 2;
}

The above would yield a sequence of {0, 1, 2}.

Well, and while I'm here, I have another idea that would further improve iterators ... the ability to yield return an existing enumerable ... so in the example above, instead of having to enumerate through the result of anon() to yield return, I could just short circuit it by yield return anon(), which would essentially have the compiler write out the foreach, yield return, etc. :)

[HaloFour]

@joelmartinez

So I think the trick here would be to define exactly how this would work ... you'd probably have to state that any yield return will return to the "most recent" iterator. So ...

I doubt that it would be possible any other way. A delegate may be invoked any number of times (including zero), or could be invoked outside of the context of the containing function.

[The-Zenix]

We use Coroutines extensively, so this functionality would be extremely useful.
The return values are handled by the coroutine system itself, so getting access to them is not an issue.

eg. If I want my character to return to a set position after 2 seconds. right now we might have something like this...

public void ReturnToTarget() { new Coroutine(InternalReturnToTarget); }

   ` IEnumerator InternalReturnToTarget()
    {
        yield return 2000;
        _controller.MoveToLocation = _targetPos;
    }`

Where something like the following would be a lot nicer.

public void ReturnToTarget() { new Coroutine(() => { yield return 2000; _controller.MoveToLocation = _targetPos; }); }

edit: Can't get code tags to work nicely.

[andreychizhov]

Maybe syntax like that (anonymous lambda-iterator + IIFE)?

IEnumerable<(int, bool)> SomeTuplesReturningMethod()
{
    return
        (() => 
        {
            yield return (1, true);
            yield return (2, false);
        })();
}

It can be converted to local functions

IEnumerable<(int, bool)> SomeTuplesReturningMethod()
{
    IEnumerable<(int, bool)> LocalFoo() 
    {
        yield return (1, true);
        yield return (2, false);
    }
    return LocalFoo();
}

[MadsTorgersen]

This scenario is better served with local functions, which can be iterators. They are going into C# 7.0.

[Thaina]

@MadsTorgersen The need of local function means it need to specify a name of function that should actually be anonymous. So if you really think this should implement by local function then should we have syntactic sugar for anonymous local function???

And I think that it would became lambda anyway. So please just allow yield return in lambda. You could just add feature to convert lambda to local function at compile time or anything

[eyalsk]

@Thaina local functions aren't emitted as lambdas.

[Thaina]

@eyalsk You misunderstand. What I said is opposite

What I mean is you should convert lambda to local function when we write lambda syntax with yield

[SLaks]

The term "emitted as lambdas" is meaningless; IL does not have any notion of lambdas.
Both local functions and lambdas are emitted as normal functions, sometimes inside closure classes.

The codepaths in the compiler that emit them are quite different (to avoid an extra allocation), unless the local function is converted to a delegate.

[alrz]

iterator lambdas would definitly required to have a block body and can easily get complicated. in that case you should actually prefer a local function for the sake of readability. Since return type is always inferred by target typing or overload resolution, I think this feature requires an even more extensive analysis on both sides which considering how common this is, it won't be reasonable to make the change to be useful for its rare use cases.

[eyalsk]

@SLaks You're right but you misunderstood my statement.

@Thaina wrote:

And I think that it would became lambda anyway.

I misunderstood her/him and I thought that he/she meant that local functions are emitted as lambdas so I merely stated that it isn't, nonetheless, thank you for the elaboration.

[Thaina]

@eyalsk Sorry for not being clear. What I mean is if you made an "anonymous local function" it would syntactically similar to lambda

[rummelsworth]

For folks like me who are landing here long after the dust has settled on the discussion, there's another good comment on anonymous iterators from Eric Lippert all the way back in 2009.

I agree with most of what's been said against this feature, and I'd much rather the Roslyn team spend their time on other more valuable features. That being said, iterators can be _one_ nice way (among others, for sure) to express sequences with conditional structure and/or values, and it'd be nice (sometimes) to avoid having to define a local function and add another name to local scope.

[mariusGundersen]

One thing that has happened since the answer by Eric Lippert in 2009 is async await, which are allowed inside lambdas and do behave a lot like yield return. So since the compiler is able to rewrite both lambdas and async await, it should be able to rewrite generators and lambdas in the same way, right?

[CyrusNajmabadi]

@mariusGundersen From teh compiler's perspective, there would be nothing limiting or preventing this. However, the compiler simply implements the language specification. And there is no-one on the language design team championing the change to allow this inside the language. If you want this feature you'd either have to open a proposal or find an existing proposal for this in dotnet/csharplang and vote on it. If ever accepted by the LDM it could happen. That said, it's a niche enough area, with lots of viable workarounds, that it's unlikely to garner any interest for the foreseeable future.

[Thaina]

Shouldn't this be closed and reopen in https://github.com/dotnet/csharplang/issues instead?

Also I have file a feature request that cover this too https://github.com/dotnet/csharplang/issues/249