Nim: Iterating closure iterator in nested function is empty

[michael72]

this is not the same but similar to #7787

When iterating a closure in an anonymous function the result is empty.
It also looks like there is no actual C-code generated for the loop inside the anonymous proc - I tried to add & "hello" to the string, but there was no "hello" in the C code.

# this works: directly iterating a closure
proc chk(cl: iterator: int {.closure.}): seq[string] =
  result = @[]
  for i in cl():
    result.add($i)

# this alos works: iterating an array inside an anonymous proc
proc chk_arr(a: array[3,int]): seq[string] =
  result = (proc(): seq[string] = 
    result = @[]
    for i in a:
      result.add($i))()

# this does not work
proc chk_fail(cl: iterator: int {.closure.}): seq[string] =
  result = (proc(): seq[string] = # auto or seq[string] doesn't matter here
    result = @[]
    for i in cl():
      # the code does not seem to be created
      # when adding result.add($i & "hello") - there is no "hello" in the C-code 
      result.add($i))()

iterator cl(): int {.closure.} =
  for i in [1,2,3]:
    yield i

echo(chk(cl))           # prints @["1", "2", "3"]
echo(chk_arr([1,2,3]))  # also prints @["1", "2", "3"]
echo(chk_fail(cl))  # prints @[]

[timotheecour]

doesn't look like it's related to anonymous, but instead it's related to being nested:

#not ok
proc chk_fail3(cl: iterator: int {.closure.}): seq[string] =
  proc bar(cl2: iterator: int): seq[string] =
    result = @[]
    for i in cl2():
      result.add($i)
  result = bar(cl)

proc bar4(cl2: iterator: int): seq[string] =
  result = @[]
  for i in cl2():
    result.add($i)
#ok
proc chk_ok4(cl: iterator: int {.closure.}): seq[string] =
  result = bar4(cl)

maybe change the title to reflect this?

[LemonBoy]

The problem is in transformFor, when c.tooEarly is set the transform drops the body and replaces it with a nkEmpty node and that's why no code is emitted at all.

The tooEarly variable is set in liftLambdas when the procedure body is empty or when the function owner is not a skModule, like it happens in this case.

Sadly I don't get what that flag is for, @Araq have any idea about this?

[Araq]

I don't remember but I do know this is all covered by tests, so change it and if the tests are still green, it's good.

[michael72]

OK - then lets try this out 😅
Created a pull request - this would countermand the following 2 commits:
https://github.com/nim-lang/Nim/commit/9c6d3e26cb7e0ece92694b15e649e7cb6f716ac8
and
https://github.com/nim-lang/Nim/commit/813f98fb342418606f82a082373636137b811bae
The latter is supposed to fix tforum.nim, but that's still working after the change... 🤔

wow - overlooked in my summerheat-delirium that @LemonBoy already did the change. Should really go to 🛌

[michael72]

Working now 😸

[timotheecour]

@michael72 you wrote:
"The latter is supposed to fix tforum.nim, but that's still working after the change..."
maybe edit your comment to:
"The latter is supposed to fix tforum.nim, but that's still not working after the change..."
(confusing...) abd maybe linke to the issue with tforum.nim ?

[michael72]

@timotheecour You're right https://github.com/nim-lang/Nim/commit/813f98fb342418606f82a082373636137b811bae#diff-9ed7d1759fc9c288c2fdde788c8c5aae was only documented as "further progress on closure iterators; tforum still failing".
OK - the "progress" itself due to this change does not seem to be apparent anymore - and all tests succeeded so...