Crystal: Type of value from infinite iterator includes Iterator::Stop

[digitalcora]

Iterator#cycle with no arguments returns an infinite iterator. Despite this, the type system thinks the iterator could eventually stop:

typeof([1, 2, 3].cycle.next) # => (Int32 | Iterator::Stop)

This can be worked around using cycle.next.as(Int32), but if the iterator is later changed to be finite, the compiler will not catch the error.

This can also be seen when iterating an endless range, such as (0..).

$ crystal -v
Crystal 0.30.1 [5e6a1b672] (2019-08-12)
LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu

[asterite]

This is a good point! The compiler should be able to detect this. I thought there might be a bug but in the code there's an explicit stop: if the array is empty then next will return Iterator::Stop. And the compiler has no way to know whether the array is empty or not.

[asterite]

With the above I mean: there's nothing we can do here, I think this issue should be closed.

[j8r]

Theoretically, the compiler could know if the collection is a Tuple, {1, 2, 3}.cycle.next, but the return type is still Int32 | Iterator::Stop.

[asterite]

We could always make a different cycle for tuple if that's what we really want. I wonder what's the use case here, though. next isn't usually directly used from iterators.

[asterite]

But note that if we do that and you chain it with any other iterator method like map then you'll get the stop back. We'll have to make an entire hierarchy of iterators that don't stop. I don't think it's a good idea.

[digitalcora]

I wonder what's the use case here, though.

Here's a simplified version of what I was doing when I encountered this:

class Widget
  @color : Symbol
  @@colors : Iterator(Symbol) = {:red, :green, :blue, :yellow}.cycle

  property color

  def initialize
    @color = @@colors.next
  end
end

puts Widget.new.color

Each new Widget is given the next available color, looping back around once all colors have been used. However, unless you add .as(Symbol), you get this:

instance variable '@color' of Widget must be Symbol, not (Iterator::Stop | Symbol)

But it sounds like, because an "infinite" iterator still has to stop if the collection is empty, there might not be a good way to make the compiler aware that some iterators will never stop...?

[asterite]

Well, if anyone wants to implement Tuple#cycle then please send a PR.

I think we shouldn't do anything here. Casting the value is fine. Another alternative is to use an array, an index and modulo.

[j8r]

@grantovich in your example you can cycle without using an iterator, by using an enum:

enum Color
  Red
  Green
  Blue
  Yellow

  def next : Color
    return self + 1 if value < {{ @type.constants.size - 1 }}                          
    self.class.new 0
  end
end

color = Color::Red
9.times do
  puts color = color.next
end

[digitalcora]

Thank you @asterite and @j8r for the context and suggestions. I've also found this can be done by implementing a custom Iterator, for which the code is pretty similar to the enum approach. Of course, defining custom enums and iterators is a bit more heavyweight than my original one-liner.

Personally I think it'd be confusing/surprising if just Tuple had special logic for infinite cycling. I could see maybe creating a cycle! or something that guarantees an infinite sequence by e.g. returning nil endlessly if the collection is empty... but then you'd need a Range#each! to get the same behavior for infinite ranges, and it starts looking messier.

Feel free to close, I'm happy just for this explanation and workarounds to be searchable by others 🙂

[j8r]

Note that using Symbol isn't type safe, but enum is.

[asterite]

We could add next! which raises on stop. That way you only need a bang instead of a cast (which also requires to type a full type).

[digitalcora]

I don't think a next! would offer a practical advantage over casting; in either case you get a runtime error, and not a compiler error, if you mistakenly use it with a finite iterator. It would be slightly more concise, but perhaps not as obvious that you're introducing a potential runtime error.

Anyway, I'm fine with closing this now, since I think the proposed workarounds are reasonable and I can see the difficulty in enabling the compiler to know whether a given iterator can ever stop.

[vlazar]

I've also faced this issue with Iterator#cycle.next being (Int32 | Iterator::Stop) recently, but I think sometimes infinite iterator just needs to be stopped.

And if somehow Iterator::Stop is removed, then it won't be possible to stop infinite iterator anymore?