Julia: Abstract iterator type?

[rofinn]

Would there be any interest in adding an AbstractIterator{T} type in base? There seems to be a common pattern of defining an iterator type anyways (e.g., KeyIterator, ValueIterator) and I think it would help with a lot of design issues within the package ecosystem if we could just accept an AbstractIterator{T} where T is some common eltype we want to support.

[stevengj]

Iterators in Julia are usually duck-typed, with traits to get the eltype. Because iterability is such a common thing to implement, if we tried to use an abstract type for this then then the lack of multiple inheritance would probably bite us — there would be no way to make a type iterable if its supertype was not iterable.

[stevengj]

You'd also run into problems from the fact that numbers are iterable (#7903). As I understand it, it would not be valid in Julia to recursively declare abstract type Number <: AbstractIterator{Number}.

[rofinn]

Yeah, I don't think this would cover all situations, but I figured it might be nice when designing an API to have the option of dispatching on an iterator type rather than including checks in your methods. Maybe the lack of multiple inheritance could be addressed by a default Iterator wrapper that uses duck-typing when it's constructed?

[stevengj]

If you dispatch on an iterator type, then you won't handle all iterators.

What checks are you doing in your methods? Duck typing means that you just iterate, and let Julia throw a MethodError if start etcetera aren't defined. Or do you mean checks for the eltype trait?

Also, there is a distinction between the case where the eltype is explicitly declared as Any and where the eltype is unknown, because in the latter case the caller may detect and narrow the type at runtime. For example:

julia> collect(Any[1,2,3])
3-element Array{Any,1}:
 1
 2
 3

julia> collect(x for x in Any[1,2,3])
3-element Array{Int64,1}:
 1
 2
 3

[rofinn]

If you dispatch on an iterator type, then you won't handle all iterators.

That seems fine as long as there's a way for someone to create an iterator type.

Duck typing means that you just iterate, and let Julia throw a MethodError if start etcetera aren't defined. Or do you mean checks for the eltype trait?

Yeah, I'd mostly like to exit early rather than waiting till I try to iterate. Currently, I'm just checking the eltype cause that's the most likely error condition.

[davidanthoff]

I think this is really something to be handled via traits (or interfaces or multiple inheritance or something like that). Note that there is an IsIterator trait in SimpleTraits.jl that you can use today to dispatch on iterators. I don't remember whether we managed to make it work as a static dispatch or whether you end up with a dynamic dispatch, though... The trait definition is here. @mauro3 do you remember whether that ends up with a static dispatch or not? I think the question was whether method_exists is pure or something like that, I never really understood the details of that :) @vtjnash also looked at that, back when we added that.

[oxinabox]

I have a package for checking that something implements the the Iterator interface.
https://github.com/oxinabox/InterfaceTesting.jl

Which is semi-orthogonal to this -- since the question is about dispatch not contract checking.
But I think relevant.
(Given in static OO languages (eg Java, C#), interfaces checking is done via interface inheritance)

Also the code in there shows what promises interfaces make, and how to check those.

[mauro3]

@davidanthoff: The IsIterator trait is somewhat broken at the moment (#40) as it still uses a problematic generated function. But it seems to work. @vtjnash made an interesting comment in that issue: "We should be able to make method_exists (and applicable) inferable now..." (here). Which would mean that we could ditch generated functions to check interfaces with method_exists and instead use @pure-functions.

[davidanthoff]

@mauro3 Ah, yes, now I remember! Would be nice if that would work eventually :)