Crystal: Splattable free vars

[jhass]

Let's generalize the special case Tuple currently has:

module MultiEnum(*T)
  def each_with_object(object)
    each do |{% T.map(&.class_name.downcase.id).argify %}|
      yield {% T.map(&.class_name.downcase.id).argify %}, object
    end
  end
end

class Hash(K, V)
  include MultiEnum(K, V)
  def each
    yield k,  v
  end
end

[asterite]

Interestingly, we were talking about generalizing *T with @bcardiff some days ago. That's why he also asked about having splats in generic instantiations.

There are several things here. First, I always thought about Ruby's enumerable as being implemented in a more complex way than how would one implement it in the first try. Probably something like this:

module Enumerable
  def each_with_index
    i = 0
    each do |*values|
      yield *values, i
      i += 1
    end
  end
end

That is, I believe the key is using a splat in the block argument and a splat in the yield. Of course, it's hard to know if Ruby does that because it's implemented in C.

However, if we generalize the notion of splats in generic type arguments we can probably do it without having those splats, just using macros, like you did. I was going to experiment with all of that soon (I want to do a few more important things first).

Another thing splats can be useful is to include Enumerable in Tuple. Right now it's pretty hacky:

struct Tuple
  include Enumerable(typeof((i = 0; self[i])))
end

That's cheating, because typeof shouldn't be used in the first pass where types are computed. A better, easier way is:

struct Tuple
  include Enumerable(Union(*T))
end

Basically, make a union of all the types in T (could result in just one type).

I'll definitely play and experiment with this :-)

The good thing is that Tuple, and now Union, have support for this in the language. What's missing is just the syntax bit, so it shouldn't be that hard. I still don't know if something like Foo(T, *U) would be needed or wanted. There's also the case of NamedTuple(**T) which has a double splat, which should also maybe be generalized (but we can start with *T).

As a side note, I always noticed that Enumerable doesn't really need the T parameter, it can be computed with typeof(first) (or using a different first method that makes sure to get the union of all yielded values... mostly because Tuple#first has the type of the first element). But that sometimes leads to "can't infer type of block", so maybe it's better to keep the T for now.

[asterite]

Oh, and I don't think we'll need MultiEnum, Enumerable will be that type.

[jhass]

Yes, MultiEnum was just for the example to not conflict with the existing one.

[asterite]

As a side note, I never really liked that Hash is Enumerable. I mean, you invoke map and get an Array back... what do you do, you Hash[...] the result? I think it's much faster to map directly to a new hash. Same goes for select, and many other methods. It promotes slow code to be used.

On the other hand, being able to see a Hash as an Array of tuples isn't bad either. But you can already do that with the non-yielding each method.

[kostya]

topaz and runbinus implement full enumerable by rubyspec, this is how looks each_with_index: https://github.com/topazproject/topaz/blob/master/lib-topaz/enumerable.rb#L51
https://github.com/rubinius/rubinius/blob/2bf206d50c0fb1916e17d936de2e7a4c2a42145b/core/enumerable.rb#L285

[jhass]

yeah, Ruby makes use of the autosplat feature here, that is yield only takes one argument, yield a, b is actually yield [a, b]. If the yielded value is an array and the receiving block takes more than one argument, unpack the array.

[asterite]

So...

This is now possible :-)

I'm sure there are a few cases left to consider, but now one can define a class/struct/module with a single splat type argument. In fact, with this one can create a custom tuple type:

struct MyTuple(*T)
  include Enumerable(Union(*T))

  @tuple : T

  def initialize(*@tuple : *T)
  end

  def each
    @tuple.each do |elem|
      yield elem
    end
  end

  def size
    {{T.size}}
  end
end

tuple = MyTuple.new 1, 'a'
p tuple # => MyTuple(Int32, Char)(@tuple={1, 'a'})
p tuple.size # => 2
p tuple.map(&.to_s) # => ["1", "a"]

I also tried making Enumerable be Enumerable(*T) and then doing include Enumerable(K, V) in Hash, using the hack of using macro code to traverse T. For map to work I also needed to have this possible:

module Enumerable(*T)
  # The block yields the types of T, splatted. From there, the compiler
  # is able to figure out the type for U
  def map(&block : *T -> U)
    # ...
  end
end

It worked! But I wouldn't use the macro hack yet... well, we could use it starting from the next release, but the most correct thing to do is to implement splat in yield and splat in block arguments. With that, we'll be able to write:

module Enumerable(*T)
  def map(&block :* T -> U)
    ary = [] of U
    each { |*e| ary << yield *e }
    ary
  end
end

and that's it, without too much noise. But we need all the pieces working together: splats in a type argument, splat in yield and splat in block arguments. I'll try to implement these two in the next days.

[ozra]

Neat! :-)

[zatherz]

This works: https://carc.in/#/r/1pof
This doesn't: https://carc.in/#/r/1pog

[Sija]

Just a reminder that the example given by @zatherz in the previous comment is still failing (as of 0.27.1). Maybe it would be worth reopening this issue or opening separate one for that?