Crystal: Can't use Array(T) in unions yet.

[q9f]

It's not possible to dynamically generate nested arrays of arbitrary depth due to a type restriction. It's not possible to define an array of type Array(T).

Minimal Code:

a = [] of (String | Array(String))
a.push "" # Ok
a.push [] of String # Ok
a.push [] of Array(String)
# => Error: no overload matches 'Array(Array(String) | String)#push' with type Array(Array(String))

b = [] of (String | Array(String) | Array(Array(String)))
b.push "" # Ok
b.push [] of String # Ok
b.push [] of Array(String) # Ok
b.push [[] of Array(String)]
# => Error: no overload matches 'Array(Array(Array(String)) | Array(String) | String)#push' with type Array(Array(Array(String)))

c = [] of (String | Array)
# => Error: can't use Array(T) in unions yet, use a more specific type

Why?

• I'm writing an RLP encoder and decoder that encodes arbitrary strings and arrays. Arrays can be nested of arbitrary depth. Apparently, there is no way of knowing the depth and it would require to use a generic Array(T) type declaration.

Related:

• #4913 but it's closed with a link to a private youtube video
• #2732 same same for enums
• #2733 discussing the potential of objects but not arrays

Crystal Version:

~/.opt/rlp.cr q9-decode*
❯ crystal -v
Crystal 0.32.1 (2019-12-18)

LLVM: 9.0.0
Default target: x86_64-pc-linux-gnu

~/.opt/rlp.cr q9-decode*
❯ uname -a
Linux ceres 5.4.13-arch1-1 #1 SMP PREEMPT Fri, 17 Jan 2020 23:09:54 +0000 x86_64 GNU/Linux

Checklist:

• [x] Make sure a similar issue doesn't exist yet: use the search box
• [x] Include reproducible code: we should be able to paste it into an editor, compile and run it and get the same error as you. Otherwise it's impossible for us to reproduce the bug.
• [x] Don't only use play.crystal-lang.org or carc.in: code might be lost and the issue will remain incomplete. Write code in the issue itself.
• [x] Reduce code, if possible, to the minimum size that reproduces the bug.
• If all of the above is impossible due to a large project, create a branch that reproduces the bug and point us to it.
• [x] Include Crystal compiler version (crystal -v) and OS. If possible, try to see if the bug still reproduces on master.

Is there any way I can circumvent not knowing the depth of the array to be created?

[q9f]

This is basically my recursive decoder in minimal code:

def foo(r : Number)
    if r === 0
        return ""
    elsif r === 1
        return [] of String
    elsif r === 2
        return Bytes[0x0]
    else
        bar = [] of (String | Bytes | Array(String | Bytes))
        return bar << foo Random.rand(3)
    end 
end

pp foo Random.rand(3)

Causes:

Showing last frame. Use --error-trace for full trace.

In so.cr:10:20

 10 | return bar << foo Random.rand(3)
                 ^-
Error: no overload matches 'Array(Array(Slice(UInt8) | String) | Slice(UInt8) | String)#<<' with type (Array(Array(Slice(UInt8) | String) | Slice(UInt8) | String) | Array(String) | Slice(UInt8) | String)

Overloads are:
 - Array(T)#<<(value : T)
Couldn't find overloads for these types:
 - Array(Array(Slice(UInt8) | String) | Slice(UInt8) | String)#<<(value : Array(Array(Slice(UInt8) | String) | Slice(UInt8) | String))

Even if I wanted to hardcode the type of some depth n, the compiler would understand the recursion and tell me at some point it would fail.

[jhass]

The standard solution to this is a recursive alias:

alias MyArray = String|Array(MyArray)
a = [] of MyArray
a << "" 
a << [] of String
a << [] of MyArray

[q9f]

Thanks, that's both an elegant and sufficient solution for my problem.

Will close this for now.

[straight-shoota]

Please note that recursive aliases are buggy and might be removed from the language (#5155). If it works for what you're doing, I guess that's fine.

But the following struct would do the same job and won't run into any issues with recursive aliases:

record MyArray, value : String | Array(MyArray)

[q9f]

Thanks. I came across #5155 while researching alias and am aware. I will consider a custom struct instead.

[asterite]

Please note that recursive aliases are buggy and might be removed from the language

In fact, the snippet provided by @jhass should not compile: a << [] of String is not good. If you do p typeof(a[0]) at the end of the program you'll get different results depending on whether you append that array or not.

It's buggy, don't use it. I wish the core team removes it eventually.

[RX14]

Let's disallow it sooner rather than later if it's broken.

Even if we can't find and remove all the broken supporting code, we can disallow creating them. This allows us to re-add them later if we can fix all the bugs.

(core team, :+1: to agree?)

[straight-shoota]

Already suggested this in https://github.com/crystal-lang/crystal/issues/5155#issuecomment-474880560. Let's continue this discussion there.

[jhass]

I don't know, the struct alternative with all its newing and having to go through the value accessor all the time seems just so much less elegant. If it works fine with a struct, can't we "just" make recursive "aliases" (wouldn't even be opposed to a new keyword for this) be a syntax level rewrite to a struct? Everytime a method argument is restricted to a recursive alias we rewrite the call side to wrap into the struct and everytime a recursive alias typed variable is accessed we append the value getter. This sounds too easy, please point out what I'm missing :)

[q9f]

using a struct instead of an alias loses all the benefits of the types used. in my case, I would have to figure out how to efficiently inherit all array features, i.e.,

Error: undefined method 'empty?' for MyArray

[Blacksmoke16]

Could use https://crystal-lang.org/api/master/Object.html#forward_missing_to(delegate)-macro.

[straight-shoota]

@q9f You just need to call empty? on value, not the instance directly. (And make sure it's actually an Array, of course)

[RX14]

This sounds too easy, please point out what I'm missing

You're missing the people to implement this.

It's clear nobody is willing to make this PR, so since the feature is broken the only thing we can do until we get a PR which fixes it is to deprecate the feature.