Crystal: Subclasses macro and inheritance

[Svenskunganka]

Consider this:

class A
  def self.run
    self.subclasses.each do |subclass|
      pp subclass.subclasses
    end
  end

  def self.subclasses
    {{@type.subclasses}}
  end
end

class B < A; end

class C < B; end
class D < B; end

This works:

B.subclasses # => [C, D]

But this gives compiler error:

A.run

in test.cr:9: macro didn't expand to a valid program, it expanded to:

================================================================================
--------------------------------------------------------------------------------
   1. []
--------------------------------------------------------------------------------
Syntax error in expanded macro: macro_140734105469040:1: for empty arrays use '[] of ElementType'

[]
^

================================================================================

    {{@type.subclasses}}
    ^

What I am trying to accomplish

I am writing a Filetype matching library that checks for magic numbers, and I want it to be extendable with very little overhead. I found that using Crystal's built-in Type System and some recursion over class children would end up with a really nice API.

This is the main Filetype class:

class Filetype
  def self.match(slice : Bytes) : self?
    classes = self.subclasses
    classes.each do |obj|
      result = obj.match(slice)
      return result if result != nil
    end
    nil
  end

  def self.subclasses
    {{@type.subclasses}}
  end
end

By inheriting it, you can create "categories" that group "matchers" together, like this Audio one:

class Filetype::Audio < Filetype
  class Midi < Audio
    def self.match(slice)
      return self if slice.size > 3 &&
        slice[0] == 0x4D && slice[1] == 0x54 &&
        slice[2] == 0x68 && slice[3] == 0x64
      nil
    end
  end

  class Mp3 < Audio
    def self.match(slice)
      return self if slice.size > 2 &&
        ((slice[0] == 0x49 && slice[1] == 0x44 && slice[2] == 0x33) ||
          (slice[0] == 0xFF && slice[1] == 0xFB))
      nil
    end
  end

  class M4a < Audio
    def self.match(slice)
      return self if slice.size > 10 &&
        ((slice[4] == 0x66 && slice[5] == 0x74 && slice[6] == 0x79 &&
          slice[7] == 0x70 && slice[8] == 0x4D && slice[9] == 0x34 && slice[10] == 0x41) ||
          (slice[0] == 0x4D && slice[1] == 0x34 && slice[2] == 0x41 && slice[3] == 0x20))
      nil
    end
  end
end

and with it, you can use case with Crystal's built-in Type System. Some examples:

Checking specifically if it's some form of Audio file:

slice = Bytes[0x4D, 0x34, 0x41, 0x20] # Dummy data

case Filetype.match(slice)
when Filetype::Audio
  puts "is an audio"
else
  puts "other"
end

Or if you want to handle specific Audio files you could do:

case Filetype::Audio.match(slice)
when Filetype::Audio::Mp3
  puts "is an mp3"
when Filetype::Audio::M4a
  puts "is an m4a"
else
  puts "other"
end

Adding types to an existing category:

class Filetype::Audio::Ogg < Filetype::Audio
  def self.match(slice)
    return self if slice.size > 3 &&
      slice[0] == 0x4F && slice[1] == 0x67 &&
      slice[2] == 0x67 && slice[3] == 0x53
    nil
  end
end

Adding a new category:

class Filetype::Document < Filetype; end

And populating it:

class Filetype::Document::Pdf < Filetype::Document
  def self.match(slice)
    return self if ...
    nil
  end
end

And finally using it:

case Filetype.match(slice)
when Filetype::Document
  puts "is a document"
else
  puts "other"
end

[asterite]

C and David don't have subclasses and you end up with an empty array literal.

[asterite]

David says the autocompletion, I'd say D.

[Svenskunganka]

But I'm never calling subclasses on C and D with the A.run method, only on A and B. I'd understand this error if I was using the all_subclasses macro instead of only the subclasses macro.

[Svenskunganka]

Is there a way to put the {{@type.subclasses}} into an Array with a set type restriction via that macro? It seems like that macro returns B:Class and not B. The type system doesn't recognize B:Class as an A type unless it is B.

This for example:

class A
  def self.run
    self.subclasses.each do |subclass|
      print subclass.subclasses
    end
  end

  def self.subclasses
    arr : Array(self) = {{@type.subclasses}}
  end
end

class B < A; end

class C < B; end
class D < B; end

A.run

gives error:

in test.cr:9: type must be Array(A), not Array(B:Class)

    arr : Array(self) = {{@type.subclasses}}
    ^~~

Since B inherits from A, it should be allowed to create an Array(B) with the type restriction of Array(A).

[asterite]

If subclasses is empty you want to return [] of NoReturn. You need to write that if and that hardcoded value in subclasses. I don't know if there's another way to do it.

[asterite]

By the way, run calls subclasses, and for each it calls subclasses. That's how you end up invoking it on C and D.

[Svenskunganka]

But the macro @type.subclasses only return direct subclasses, not the whole hierarchy. So that self.subclasses.each is actually iterating an array with a single value of B, which it then calls B.subclasses on, which should return [C, D], but it returns an empty array that makes the compiler complain. It's never calling subclasses on C or D.

See this: https://carc.in/#/r/3inc

[asterite]

We could probably modify macro calls to do this automatically, I'll give it a try...

[asterite]

Ah, but {{@type.subclasses}} makes the method be a macro method, so it's expanded for every possible subclass in a different way. That's why it happens.

[asterite]

It's a bit tricky to understand, in the case of A.subclasses you get [B, C], which has a type of A:Class+ (so a virtual type) and then calling subclasses on this will make it happen. It's a little mess, but that's the reason.

[Svenskunganka]

Ohhh I understand now. So, to test this I have implemented the following, which compiles but for some reason does not match properly. See: https://carc.in/#/r/3inp

In that example, Midi class inherits Audio, but it does not match properly in the case, which according to the docs should. Also note that the Midi class overrides the inherited self.match method with its own.

[asterite]

You want to when Filetype::Audio.class. That matches classes. If you do when Filetype::Audio it will match instance types.

[asterite]

And I think you might want to use the all_subclasses method instead of recursively traversing subclasses?