Crystal: Add TypeNode#docs method

[vladfaust]

It would be nice to be able to extract docs related to a node just like TypeNode#annotation does with annotations:

# It's the `Foo` class
class Foo
  # It's the `@bar` variable
  @bar : String

  # It's the `#foo` method
  def foo
  end
end

{% pp Foo.docs %}
{% for ivar in Foo.instance_vars %}
  {% pp ivar.docs %}
{% end %}
{% for method in Foo.methods %}
  {% pp method.docs %}
{% end %}

"It's the `Foo` class"
"It's the `@bar` variable"
"It's the `#foo` method"

A possible (and actual) use-case is generating REST API docs with ease:

# Return user by its ID
class Users::Get < Action
  # The User's ID
  @id : Int32
end

crystal src/generate-docs

{
  "actions": [
    {
      "Users::Get": {
        "description": "Return user by its ID",
        "params": {
          "id": {
            "type": "Int32"
          }
        }
      }
    }
  ]
}

Of course, it could be achieved with annotations, but .docs methods will make code documentation serve multiple purposes. I'm sure that there are tons of other use-cases where extracting the docs could be useful.

[straight-shoota]

What's the use case for having a #docs methods available in Crystal code?
The example usage of crystal src/generate-docs can probably already be implemented with crystal docs.

[vladfaust]

@straight-shoota, the schema JSON could be served in a handler, say Handlers::APISchema. I.e. in compilation time the Users::Get action could have defined def schema, which returns this JSON.

[straight-shoota]

But the schema doesn't change at runtime. You can easily embed it at compile time (simplified: {{ run `crystal docs --format json` }}).

[vladfaust]

@straight-shoota,

$ crystal -v
Crystal 0.27.0 [c9d1eef8f] (2018-11-01)

LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu
$ crystal docs --format json
Error: Invalid option: --format
$ crystal docs -h
Usage: crystal docs [options]

Generates API documentation from inline docstrings in all Crystal files inside ./src directory.

Options:
    --output=DIR, -o DIR             Set the output directory (default: ./docs)
    --canonical-base-url=URL, -b URL Set the canonical base url
    -h, --help                       Show this message

[vladfaust]

Plus it would not be an "easy embedding" if I wanted to extract REST Actions docs explicitly (omitting all other docs), as macros don't have that much tools to work with JSON strings. It would be easier to actually put docs one-by-one into a JSON instead.

[straight-shoota]

The --format option is available in master and will be included in 0.27.1.

[vladfaust]

@straight-shoota still requires to dive into the JSON in macros, which ain't easy at all.

[Sija]

@vladfaust you could write a helper script for that, use jq or other means.

[vladfaust]

@Sija true. Still, calling crystal docs will build docs for the whole program, while what I want is certain namespace. Building docs for the whole program definitely is slower than extracting docs for a limited set of types. And it's a common scenario to require these generated docs in development, i.e. to have fresh REST API schema on every development server start.

[buckle2000]

With this functionality, we can build something like pydoc in python.

$ pydoc3 bytes.fromhex
Help on built-in function fromhex in bytes:

bytes.fromhex = fromhex(string, /) method of builtins.type instance
    Create a bytes object from a string of hexadecimal numbers.

    Spaces between two numbers are accepted.
    Example: bytes.fromhex('B9 01EF') -> b'\\xb9\\x01\\xef'.

[Sija]

@buckle2000 isn't it like ri in ruby-land?

[buckle2000]

@buckle2000 isn't it like ri in ruby-land?

I don't use ruby. It's like ri but non-interactive.

In python, you can use help(bytes.fromhex) to get documentation of objects. In fact, pydoc3 is built on help. In you want to get docstring, you can use bytes.fromhex.__doc__.

[matthewmcgarvey]

Another use case is for a language-server-protocol like Scry to be able to load the documentation for a module/class/method on hover.

[Blacksmoke16]

Also had a plan to use this to auto fill out the description field of OAS 3.0 Schema Object to the docs applied to an ivar/property. Would also allow for setting the title to like the first line/sentence etc.

[asterite]

Note that for performance reasons documentation on top of AST nodes is not stored unless the compiler operates on doc mode. That means docs can't be available using reflection, unless we are willing to increase the memory occupied at compile-time... (for me it's a "no").

[Blacksmoke16]

Fair enough, I'll go to plan B then.