Elixir: Rename @since ... to @doc since: ...

[josevalim]

The new docs chunk has a metadata field but currently there is no way for us to attach metadata besides using specific attributes like @since. However, if we need to add a new module attribute for each possible documentation, it won't scale. Things are made worse by the fact that we would need @since, @modulesince and @typesince.

To solve this problem, we should allow a keyword list to be given to @doc, @moduledoc and @typedoc which will be stored and attached as metadata.

So this code:

@since "1.3.0"
def foo

will become:

@doc since: "1.3.0"
def foo

This means this feature is now automatically available to types and modules too:

@moduledoc since: "1.3.0"

and we can attach any metadata we want, for example:

@moduledoc authors: ["Alice", "Bob"],
           cross_references: [AnotherModule]

The format is a bit more verbose but the change is worth in the sense it supports multiple metadata across different places.

This feature depends on us finishing adopting EEP 48 which is still work in progress.

Note: we won't change @deprecated because it has functional implications and it is not only used for documentation purposes.

[yordis]

How would you use it all together with comments?

@doc since: "1.3.0" """  <--- I am confused here
"""
def foo

or is just another @doc call?

[josevalim]

@yordis separate @doc call.

[lackac]

@josevalim I was thinking along the same lines for specifying the locale of the documentation if it's not "en". This would certainly open up the possibilities of what we could do with metadata.

[yordis]

Definitely this approach is better, specially that the core do not need to keep implementing more macros for this.

[lackac]

Still thinking about how we could use this for the doc text locales, do you think something like this would be feasible?

@doc "Welcome message"
@doc locale: "pt_BR"
@doc "Mensagens de Boas-vindas"
@doc since: "1.7.0"
@spec hello() :: binary
def hello, do: "Hello World!"

[lackac]

I think it's a bit too much, and I would be surprised to find some code that has documentation translated in the code to multiple languages. It's more likely that it would come from an external source.

[josevalim]

@lackac I feel like locales should not be in the source code, because it will cause a lot of churn, especially regarding to automations we may apply from the source documentation. We should push this to a separate discussion (but it is honestly low priority at the moment).

[lackac]

@josevalim I agree. This was coming from the fact that EEP 48 prepares for them and handling that made the implementation of the chunk and the usage of it a bit more complicated. It's good to know that it's low priority.

I like about this proposal that it allows the code to handle all these metadata in a unified way. I'm wary that it could lead to a overuse or even misuse of these keys.

Do you think of allowing arbitrary keys here or would there be a filter list? If allowing arbitrary keys, would there be reserved ones (e.g. since) and do we still put validations on some of them (value of since must be a binary)?

[josevalim]

@lackac some metadata is specified in EEP 48, so we can validate those, but any other field would be stored as is. I don't think we should filter because it defeats the purpose of metadata. Abusing a key-value store by storing useless keys is a light abuse as far as abuses go. :)

[lackac]

To clarify:

• overuse: if allowing arbitrary keys, developers and library authors might start to put all kinds of things here that do not necessarily belong there

• misuse: someone starts to use this to specify a key-value pair for a certain niche use and later the core team decides to use the same key for something else (or the other way around)

[lackac]

Abusing a key-value store by storing useless keys is a light abuse as far as abuses go. :)

A couple recent discussions on elixirforum come to mind. ;)

[michalmuskala]

While I believe we should have core accepting arbitrary metadata, we could have ex_doc raise on unknown elements - that way we could make accidental misuses (like typos) discoverable, while also not tying this to the release schedule of Elixir.

[fertapric]

Changing the topic...

Is @since going to be hard deprecated? or should both @since and @doc since: coexist at least for one Elixir version? I guess maybe coexist and emit some deprecation message when @since is used in 1.7, and remove @since in 1.8.

It’s not mentioned in the issue description and I though it might be good to define what to do with the current @since :)

[lackac]

@michalmuskala I like the idea of raising a warning. It could also come with a configurable whitelist so that people can add their own keys if they find a valid use.

[yordis]

@lackac that valid reason is a rabbit hole, I would prefer to that the Core does the thinking and do not allow whitelist.

At any point you can validate your valid reason by communicating with others but I don't believe we should be messing around with this.

[josevalim]

@fertapric yup, I will handle the deprecation afterwards. :)

[lackac]

Are there libraries that also use @since? If there are none we could handle the changes inside Elixir and replace @since completely. I'm guessing it is being used through an interface which wouldn't necessarily have to change.

@josevalim would @deprecated receive the same treatment?

[fertapric]

@lackac The original issue was updated. Check the note at the footer.

[yordis]

@lackac I have 5 packages using @since 😭and even private functions has it so.

[lackac]

@yordis if arbitrary keys will be permitted then ex_doc should either not warn for any of them or allow to suppress some of those warnings.

For example, someone could decide to assign colors to functions in their library and extend ex_doc to use these colors while rendering its UI in some way. It's something that only a limited number of people would use, but it could be a valid use for them.

[yordis]

Then do not add a whitelist at all because ex_doc shouldn't dictate what goes inside the metadata and should allow everything, which I am fine with it.

[michalmuskala]

To sum up, I think we should:

• introduce @doc keyword and @moduledoc keyword accepting any data
• deprecate @since in favour of @doc since: ...
• teach ex_doc to use the new docs chunk and take advantage of that
• make ex_doc raise on unknown metadata, allow overriding that behaviour by adding to the documentation configuration in mix.exs similar to other ex_doc options.

As a bonus, we should explore storing deprecated information in the docs chunk as well - this would mean deprecating @deprecated attribute in favour of @doc deprecated: "...". The primary concern here is speed of the mix xref task and compiler.

[lackac]

@josevalim currently when @doc "..." is specified more than once before a function we raise a warning. With introducing @doc since: "1.2.3" we should only raise if @doc is used more than once with a binary.

If it is used multiple times with keywords one option would be to merge them, allowing the latter to override keys from the former. I can't think of a case where this would be preferred over specifying it only once with all keys present, so it might make more sense to raise a warning in this case as well.

  @doc "Says hello"
  @doc since: "1.0.0"
  @doc "Says Hello World" # raises warning
  @doc color: :blue       # merges into current keywords with optionally raising a warning
  def hello, do: "hello"

We could also decide to use 2 arguments for @doc, @moduledoc, and @typedoc where the last argument would be the list of metadata defaulting to an empty list. While this removes the ambiguity of multiple use (we could raise a warning like we do now) it would create a special case for these attributes as we currently only allow a single argument for them.

  @doc "Says hello", since: "1.0.0", color: :blue
  @doc """
  Says Hello World
  """, since: "1.0.0", color: :blue # would raise warning in this case, included for demonstration

I'm not sure if this latter form would have any other repercussions (e.g. also creating a special case in the formatter).

Thoughts?

[josevalim]

We should go with this:

With introducing @doc since: "1.2.3" we should only raise if @doc is used more than once with a binary.

And this:

If it is used multiple times with keywords one option would be to merge them, allowing the latter to override keys from the former.

Overriding is likely irrelevant but the ability to specific @doc per part is important, especially because you can override docs altogether with things like:

@doc """
foo
"""
def previous_def_function(arg1, arg2)