Ponyc: Missing actor in generated documentation

[EpicEric]

When building docs for the stdlib, it seems that the file for the _TestRunner actor is not being generated, which results in the following warnings in MkDocs:

$ ponyc packages/stdlib --docs --pass expr
[snip]
$ cd stdlib-docs
$ mkdocs build
INFO    -  Cleaning site directory 
INFO    -  Building documentation to directory: /home/eric/tmp/stdlib-docs/site 
WARNING -  Documentation file 'ponytest-TestHelper.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_ExclusiveGroup.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_ExclusiveGroup.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_Group.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_Group.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_SimultaneousGroup.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files. 
WARNING -  Documentation file 'ponytest-_SimultaneousGroup.md' contains a link to 'ponytest-_TestRunner.md' which is not found in the documentation files.

Sure enough, the actor is missing from the ponytest documentation.

[SeanTAllen]

Are we only generating public documentation but, there's a bug in that?

[EpicEric]

I don't understand your question.

Every other type (private or public) is generating documentation. This specific actor is not. Here's what the stdlib-docs/docs/ directory looks like:

$ ls -1 stdlib-docs/docs
assert-Assert.md
assert-Fact.md
assert--index.md
backpressure-ApplyReleaseBackpressureAuth.md
backpressure-BackpressureAuth.md
backpressure-Backpressure.md
[...]
ponybench-_TerminalOutput.md
ponytest-_Color.md
ponytest-_ExclusiveGroup.md
ponytest-_Group.md
ponytest--index.md
ponytest-ITest.md
ponytest-PonyTest.md
ponytest-_SimultaneousGroup.md
ponytest-TestHelper.md
ponytest-TestList.md
ponytest-UnitTest.md
process-CapError.md
[...]

For some reason, the ponytest-_TestRunner.md file is missing.

---

The logs in the OP essentially mean that every link to ponytest-_TestRunner.md will be broken and return a 404. For example, the reference to _TestRunner here.

[SeanTAllen]

There's an option to only generate public documentation. My question was, are we using that option when it fails?

[EpicEric]

I'm still not sure I understand you. Sorry.

The command I'm using is the same used to deploy to stdlib.ponylang.io. (reference: .travis_script.bash -> .travis_commands.bash -> Makefile-ponyc). There isn't any other option as far as I can tell.

Also, it only generates warnings. It doesn't stop documentation from being built; it only doesn't know which file to link to and the link will be broken.

[SeanTAllen]

@EpicEric there's an option to pony to only create docs for public types. looking at the makefile, that option is not being used when creating stdlib docs via the Makefile. See output of ponyc --help for more info about options around generating documentation.

[EpicEric]

I've opened issue #3340 for that.

This issue is about "why isn't that file being generated" when it should. There's probably some issue with docgen that prevents that file from being generated. I don't know anything about the compiler -- in this case, how it generates documentation -- to figure out why it didn't, eg. if it's related to the generated AST or something else.

[SeanTAllen]

So, if the file "isn't used" then it would be excluded. However, it is definitely included in the code that is exercised by the stdlib test program. I suppose that it is possible that it is "optimized away" prior to docgen but I would be shocked if that is the case. The documentation generation pass, as far as I know, happens before any optimization such as dead code removal.

[mfelsche]

The _TestRunner is excluded, because the docgen command excludes tests using some naming conventions. E.g. excluding everything starting with _Test:

https://github.com/ponylang/ponyc/blob/master/src/libponyc/pass/docgen.c#L130 and https://github.com/ponylang/ponyc/blob/master/src/libponyc/pass/docgen.c#L182

I also have some issues with this when building docs for https://github.com/mfelsche/ponycheck but it is not a biggie.

As a long-term goal i would love to have some kind of annotation for excluding types from documentation or forcibly including them (whetever is less verbose), that docgen would use to exclude or include certain types. But that might be just me.

[SeanTAllen]

@mfelsche I should have known that! I'm the one that put that awful hack in.

[mfelsche]

What is the general feeling about some kind of annotation for that?

If the feeling is positive overall I might start an RFC for that. Maybe we can probe for the feeling with :+1: and :-1: reactions on this comment?

[SeanTAllen]

I did 👎 because I'd rather revisit just about everything about documentation generation, not because I don't like the idea given the current system.

[SeanTAllen]

This could be "temporarily" solved by renaming _TestRunner to _RunnerOfTests. Please don't shoot me.

[jemc]

To add another possibly unpopular suggestion, I'll say what I've been saying since the start of my involvement with Pony - tests shouldn't be part of the package they are testing, they should be in a subdirectory or nearby companion directory.

This is what I do on all my Pony projects, and it works quite well, despite some limitations that have been cited by some in the past about not being able to write tests that engage directly with private types (this has some workarounds I could explain if desired).

If we did this, we wouldn't need the silly check that looks for _Test at the beginning of types.