I suppose running these tests like #7569 is pretty much automated and it shouldn't require much manual effort to get a list of broken examples.
Maybe we could integrate that into the pre-release cycle? Or even into CI with nightly-builds and maybe PR tests? The latter could be limited to files/examples changed in the pull request.
This would allow us to fix errors in the examples sooner and avoid distributing errors in the documentation.
/cc @maiha @bcardiff
Weekly seems enough. I am not sure how automated is the tool.
But I am reluctant to include dependencies outside crystal-lang org since it's outside of our control. It could act as an optional check before the release. But in that case, it could be automated outside the organization and still be valuable to know if everything is ok or not.
In the mid/long term, it sounds awesome, of course!
It would probably even suffice to run it manually for now. Maybe at the same time as preparing release notes.
Because there would be various examples such as pseudo code and random output, I think it is dangerous to put in CI now.
Greetings, <%= @name %>!
Random.rand(5) # => 2
For the time being, as you say, I agree with the idea of checking before release. Please contact me once the code is fixed. :pray:
Most helpful comment
Weekly seems enough. I am not sure how automated is the tool.
But I am reluctant to include dependencies outside crystal-lang org since it's outside of our control. It could act as an optional check before the release. But in that case, it could be automated outside the organization and still be valuable to know if everything is ok or not.
In the mid/long term, it sounds awesome, of course!