Truffleruby: ripper is missing in the standard library

[deepj]

See https://ruby-doc.org/stdlib-2.0.0/libdoc/ripper/rdoc/Ripper.html

To reproduce:

require 'ripper'

Error:

LoadError: cannot load such file -- ripper
    from ~/.rubies/truffleruby-1.0.0-rc12/lib/mri/rubygems/core_ext/kernel_require.rb:55:in `gem_original_require'
    from ~/.rubies/truffleruby-1.0.0-rc12/lib/mri/rubygems/core_ext/kernel_require.rb:55:in `require'
    from (irb):1
    from ~/.rubies/truffleruby-1.0.0-rc12/bin/irb:29:in `<main>'

[eregon]

@deepj Thanks for the report.
Do you know which applications or gems rely on Ripper?

[deepj]

@eregon rdoc or skippa

[eregon]

• rdoc: Is it used beyond generating documentation when installing gems (which I think most users disable)? I think basic rdoc already works.
• https://github.com/jnunemaker/crack doesn't seem to use Ripper.
• skippa seems rather original, what's the purpose to parse schema.rb without evaluating it?

It's a stdlib and it's a rather portable way to get an AST out of Ruby source code, so we should support it at some point, but so far it doesn't seem a priority.

[deepj]

@eregon definitely it's not a priority. The crack was a mistake. I forgot to delete it.

Yes, the purpose of skippa is parse schema.rb without evaluating it.

[deepj]

@eregon: just to rdoc, at least many rdoc tests fail on the issue

[nirvdrum]

busser is one I've just come across. It's frequently used with Test Kitchen for testing provisioning scripts in devops environments.

[deepj]

@eregon I just remember out why I reported this. It was due to rubyfmt

[chrisseaton]

We've stubbed ripper so it's now there... but it treats every Ruby file as an empty file. Probably not very useful to you.

[deepj]

Another library uses it. cane from Square

[eregon]

An idea to get Ripper support and not have to maintain a lot of code would be to reuse the Ripper C extension from MRI sources.
@chrisseaton started a branch with that: https://github.com/Shopify/truffleruby/commits/ripper
It does not compile yet though.

Tracked internally as GR-20329.

[chrisseaton]

My approach is to copy across ripper as a C extension, and then copy across supporting parts of MRI as needed, hacking it around to work in TruffleRuby, and that's sort of where I left it.

[bjfish]

Ripper is now implemented at commit: https://github.com/oracle/truffleruby/commit/7bdeca8546ba547af205ba41e7fbe4877c8d9ef7.

[chrisseaton]

Thanks @bjfish! Could you talk about how you did this in the end? Would make a great blog post to say that we finally implement this.

[bjfish]

@chrisseaton This was implemented using the C extension. First step was getting this to compile. Then, running new specs and MRI tests while fixing any issues that came up. This resulted in a lot of C API compatibility fixes that were implemented in separate PRs with new C API specs. For testing, it passes all of the Ripper MRI tests that were practical to to run (285 tests, 1755 assertions) and also a few gems that were known to use Ripper were also tested.