Rspec-core: SnippetExtractor with Ruby 2.3 squiggly heredoc cause error, exit without message

Created on 2 Feb 2016  路  10Comments  路  Source: rspec/rspec-core

It caused by Ruby's Ripper.lex bug.
I send a pull request to https://github.com/ruby/ruby/pull/1234

reproduce code:

describe 'squiggly heredoc in spec' do
  specify do
    foo.
    bar
  end

  BAR = <<~EOS
    test
  EOS
end

% be rspec foo_spec.rb
F

/usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:70:in `block in on_heredoc_dedent': undefined method `event' for #<Array:0x007f844b935b08> (NoMethodError)
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:69:in `each'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:69:in `on_heredoc_dedent'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:61:in `parse'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:61:in `parse'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:55:in `lex'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/2.3.0/ripper/lexer.rb:44:in `lex'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/source.rb:38:in `tokens'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/source.rb:52:in `tokens_by_line_number'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/snippet_extractor.rb:81:in `block in unclosed_paren_tokens_in_line_range'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/flat_map.rb:7:in `each'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/flat_map.rb:7:in `flat_map'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/flat_map.rb:7:in `flat_map'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/snippet_extractor.rb:80:in `unclosed_paren_tokens_in_line_range'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/snippet_extractor.rb:67:in `line_range_of_expression'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/snippet_extractor.rb:51:in `expression_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/snippet_extractor.rb:40:in `extract_expression_lines_at'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:207:in `read_failed_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:158:in `failure_slash_error_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:145:in `block in failure_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:144:in `tap'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:144:in `failure_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:32:in `colorized_message_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:229:in `formatted_message_and_backtrace'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:84:in `fully_formatted_lines'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/exception_presenter.rb:76:in `fully_formatted'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/notifications.rb:200:in `fully_formatted'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/notifications.rb:114:in `block in fully_formatted_failed_examples'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/notifications.rb:113:in `each'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/notifications.rb:113:in `each_with_index'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/notifications.rb:113:in `fully_formatted_failed_examples'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/formatters/base_text_formatter.rb:33:in `dump_failures'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:189:in `block in notify'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:188:in `each'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:188:in `notify'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:160:in `block in finish'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:175:in `close_after'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:156:in `finish'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/reporter.rb:79:in `report'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/runner.rb:117:in `run_specs'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/runner.rb:93:in `run'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/runner.rb:78:in `run'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/lib/rspec/core/runner.rb:45:in `invoke'
    from /usr/local/var/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.2/exe/rspec:4:in `<top (required)>'
    from /usr/local/var/rbenv/versions/2.3.0/bin/rspec:23:in `load'
    from /usr/local/var/rbenv/versions/2.3.0/bin/rspec:23:in `<main>'
picture hanachin
👍2

Most helpful comment

Monkey patch:

# https://github.com/ruby/ruby/commit/a43f2cbaa11c792cf417c6400d76710df77cd125
# Fix Ripper.lex error in dedenting squiggly heredoc
require 'ripper'
Ripper::Lexer.class_eval do
  def on_heredoc_dedent(v, w)
    @buf.last.each do |e|
      if e.event == :on_tstring_content
        if (n = dedent_string(e.tok, w)) > 0
          e.pos[1] += n
        end
      end
    end
    v
  end
end
picture zlobz on 8 Mar 2016
👍4

All 10 comments

As this is a Ruby issue, I'm not sure theres anything we can do.

JonRowe picture JonRowe on 3 Feb 2016

I agree with you, thanks!

hanachin picture hanachin on 3 Feb 2016

Is there a workaround for this short of removing all squigglies from the relevant code?

halilim picture halilim on 20 Feb 2016

The work around would not to be use the the squiggle to declare heredocs, <<EOS ... EOS is just as valid

JonRowe picture JonRowe on 22 Feb 2016

I'm using ActiveSupport strip_heredoc for now.

hanachin picture hanachin on 23 Feb 2016

Thanks. I've converted them to indented heredocs (<<-) for now.

halilim picture halilim on 23 Feb 2016

Seems there was a fix on ruby
https://github.com/ruby/ruby/commit/a43f2cbaa11c792cf417c6400d76710df77cd125

So guess we've gotta wait for 2.3.1

matthewrudy picture matthewrudy on 24 Feb 2016
👍2

Monkey patch:

# https://github.com/ruby/ruby/commit/a43f2cbaa11c792cf417c6400d76710df77cd125
# Fix Ripper.lex error in dedenting squiggly heredoc
require 'ripper'
Ripper::Lexer.class_eval do
  def on_heredoc_dedent(v, w)
    @buf.last.each do |e|
      if e.event == :on_tstring_content
        if (n = dedent_string(e.tok, w)) > 0
          e.pos[1] += n
        end
      end
    end
    v
  end
end
zlobz picture zlobz on 8 Mar 2016
👍4

2.3.1 was released with this fix: http://svn.ruby-lang.org/repos/ruby/tags/v2_3_1/ChangeLog

gerrywastaken picture gerrywastaken on 27 Apr 2016

Alternatively, I've just figured out that calling strip on the squiggly heredoc open statement seems to also address the issue.

str = <<~SQUIGGLY.strip
  Your String
SQUIGGLY
gerrywastaken picture gerrywastaken on 27 Apr 2016
Was this page helpful?
0 / 5 - 0 ratings

Related issues

wwood picture wwood  路  5Comments
jfelchner picture jfelchner  路  3Comments
marcparadise picture marcparadise  路  6Comments
RodneyU215 picture RodneyU215  路  3Comments
Drenmi picture Drenmi  路  4Comments