Rspec-core: Reporting multiple errors

[ioquatix]

Sorry, this is more of a question, but how does one report one or more errors in an around(:each) block?

around(:each) do |example|
  duration = example.metadata.fetch(:timeout, 10)

  begin
    run_example(reactor, example, duration)
  ensure
    reactor.close
  end
end

It's possible for the user code to throw one or more exceptions.

I don't know how to correctly report these and couldn't figure it out from the documentation. This will either result in:

• A PR to add this capability.
• A PR to add documentation about this capability.

If you can advise me of where to start looking, I'll do the rest.

[JonRowe]

Change your around(:each) to around(:each, aggregate_failures: true) and it should be done automatically for you, we rescue them and aggregate them into one failure.

Closing as this was, as stated by OP, more of a question :)

[JonRowe]

In general the feature you are looking for is aggregate_failures though.

https://relishapp.com/rspec/rspec-core/v/3-9/docs/expectation-framework-integration/aggregating-failures

[ioquatix]

@JonRowe what about general exceptions, how do I add them?

[JonRowe]

Same way, an exception is a failure to us. If it doesn't work for you its because the exception will be one of our "do not rescue" exceptions which is reserved for system level failures which we respect.

[ioquatix]

What do I use for add_error in the following example?

around(:each) do |example|
  example.run
  # several jobs started
  jobs.each do |job|
    job.wait rescue add_error($!)
  end
end

[JonRowe]

around(:example) do |example|
  example.run

  # several jobs started

  # if job.wait does not block
  aggregate_failures("jobs") do
    jobs.each { |job| job.wait }
  end

  # if job.wait blocks, because the rescue needs to be inside the loop
  # note this is copied from `rspec-expectations/lib/rspec/expectations/failure_aggregator.rb`
  aggregator = RSpec::Expectations::FailureAggregator.new(label, metadata)

  jobs.each do |job|
    RSpec::Support.with_failure_notifier(aggregator) do
      begin
        job.wait
      rescue ExpectationNotMetError => e
        # Normally, expectation failures will be notified via the `call` method, below,
        # but since the failure notifier uses a thread local variable, failing expectations
        # in another thread will still raise. We handle that here and categorize it as part
        # of `failures` rather than letting it fall through and be categorized as part of
        # `other_errors`.
        aggregator.failures << e
      rescue Support::AllExceptionsExceptOnesWeMustNotRescue => e
        # While it is normally a bad practice to rescue `Exception`, it's important we do
        # so here. It's low risk (`notify_aggregated_failures` below will re-raise the exception,
        # or raise a `MultipleExpectationsNotMetError` that includes the exception), and it's
        # essential that the user is notified of expectation failures that may have already
        # occurred in the `aggregate_failures` block. Those expectation failures may provide
        # important diagnostics for understanding why this exception occurred, and if we simply
        # allowed this exception to be raised as-is, it would (wrongly) suggest to the user
        # that the expectation passed when it did not, which would be quite confusing.
        aggregator.other_errors << e
      end
    end
  end

  aggregator.notify_aggregated_failures
end

[ioquatix]

I see what you are doing.

Unfortunately:

     NoMethodError:
       private method `notify_aggregated_failures' called for #<RSpec::Expectations::FailureAggregator:0x00007feb548d7218>

=)

Do you want me to make a PR to make this possible?