Rcpp: forward_exception_to_r prevents destructors from executing

[wch]

When you catch an exception from an R function and pass it to forward_exception_to_r, it prevents destructors from executing.

Example:

Rcpp::cppFunction(
  includes = '
  class MyClass {
  public:
    MyClass() {
      fprintf(stderr, "MyClass constructor\\n");
    }
    ~MyClass() {
      fprintf(stderr, "MyClass destructor\\n");
    }
  };
  ',

  code = '
  void invoke(Function func) {
    MyClass m;
    try {
      func();
    } catch(std::exception &ex) {   
      forward_exception_to_r(ex);
    }
  }
  '
)


# Normal case: destructor runs
invoke(function() {
  cat("R code\n", file=stderr())
})
#> MyClass constructor
#> R code
#> MyClass destructor


# Error case: destructor does not run
invoke(function() {
  stop("Some sort of error")
  cat("R code\n", file=stderr())
})
#> MyClass constructor
#> Error in invoke(function() { : Evaluation error: Some sort of error.


# Quitting out of browser: destructor does not run
# When in browser, press Q to quit
invoke(function() {
  browser()
  cat("R code\n", file=stderr())
})
#> MyClass constructor
#> Called from: (function () 
#> {
#>     browser()
#>     cat("R code\\n", file = stderr())
#> })()
# Browse[1]> Q

[eddelbuettel]

Tricky one.

[coatless]

@wch First, take this response with a grain of salt as I'm sleep deprived at the moment. But, glancing at your code, what is happening is the _Rcpp_'s error classes are being negated by directly using the forward_exception_to_r function. To successfully have a clean session at the end, you need to use the escape hatches designed via Rcpp::stop() or Rcpp::warning().

c.f.

Rcpp::cppFunction(
  includes = '
  class MyClass {
  public:
  MyClass() {
  fprintf(stderr, "MyClass constructor\\n");
  }
  ~MyClass() {
  fprintf(stderr, "MyClass destructor\\n");
  }
  };
  ',

  code = '
  void invoke(Function func) {
  MyClass m;
  try {
  func();
  } catch(const Rcpp::eval_error& ex) { 
     std::string ex_str = ex.what();

      // Rethrow the cleaned exception with 100% control.
      Rcpp::stop(ex_str);  }
  }
  '
)


# Normal case: destructor runs
invoke(function() {
  cat("R code\n", file=stderr())
})
#> MyClass constructor
#> R code
#> MyClass destructor


# Error case with the C++ mechanism changed...
invoke(function() {
  stop("Some sort of error")
  cat("R code\n", file=stderr())
})
# MyClass constructor
# MyClass destructor
# Error in invoke(function() { : Evaluation error: Some sort of error.

Switching to Rcpp::warning() would give:

MyClass constructor
MyClass destructor
Warning message:
In .Primitive(".Call")(<pointer: 0x107d65ee0>, func) :
  Evaluation error: Some sort of error.

Sleep now.

[kevinushey]

This is the expected behavior. forward_exception_to_r() calls Rf_error(), which causes a jump to the top level (unless that error is caught by e.g. tryCatch() handler).

forward_exception_to_r() should be considered a somewhat internal function -- it's used by the auto-generated wrapper functions from Rcpp::compileAttributes() [edit: its companion function exception_to_r_condition() is used, and then stop() is called manually], and it effectively ensures that Rf_error() is called only after the C++ stack has been unwound and destructors have been run.

In other words, forward_exception_to_r() should only be used at the 'highest' level of a C++ context; any deeper and you risk skipping destructors.

[kevinushey]

Where are the callbacks registered in https://github.com/r-lib/later/blob/1ab43a49/src/callback_registry.h#L28-L34 called? If they're called within the context of an Rcpp function (ie: you're calling them somewhere within an // [[Rcpp::export]]ed function) then you might consider just letting the exception get caught at the top level and have the regular Rcpp machinery handle it.

Otherwise, you might want to use a local try-catch but avoid re-throwing the error (e.g. just warn, or swallow it)

[jjallaire]

Yes, forward_exception_to_r is an internal function not meant for calling
by library users.

If you want to forward an exception to R from within Rcpp you should just
throw it and the normal machinery will catch it and then forward it only at
the very, very top level so that all destructors execute.

On Tue, Sep 19, 2017 at 11:16 PM, Kevin Ushey [e-mail]
wrote:

Where are the callbacks registered in https://github.com/r-lib/
later/blob/1ab43a49/src/callback_registry.h#L28-L34 called? If they're
called within the context of an Rcpp function (ie: you're calling them
somewhere within an // [[Rcpp::export]]ed function) then you might
consider just letting the exception get caught at the top level and have
the regular Rcpp machinery handle it.

Otherwise, you might want to use a local try-catch but avoid re-throwing
the error (e.g. just warn, or swallow it)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/RcppCore/Rcpp/issues/753#issuecomment-330733689, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAGXx1E0YGdxSfcbfluob-wCLgDWuJ49ks5skIN2gaJpZM4PdQUl
.

[eddelbuettel]

Good stuff. We've been through this in one or two of our iterations. May make sense the comment this
more clearly in our headers.

[jcheng5]

Unfortunately, the top level in this case is an input handler, not an Rcpp function. So it seems like my best bet is to use a local try/catch, or call forward_exception_to_r or similar at the very top of my part of the C call stack. Thanks!

[jjallaire]

There are macros you should use to handle all cases correctly (including checking for interrupts). The macros are BEGIN_RCPP and END_RCPP and you can see their usage here:

https://github.com/rstudio/httpuv/blob/master/src/RcppExports.cpp#L10-L18

The END_RCPP macro assumes you are going to return an SEXP from the function. If you don't want to do this just use VOID_END_RCPP and then return whatever you like. Definitions of all the macros are here: https://github.com/RcppCore/Rcpp/blob/e1ef61d7b110e49929279eaf3e01950bcc3b4c07/inst/include/Rcpp/macros/macros.h#L30

END_RCPP is just this:

#define END_RCPP VOID_END_RCPP return R_NilValue;

[wch]

Thanks for all the great advice. For our use case, we've added BEGIN_RCPP and VOID_END_RCPP to the code that's called from an input handler, and it seems to fix the problem, at least when stop() is called.

There's still one case that's problematic, though: when quitting from the debugger. I'll illustrate with a simplified version of the original code I posted:

Rcpp::cppFunction(
  includes = '
  class MyClass {
  public:
    MyClass() {
      fprintf(stderr, "MyClass constructor\\n");
    }
    ~MyClass() {
      fprintf(stderr, "MyClass destructor\\n");
    }
  };
  ',

  code = '
  void invoke(Function func) {
    MyClass m;
    func();
  }
  '
)

# With stop(), destructor works.
invoke(function() {
  stop()
  cat("R code\n", file=stderr())
})
# MyClass constructor
# MyClass destructor
# Error in invoke(function() { : Evaluation error: .

# When in browser, press Q to quit.
# Destructor does not run.
invoke(function() {
  browser()
  cat("R code\n", file=stderr())
})
#> MyClass constructor
#> Called from: (function () 
#> {
#>     browser()
#>     cat("R code\\n", file = stderr())
#> })()
# Browse[1]> Q

I assume that since invoke() was created with cppFunction, it's automatically wrapped with BEGIN_RCPP and END_RCPP. With stop(), it works fine: the destructor runs. However if it's in the debugger and you quit, the destructor does not run. Should I file a separate issue for this?

[wch]

Also, since forward_exception_to_r() is meant for internal use, I'd suggest modifying this article to reflect that so that others don't run into the same problem:
http://gallery.rcpp.org/articles/intro-to-exceptions/

[eddelbuettel]

While all this is fresh on your mind, do you feel like putting down three or four bullet points which with a good cup of tea become a sibbling Gallery piece?

(I think historically forward_exception_to_r() wasn't quite as internal. This all evolved, thanks to pain-staking debugging and re-design.)

[jjallaire]

You can file an issue but I'm not sure if we'll be able to fix it. I think that Q does a C jump to top, effectively bypassing all destructors. The Rcpp::Function class actually sets up a top-level execution handler so in theory can catch long jumps, but the one done by Q must have some special properties that lets it evade our handler.

[jjallaire]

That article is actually demonstrating how to handle exceptions without the
benefit of the macros. It's correct as it stands, but is somewhat deceptive
because it doesn't caution about C++ destructors.

I think we should retire that article as correct manual exception
handling now effectively requires a copy paste of BEGIN_RCPP / END_RCPP.

On Wed, Sep 20, 2017 at 2:48 PM, Winston Chang [e-mail]
wrote:

Thanks for all the great advice. For our use case, we've added BEGIN_RCPP
and VOID_END_RCPP to the code that's called from an input handler, and it
seems to fix the problem, at least when stop() is called.

There's still one case that's problematic, though: when quitting from the
debugger. I'll illustrate with a simplified version of the original code I
posted:

Rcpp::cppFunction(
includes = ' class MyClass { public: MyClass() { fprintf(stderr, "MyClass constructor\n"); } ~MyClass() { fprintf(stderr, "MyClass destructor\n"); } }; ',

code = ' void invoke(Function func) { MyClass m; func(); } '
)

With stop(), destructor works.

invoke(function() {
stop()
cat("R code\n", file=stderr())
})# MyClass constructor# MyClass destructor# Error in invoke(function() { : Evaluation error: .

When in browser, press Q to quit.# Destructor does not run.

invoke(function() {
browser()
cat("R code\n", file=stderr())
})#> MyClass constructor#> Called from: (function () #> {#> browser()#> cat("R code\n", file = stderr())#> })()# Browse[1]> Q

I assume that since invoke() was created with cppFunction, it's
automatically wrapped with BEGIN_RCPP and END_RCPP. With stop(), it works
fine: the destructor runs. However if it's in the debugger and you quit,
the destructor does not run. Should I file a separate issue for this?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/RcppCore/Rcpp/issues/753#issuecomment-330945754, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAGXx2XeE4p-JcYDga1L7227mt0SEAG5ks5skV3tgaJpZM4PdQUl
.

[wch]

Here are some ideas for a future article:

• Exceptions must be handled with try-catch, but in the typical use case, you don't need to add that to your code because [[Rcpp::export]] wraps the code in the relevant macros.
• If the entrypoint to your code is not auto-generated, then you need to surround code with BEGIN_RCPP/END_RCPP. Also mention that VOID_END_RCPP should be used for void functions.
• If you don't handle exceptions, C++ destructors will not execute. (Should also mention other things that are handled by the macros.)
• For manual exception handling, you can use try-catch and forward_exception_to_r(), but this has to happen at the top level for destructors to work, and in practice it's not better than using the macros.
• If you're at the R debugger and quit from it, destructors will not execute.

I'll leave this issue open in case it'll serve as a reminder for the article; feel free to close if you don't think it needs to stay open.

I'll also file an issue for the debugger-quitting thing.

[jjallaire]

Great, that does indeed sound like the bones of a much improved article!

On Wed, Sep 20, 2017 at 5:16 PM, Winston Chang [e-mail]
wrote:

Here are some ideas for a future article:

• Exceptions must be handled with try-catch, but in the typical use
  case, you don't need to add that to your code because [[Rcpp::export]]
  wraps the code in the relevant macros.
• If the entrypoint to your code is not auto-generated, then you need
  to surround code with BEGIN_RCPP/END_RCPP. Also mention that VOID_END_RCPP
  should be used for void functions.
• If you don't handle exceptions, C++ destructors will not execute.
  (Should also mention other things that are handled by the macros.)
• For manual exception handling, you can use try-catch and
  forward_exception_to_r(), but this has to happen at the top level for
  destructors to work, and in practice it's not better than using the macros.
• If you're at the R debugger and quit from it, destructors will not
  execute.

I'll leave this issue open in case it'll serve as a reminder for the
article; feel free to close if you don't think it needs to stay open.

I'll also file an issue for the debugger-quitting thing.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/RcppCore/Rcpp/issues/753#issuecomment-330983439, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAGXxwG6gMmv7lEf1h4wW8I2j482BK6Wks5skYCZgaJpZM4PdQUl
.

[github-actions[bot]]

This issue is stale (365 days without activity) and will be closed in 31 days unless new activity is seen. Please feel free to re-open it is still a concern, possibly with additional data.