Nim: Rework Nim's exception handling

[Araq]

Nim's exception handling is currently tied to Nim's garbage collector and
every raised exception triggers an allocation. For embedded systems this is
not optimal, see http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r1.pdf
for more details.

So, here is what I propose: Clearly distinguish between "bugs" and "errors". Bugs
are not mapped to exceptions.

The builtin "bugs" are mapped to the new system.FatalError enum,
runtime errors are mapped to system.Error. FatalErrors are usually not
recovered from, unless a "supervisor" is used. (See the section about them.)

type
  FatalError* = enum         ## Programming bugs and other error
                             ## conditions most programs cannot
                             ## realistically recover from.
                             ## However, a "supervisor" can be
                             ## registered that does allow to
                             ## recover from them.
    IndexError,              ## index out of bounds
    FieldError,              ## invalid object field accessed
    RangeError,              ## value out of its valid range
    ReraiseError,            ## nothing to re-raise

    ObjectAssignmentError,   ## object assignment would lose data (object slicing problem)
    ObjectConversionError,   ## object is not a subtype of the given type
    MoveError,               ## object cannot be accessed as it was moved
    DivByZeroError,          ## divison by zero

    OverflowError,           ## integer arithmetic under- or overflow
    AccessViolationError,    ## segfault
    NilAccessError,          ## attempt to deref a 'nil' value
    AssertionError,          ## assertion failed. Is used to indicate wrong API usage
    DeadThreadError,         ## the thread cannot be accessed as its dead.
    LibraryError,            ## a DLL could not be loaded

    OutOfMemError,           ## the system ran out of heap space
    StackOverflowError,
    FloatInvalidOpError,
    FloatDivByZeroError,
    FloatOverflowError,
    FloatUnderflowError,
    FloatInexactError

  Error* = enum ## NOTE: Not yet the real, exhaustive list of possible errors!
    NoError,
    SyntaxError,
    IOError, EOFError, OSError,

    ResourceExhaustedError,
    KeyError,
    ValueError,
    ProtocolError,
    TimeoutError

proc isBug*(e: FatalError): bool = e <= AssertionError

Every error that can be caught is of the type Error. The effect system
tracks if a proc can potentially raise such an error. This means that
an "empty" except can be used to conveniently check for any meaningful exception,
comparable in convenience to an if statement/expression:

let x = try: parseJson(input) except: quit(getCurrentExceptionMsg())

The runtime calls system.panic on a detected bug, which roughly
has this implementation:

var panicHook*: proc (e: FatalError; msg: cstring) {.nimcall.}

proc panic(e: FatalError; msg: cstring) {.noreturn.} =
  if panicHook != nil:
    panicHook(e, msg)
  else:
    if msg != nil: stdout.write msg
    quit(ord(e)+1)

Out of memory

Out of memory is a fatal error.

Rationale: "Stack unwinding with destructors invocations that free memory" is too much gambling, if it works, that's mostly by chance as in more complex programs it becomes intractable.

The systems that are designed to continue gracefully in case of an OOM situation should make use of the proposed "supervisor" feature. The systems that are proven to work with a fixed amount of heap space should allocate this space upfront (the allocator already has some support for this) and then an OOM situation is a fatal, unrecoverable error too.

Implementation of 'try'

There are different implementation strategies possible for these light weight
exceptions:

1. C setjmp. Since this is pretty expensive, this will not be used.
2. C++'s try statement. Will be used for the C++ target unless the C++ dialect does not support exceptions. This is likely to offer the best performance.
3. Map Error to an additional return value and propagate it explicitly in the generated code via if (retError) goto error;
4. Map Error to a hidden pointer parameter so and propagate it explicitly in the generated code via if (*errorParam) goto error;. This is probably slower than using a return value, but this requires benchmarking.

The supervisor

Fatal errors cannot be caught by try. For servers in particular or for embedded devices that lack an OS a "supervisor" can be used to make a subsystem shut down gracefully without shutting down the complete process. The supervisor feature requires extensive stdlib support; every open call needs to register the file descriptor to a threadlocal list so that they can be closed even in the case of a fatal error which does not run the destructors if any implementation strategy but (2) is used. Ideally also the memory allocator supports "safe points", so every allocation that was done by the failed subsystem can be rolled back. For the usual segregated free-list allocators this seems feasible to provide.

The supervisor sets the panicHook to a proc that does the raise or longjmp operation. The supervisor runs on the same thread as the supervised subsystem.

Rationale: Easier to implement, stack corruptions are super rare even in low level Nim code, embedded systems might not support threads. Erlang's supervisors conflate "recover from this subsystem" with a concurrency mechanism. This seems to be unnecessary.

A supervisor will be part of the stdlib, but one also can write his own. A possible implementation looks like:

template shield(body: untyped) =
  let oldPanicHook = panicHook
  var target {.global.}: JmpBuf
  panicHook = proc (e: FatalError; msg: cstring) =
    log(e, msg)
    longjmp(target)
  let oldHeap = heapSnapshot()
  var failed = false
  if setjmp(target) == 0:
    try:
      body
    except:
      failed = true
  else:
    failed = true
  if failed:
    heapRollback(oldHeap)
    closeFileDescs()
  panicHook = oldPanicHook

# in a typical web server, that must have some resistance against
# programming bugs, this would be used like so:
shield:
  handleWebRequest()

How well the heap can be rolled back and external resources can be freed is a quality of implementation issue, even in the best case, created files would not be removed on failure. Perfect isolation is not even provided by Erlang's supervisors, in fact, every write to a disk is not rolled back! The supervisor feature is not a substitute for virtualization.

[cheatfate]

I think LibraryError must not be fatal error. Of course in most of the cases it is fatal error, but for projects which can embed DLLs into code at compile time it will be a problem. So to implement library which can enable such feature for any Nim project, you will need to implement supervisor.

[Araq]

@cheatfate This is nasty to solve since DLLs are loaded on program start, usually before the code runs that could override any "DLL loading hook" that we may want to introduce. This is a problem for another time. Currently the failure to load a DLL is a fatal error already and this proposal does not change that.

[Skrylar]

Doesn't using an enum for this brick user-defined errors?

[andreaferretti]

@Araq It is nice to have a more lightweight exception mechanism. I must say I have not read all your proposal, but two things stand out:

1. It is often useful to use custom exceptions. This is why exceptions are usually ref types. Custom exceptions can communicate more specific conditions (say HttpHeaderError vs ProtocolError) and can also have custom fields, which are fundamental to preserve some information that may be needed during recovery - for instance HttpHeaderError may have a field to specify which header was invalid in a response.

2. I don't agree with your classification of FatalError vs Error. Most errors that you list as fatals are actually very much recoverable:

var stat: float
try:
  stat = computeStatWithFastButNumericallyUnstableAlgorithm()
except FloatUnderflowError, FloatOverflowError:
  stat = computeStatWithSlowButNumericallyStableAlgorithm()

In your list I would consider at least the following recoverable: IndexError, FieldError, RangeError, ObjectConversionError, DivByZeroError, OverflowError, NilAccessError, LibraryError, FloatInvalidOpError, FloatDivByZeroError, FloatOverflowError, FloatUnderflowError, FloatInexactError.

[Araq]

1. Additional error information can be stored in thread local variables, this is how it's done under the hood anyway. There is very little benefit in hiding this fact. "Custom" exceptions are just sophistry and work against composable systems, from the paper:

Propagating arbitrarily typed exceptions breaks encapsulation by leaking implementation detail types from lower levels. (“Huh? What’s a bsafe::invalid_key? All I did was call PrintText!”)

Propagating arbitrarily typed exceptions is not composable, and is often lossy. It is already true that intermediate code should translate lower-level exception types to higher-level types as the exception moves across a semantic boundary where the error reporting type changes, so as to preserve correct semantic meaning at each level. In practice, however, programmers almost never do that consistently, and this has been observed in every language with typed propagated exceptions that I know of, including Java, C#, and Objective-C (see [Squires 2017] for entertaining discussion).

In fact, the "custom" exception DbError is worse than the more generic SyntaxError (ah, so the SQL query was illformed) vs ProtocolError (ah, so the client doesn't understand the bytes on the wire).

1. That cannot work in practice. If I write .raises: [] and the compiler tells me "no, you use array indexing which can throw IndexError" I have to throw away the compiler. I also don't want to catch this bug with try except, try except is for handling errors, not bugs. Protection against bugs is the business of the supervisor.

[Araq]

Doesn't using an enum for this brick user-defined errors?

That's a benefit in my opinion, Posix uses about 100 different error codes and does fine. In fact, every exception hierarchy also requires you to map your custom error to some existing error as you have to answer the question of which subtype of exception to inherit from.

[andreaferretti]

1. Using thread local variables is messy - like global variables you have the issue of choosing unique names, making sure you are not writing over something already used. This is a completely unstructured way of communicating errors. That the compiler does this under the hood is of little relevance - the compiler is there to help and translate a mantainable pattern into whatever is needed to make the thing work on actual hardware. The problem is when a human has to manually do this.

2. All the things I listed may genuinely be not bugs. I made an example with FloatOverflowError, but an example can be made for all of these cases.

Maybe I have a data structure like an array with holes. You can ask to get myDataStructure[i] for an integer i, but - unlike normal arrays - there is no simple rule to figure out whether index i is accessible, such as 0 <= i and i < len(myDataStrcuture). So if you are not sure you can do

let x = try:
  myDataStructure[i]
except IndexError:
  "no value"

Not to mention multithreading and the shared heap. For data stored on the thread-local heap, one can do

if 0 <= i and i < len(myArray):
  x = myArray[i]

In a multithreaded setting, this is a race condition, and the only safe way to do this may be (assume myArray is some kind of data structure stored on the shared heap but is not thread safe)

let x = try:
  myArray[i]
except IndexError, MoveError:
  "no value"

All the cases I have listed can be used for actual runtime errors that are not programming bugs.

Moreover, sometimes bugs do happen, and a programmer may want to use exceptions to make sure that a bug will not crash the whole program after running for a week. You propose the supervisor for this, but in your proposal the supervisor is global. One may want to be able to recover from errors at various levels of granularity

for file in files:
  try:
    let requests = readRequestsFrom(file)
    for request in requests:
      try:
        make(request)
      except:
        discard # go on with the next request
    except:
      discard # go on with the next file

[andreaferretti]

To make sure: I like your proposal, and if there is a way to make exception more lightweight and usable on embedded targets, I will be happy.

Nevertheless, I wouldn't like to make exceptions less useful than they are today

[andreaferretti]

answer the question of which subtype of exception to inherit from.

Exception, if nothing seems more specific

[andreaferretti]

Propagating arbitrarily typed exceptions is not composable, and is often lossy. It is already true that intermediate code should translate lower-level exception types to higher-level types as the exception moves across a semantic boundary where the error reporting type changes, so as to preserve correct semantic meaning at each level. In practice, however, programmers almost never do that consistently

No problem

# library.nim
type MyCustomError = ref object of IoError
...

```nim

client.nim

if I know about MyCustomError:

try:
doStuff()
except MyCustomError:
# let's handle this

if I don't know what is going on under the hood:

try:
doStuff()
except IoError:
# let's handle this

[Araq]

All the things I listed may genuinely be not bugs.

True, but they are all pervasive, every floating point operation could trigger it. So then the fact that my proc uses floating point at all would be reported as .raises: [FloatInvalidOpError, FloatDivByZeroError, FloatOverflowError, FloatUnderflowError, FloatInexactError]. It's terrible and no wide-spread programming language does it this way.

let x = try:
  myDataStructure[i]
except IndexError:
  "no value"

is already invalid code. Exactly this example is mentioned in the existing spec.

You propose the supervisor for this, but in your proposal the supervisor is global.

It's not, it can be nested.

let x = try:
  myArray[i]
except IndexError, MoveError:
  "no value"

That sounds like a data structure where [] can raise an exception. This would probably raise KeyError then which you can catch.

Using thread local variables is messy - like global variables you have the issue of choosing unique names, making sure you are not writing over something already used.

Nim has a module system for that.

This is a completely unstructured way of communicating errors.

Yes, as that's used as a debugging help most of the time. APIs are free to clearly communicate errors in a "structured" way. And no, error messages in strings are not structured either and have serious i18n problems.

That the compiler does this under the hood is of little relevance - the compiler is there to help and translate a mantainable pattern into whatever is needed to make the thing work on actual hardware. The problem is when a human has to manually do this.

The pattern of attaching arbitrary stuff to an arbitrary exception subclass is not structured. Currently we attach a stack trace to a raised exception which is then gone in release builds. It's a debugging aid, nothing more.

In practice, however, programmers almost never do that consistently

No problem

You cannot argue with toy examples against what has been experienced in practice, in the real world with multiple different programming languages.

[Yardanico]

I love this RFC! One thing about fatal errors: stdlib would need some work to add procedures like getOrDefault (or other procedures with default value) for arrays, sequences and other data types.

[andreaferretti]

True, but they are all pervasive, every floating point operation could trigger it. So then the fact that my proc uses floating point at all would be reported as .raises: [FloatInvalidOpError, FloatDivByZeroError, FloatOverflowError, FloatUnderflowError, FloatInexactError]. It's terrible and no wide-spread programming language does it this way.

You are right, I wouldn't want the effect system to be littered by that. On the other hand, I would like to catch them! A division by zero, maybe hidden inside a library over whose computations I have no control, cannot be a fatal error!

Java distinguishes between checked and unchecked exceptions - maybe the same could makes sense here.

Exactly this example is mentioned in the existing spec.

Which spec? I do not see this example

Yes, as that's used as a debugging help most of the time. APIs are free to clearly communicate errors in a "structured" way. And no, error messages in strings are not structured either and have serious i18n problems.KeyError then which you can catch.

Well, by an actual exception can be a struct with fields which have useful information, I never said to use strings for that

The pattern of attaching arbitrary stuff to an arbitrary exception subclass is not structured

Well, how else would I programatically recover from the error if I do not know the context? Your proposal is to use thread local variables instead: why not find a pattern where the compiler can translate from something ergonomic like current exceptions to thread locals? For instance, with the restriction that only one exception of a type can be raised at a time.

You cannot argue with toy examples against what has been experienced in practice, in the real world with multiple different programming languages.

Fair enough, but the point is that one does not really need to handle the exact error raised in a library, a supertype will do.

[dom96]

As far as I'm concerned this diminishes exceptions in Nim greatly by removing their ability to be extended. Like I've said in the forum: I use exceptions very heavily and love them dearly. Please do not diminish them.

The rationale for this change seems to be embedded systems, but how many of Nim's users are actually targeting embedded systems? How many of those users are limited by Nim's exceptions? I have never heard anyone complain about Nim's exceptions so to me this seems like an almost theoretical problem with a solution that is very disruptive. Why don't you instead make it possible to raise stack-allocated objects (_as well as_ heap-allocated objects)? Wouldn't that solve this problem too?

[Araq]

If we decide on a size limit for the largest possible exception type (say, 128 bytes), we can statically allocate the exception data. But then that's not a ref type anymore. All the other points my RFC addresses would remain:

• Can you write try removeFile except OSError? Or is that try removeFile except OSError, IOError? How can I write "do this when 'removeFile' fails" concisely? Note, that I'm not interested in handling the bugs (!) that removeFile might have.

• How can I ensure all errors (and not the bugs!) are handled exhaustively and in a meaningful way? An enum makes that easier than an open exception hierarchy.

• It enforces a clear methodology on how error handling needs to be done, map the errors to the existing error values, map bugs to AssertionError. No pointless inherited GoogleAppEngineError exceptions that are much more of an implementation detail than a clear ProtocolError.

[Varriount]

I've already mentioned this on IRC, but would like to add it here for visibility - chief situations in which extended information is useful are HTTP wrapper libraries and RPC libraries. HTTP protocols in particular don't always map their errors to HTTP codes. I've wrapped protocols that always return success, but put error information in the response body.

Furthermore, while extended error information is mostly used for debugging, it's also vital when analyzing crashes in production systems. While one can get around the lack of extended information by printing contextual information on the exception handler, this tends to make code bloat up with logging messages everywhere.

The C++ proposal provides an escape hatch for this - there is a "payload" part of the error structure that can point to another, larger structure containing more information.

[markprocess]

For reference, I'd like to point out that a class hierarchy for exceptions seems to confuse implementation inheritance and except: classification. For example, an imaginary FileError could have a name field that is inherited by FilePermissionError that adds detail about the actual permissions. Also, it would likely have overloaded methods. On the other hand, an except: FileError clause is operating on the classification of the exception, which might be entirely different from the implementation hierarchy.

Again, error handling is hard. Everyday programming (i.e. excluding security, concurrency, and versioning) would be trivial if it weren't for error handling.

Regarding the proposal - what can except: take? Is it limited to FatalError and Error?

[Araq]

I've wrapped protocols that always return success, but put error information in the response body.

Irrelevant. You need to write extra logic to map that to an error then. You don't need an "extensible" error in order to be able to do this.

While one can get around the lack of extended information by printing contextual information on the exception handler, this tends to make code bloat up with logging messages everywhere.

You don't have to "print" this information. You can return extended error information. Either in the return value, in an output parameter or in a thread local variable. Or you keep it in the "context" parameter that every library ends up having for other reasons (re-entrancy).

[markprocess]

proc isBug*(e: FatalError): bool = e <= AssertionError

This needs to be renamed isDefect

[andreaferretti]

Can you write try removeFile except OSError? Or is that try removeFile except OSError, IOError?

I don't know, but the effect system tracks this, so I can find out by either the documentation or an IDE

How can I ensure all errors (and not the bugs!) are handled exhaustively and in a meaningful way?

Surely not by compiling a list of all possible errors and hardcondig it into an enum. The world is varied, and the errors are as well. I cannot see how enforcing some typing on that (albeit dynamic) can be bad

It enforces a clear methodology on how error handling needs to be done, map the errors to the existing error values, map bugs to AssertionError

Unfortunately, many legitimate errors are also mapped to fatal in your proposal

[mikra01]

I also like this rfc. @dom96 with Nim I target both tiny and huge envs. I've planned to get some nim code running on the ATtiny....

[Araq]

I don't know, but the effect system tracks this, so I can find out by either the documentation or an IDE

https://nim-lang.org/docs/os.html#removeFile,string only lists OSError. existsOrCreateDir lists OSError, IOError. The effect system tracks implementation details and I'd better catch IOError too when calling removeFile, maybe.

[Araq]

Unfortunately, many legitimate errors are also mapped to fatal in your proposal

These cannot be caught in today's Nim either, these all come from checks that can be disabled.

Surely not by compiling a list of all possible errors and hardcondig it into an enum. The world is varied, and the errors are as well. I cannot see how enforcing some typing on that (albeit dynamic) can be bad

Sorry, but that's like saying "the world is dynamic, we need dynamic typing, static typing won't do".

[andreaferretti]

https://nim-lang.org/docs/os.html#removeFile,string only lists OSError. existsOrCreateDir lists OSError, IOError. The effect system tracks implementation details and I'd better catch IOError too when calling removeFile, maybe.

Sorry, I am a bit lost: if removeFile only lists OSError why do you insist in catching IOError?

These cannot be caught in today's Nim either

At least some of your fatal errors can

for x in 0 .. 10:
  try:
    echo 100 div (5 - x)
  except DivByZeroError:
    echo "can catch DivByZeroError"

let s = @[1, 2, 3, 4]

try:
  echo s[5]
except IndexError:
  echo "can catch IndexError"

[markprocess]

What can except: take? Is it limited to FatalError and Error?

Can I stil use try/except/raise to do an intentional stack-unwind (e.g. ForumError in nimforum?)

If all exceptions are suffixed with Error, why isn't it called try/error instead of try/except?
Why isn't this called reworking errors instead of reworking exceptions?

Can you implement try/except alongside try/error? (error being this proposal)

[dom96]

Can you write try removeFile except OSError? Or is that try removeFile except OSError, IOError? How can I write "do this when 'removeFile' fails" concisely? Note, that I'm not interested in handling the bugs (!) that removeFile might have.

This is just a case of convention. removeFile should specify in its documentation that it will raise an IOError if an actual error occurs. It should wrap OSErrors into an IOError and then everything else can be considered a bug.

This is how I deal with errors in Nimble, choosenim and pretty much all my software. Nimble defines a NimbleError which is basically a "unrecoverable error", i.e. quit Nimble now and show the user an error message. The fact that this is an exception has many advantages:

• I can tell where this error was raised very easily.
• Users of my code can handle this error and show it in an appropriate fashion: CLI message or a GUI window.

How can I ensure all errors (and not the bugs!) are handled exhaustively and in a meaningful way? An enum makes that easier than an open exception hierarchy.

Do it in the same way you are with your enum? Give each exception type a field called isBug? This issue is IMO completely orthogonal to the problem of "ref exceptions".

[Varriount]

I've wrapped protocols that always return success, but put error information in the response body.

Irrelevant. You need to write extra logic to map that to an error then. You don't need an "extensible" error in order to be able to do this.

Mapping only works if the following conditions are met:

• There are values accurate enough to map to.
• The values to map are known ahead of time.

Given the above RFC, the values that one can map to are inherently limited - I have no way of declaring my own additions to the FatalError enum.

Take the following pseudo-code, which reflects some real-world code I have had to write:

import json

proc makeOrGetSubnet(client: Client, cidrBlock: string): Subnet =
  try:
    let
      request = makeRequest(client, {"action": "makeSubnet", "cidr": cidrBlock})
      data = request.json
  except ClientError:
    let exception = getCurrentException()
    if exception.message == "alreadyExists":
      return getSubnet(client, cidrBlock)
    else:
      raise
  # Turn `data` into `Subnet` ...

Below is how the code would need to be rewritten to use the new exception system:

type
  ClientResult[T] = object
  case isError: bool
  of true:
   errorData: JsonNode
 of false:
   res: T

proc makeOrGetSubnet(client: Client, cidrBlock: string): ClientResult[Subnet] =
  # Note - users will have to deal both with IOError through try, and ClientResult
  let request = makeRequest(client, {"action": "makeSubnet", "cidr": cidrBlock})
  if hasKey(request.json, "error"):
    if getErrorMessage(request.json) == "alreadyExists":
      result = getSubnet(client, cidrBlock) # Will also be a ClientResult
    else:
      result = ClientResult(isError: true, errorData: request.json)

Note that the protocol the above client uses has the following characteristics:

• Error codes are added over time (not evident from the code)
• Error data is sent as a JSON body from the request.

What has to be done under the proposed exception system is that any extra information about the error must be sent through a object variant return value (or a thread local variable, etc). The exception system can't be used, because translating ClientError into IOError causes too much loss of information - one can't tell whether an IOError occurred because of a bad connection, or a missing file.
Furthermore, since the Error enum is static, one can't even add an error to the system, such as FileMissingError.

[andreaferretti]

If I understand correctly, this proposal conflates (at least) four concerns:

1. Exceptions are currently ref types, and there is no mechanism to raise stack or fixed size objects
2. Distinguishing which exceptions signal bugs and which ones runtime errors
3. Exceptions currently use setjmp, which is costly
4. Introducing a supervisor mechanism

Maybe it would be easier to treat these issues separately.

As one data point, I welcome 2 (possibly with an additional field, as @dom96 proposed) and 4. I also would like very much to find a solution to 1, but I think some discussion is needed to find a mechanism that is less limited than the one proposed here. Regarding 3, I am not in the position of making informed comments

[Araq]

Sorry, I am a bit lost: if removeFile only lists OSError why do you insist in catching IOError?

Ok, to put it differently, why can existsOrCreateDir raise IOError?

Or take this https://nim-lang.org/docs/asyncdispatch.html#poll,int example. Here is its declaration raises: [Exception, ValueError, OSError, FutureError, IndexError]. IndexError is a programming bug and doesn't have to be caught. It's unclear what a FutureError is. When would ValueError be raised? I doubt I would catch that. And then there is Exception which means Nim's exception tracking had to give up. Here is my guess what I would really need to write in order to catch its errors, but not its bugs:

try:
  poll()
except OSError, IOError:
  echo "error ", getCurrentExceptionMsg()

And that was an educated guess, and I would have guessed the same without Nim's exception tracking mechanism.

And here is what I would write if my RFC becomes a reality:

try: 
  poll()
except:
  echo "error ", getCurrentExceptionMsg()

And it didn't require the guesswork, errors are mapped to exceptions, bugs are not.

[Araq]

What can except: take? Is it limited to FatalError and Error?

It is in fact limited to Error.

Can I stil use try/except/raise to do an intentional stack-unwind (e.g. ForumError in nimforum?)

Yes.

If all exceptions are suffixed with Error, why isn't it called try/error instead of try/except?
Why isn't this called reworking errors instead of reworking exceptions?
Can you implement try/except alongside try/error? (error being this proposal)

You can catch Error with try and you can catch FatalError with shield.

[andreaferretti]

@Araq This is a better example - I agree that the current system does poorly on poll.

[Araq]

Mapping only works if the following conditions are met:
There are values accurate enough to map to.

I think we can come up with a list of errors that is detailed enough.

The values to map are known ahead of time.

We can introduce an UnknownError for this.

[markprocess]

Can I stil use try/except/raise to do an intentional stack-unwind (e.g. ForumError in nimforum?)

Yes

Intentional stack-unwind is by definition not an Error, it is intentional control-flow. What actual value would be used in the raise? One that you listed above? Which one? I had the impression that exceptions were no longer available except for errors.

[markprocess]

UnspecifiedError is more accurate than UnknownError. At the point of raise, it is well-known and understood. It just doesn't provide any more specifics to the except: clause.

[Araq]

Intentional stack-unwind is by definition not an Error, it is intentional control-flow. What actual value would be used in the raise? One that you listed above? Which one?

Some value that makes sense. LoginError if there is a login failure, ProtocolError for an HTTP error, etc. Why should it be wrapped in a ForumError? What the heck is a ForumError? ;-)

[Araq]

Below is how the code would need to be rewritten to use the new exception system:

No, I think you misinterpret my proposal, much of your try except logic would be the same.

[markprocess]

Some value that makes sense. LoginError if there is a login failure, ProtocolError for an HTTP error, etc. Why should it be wrapped in a ForumError? What the heck is a ForumError? ;-)

https://forum.nim-lang.org/t/4044#25243 refactored the ForumError into an intentional CompletionException. The design of ForumError contains functional information, not diagnostic, that is passed back to the browser for presentation to the user.

There needs to be an Error that isn't an error, one that is for an intentional stack-unwind. The older frames on the stack will trap and check their data to see if the unwind was directed to them.

[andreaferretti]

It may be a completely silly idea - but what about a typed try that works with any enum?

The idea would be the following. Say I have a library that likes to define its own silly exception types. Great, let's define this as an enum:

type ForumError = enum
  LoginError, NetworkError, DbConnectionError, InvalidFormattingError

Then I can raise this enum:

proc foo(user: User) =
  if not authenticate(user):
    raise LoginError

In the place where I catch this, I can write

try:
  foo(user)
except LoginError:
 echo "failed login"

The compiler recognizes that I am trying to catch something of type ForumError, so it makes exactly the thing specified by Araq, but using our custom enum type instead of the default Error type.

Pros:

• no additional runtime cost wrt to the proposal of @Araq
• users can use custom errors
• no need to fit errors into hierarchys

Cons:

• cannot catch errors of different types in the same clause (say a LoginError together with a KeyError)

So authors of libraries are kind of forced to choose between:

• reusing the base Error enum
• providing their custom errors, in which case they have to translate all errors coming from lower levels into their own new enum

To me it seems that this mechanism would have the benefits described by Araq without losing extensibility, with the only price of using type inference to detect "different" types of except clauses.

With a little more support from the compiler, one could also raise and catch object variants: proposal is as above, but the compiler only propagates the enum in the discriminator and stores the actual object variant into a thread local variable

[Araq]

It may be a completely silly idea - but what about a typed try that works with any enum?

Ha, I was about to modify my proposal to allow just this. This "everything is not extensible" point is the least important part of the RFC.

providing their custom errors, in which case they have to translate all errors coming from lower levels into their own new enum

And that means the different "layers" in an application are enforced. It's great. (Unless it turns out to be a pita in practice.)

[andreaferretti]

This "everything is not extensible" point is the least important part of the RFC.

But the most controversial one! :-D

[markprocess]

It may be a completely silly idea - but what about a typed try that works with any enum?

This also permits try/except to handle exceptions and not just errors. (This might be at odds with the paper, but useful for intentional stack-unwind.)

[dom96]

@andreaferretti just to be clear, the proposal is to allow enums to be raised in addition to heap-allocated refs?

[andreaferretti]

@dom96 No, my proposal is to follow what Araq is saying, but recovering at least some of the flexibility we have today by allowing any custom enum instead of the hardcoded Error

Another extension would be the following:

• allow to raise any variant object
• the compiler actually propagates the discriminator only, which is of an enum type
• and stores the actual variant into a generated thread local variable

This would recover a lot of benefits:

• one can raise stack objects and discriminate them by just propagating an enum
• the context is not lost
• if one wants to store heap allocated data into the variant object, this is always possible, but this cost is not necessarily paid by everyone

[andreaferretti]

type
  ForumErrorKind = enum
    HttpError, NetworkError
  ForumError = object
    case kind: ForumError
    of HttpError:
      statusCode: int
    of NetworkError:
      discard

proc foo() =
  raise ForumError(kind: HttpError, statusCode: 404)

try:
  foo()
except HttpError:
  let e = getCurrentException() # infers ForumError
  echo e.statusCode

can be translated by the compiler into something like

var ForumErrorCurrentException {.thread.}

proc foo() =
  ForumErrorCurrentException = ForumError(kind: HttpError, statusCode: 404)
  raise HttpError

try:
  foo()
except HttpError:
  let e = ForumErrorCurrentException
  echo e.statusCode

This recovers at least enough functionality to be usable for me and does not require to allocate anything on the heap

This "extended" proposal may even be implementable with macros, even if possibly not in a very convenient syntax

[dom96]

Ahh, that example is much clearer. I'm happy with this, but let's go a step further and just implement what I proposed above: the ability to raise all stack-allocated objects. Even objects that aren't variants. The ForumError which you are using as an example is currently just one kind of error. I don't want to have to create a dummy variant just to appease the compiler.

---

As an aside, are there any valid use cases for an exception hierarchy that we will miss out on by implementing this proposal? If so we should at least discuss them.

[zah]

I see this discussion is already going in the right direction, but I'll provide my two cents.

The main distinction between recoverable errors and bugs is that the former represent situations that are planned for by the developer. A user may have entered incorrect data (potentially malicious one), a network connection may be interrupted, or an important data file may be missing. With such errors, the developer must decide what the policy of the software should be because they are expected to arise even in a perfectly implemented program. Bugs on the other hand are unexpected and there can't be any reasonable code that discriminates between the types of bugs that were encountered.

So, the recoverable errors will be domain-specific and each type of problem is likely to have a different work-around policy implemented in the code. For this reason, the error must be encoded in a precise way and not mapped to a general enum that doesn't allow you to implement these different "work-around policies".

Araq seems to motivated by the efficiency arguments suggested by the C++ paper, but even the paper has a slightly different solution that is still user-extensible. The error type described in the paper has two numeric fields: error_category and error_code. error_category is something that can take values such as "nim std error", "posix error", "windows error", etc. It can also take values such as "ID of user enum E". Through this mechanism and the high-level sugar suggested by @andreaferretti, you can have your cake and eat it too. error_code may also store a pointer in certain situations to regain more of the current exceptions capabilities.

Otherwise, I think we can find ways to keep the support for error types with fields as a high-level syntax sugar. It was already mentioned that an upper limit to the size of the error types may be one solution. You can also allocate memory on the stack just prior to raising the exception (in the C++ paper, it's mentioned that this is the implementation strategy of MSVC on Windows).

How many of you have read my additional proposals for high-level sugar:
https://gist.github.com/zah/d2d729b39d95a1dfedf8183ca35043b3

Some of the ideas are already incorporated here, but if we are planning to break backward compatibility, we can also teach try some new tricks as suggested in the proposal.

[markprocess]

On the road from here to there, can we get a compiler switch that warns or fails on the following:

• Create an Exception that is not a leaf in the hierarchy (all internal nodes are abstract.)

  • In particular an UnspecifiedError would likely be useful because except: Exception is ambiguous (does it mean any exception subtype or does it mean a concrete instance of Exception used in a general way?) The former would then be excluded.

• Likewise except: can only refer to leaves.

• Define an exception type that includes fields (i.e. definitely can't be replaced with an enum.)

We could start to refactor nim in the wild and see where it goes.

[andreaferretti]

let's go a step further and just implement what I proposed above: the ability to raise all stack-allocated objects. Even objects that aren't variants

Fine, but the problem is what one uses at runtime to propagate what actual error happened.

In @Araq proposal, this is just a member of the Error enum.

I proposed to use any enum, but then the try .. except clause will be typed due to this enum. This means one cannot mix members of different enums and catch them in the same clause. If one follows this, the ability to extend it to variant objects is right behind the corner.

Raising any stack object would be fine, but the issue would be: how do you catch it? It could be done if the is just one except clause, in which case there is no problem.

But if there is more than one except clause, the problem is where to store the runtime data to distinguish the various cases. Without using either the heap or variant objects I do not see a good way.

[Araq]

As an aside, are there any valid use cases for an exception hierarchy that we will miss out on by implementing this proposal?

Well depending on how it's done, you lose much of the type propagation properties:

proc dontCareAboutExceptions =
  subsystemThatCanRaiseForumError()
  subsystemThatCanRaiseSystemError()
# --> Error: dontCareAboutExceptions can raise 'ForumError' but also 'SystemError'. 

Of course, this can also be seen as a benefit, it forces 'dontCareAboutExceptions' to either translate from ForumError to 'system.Error' or vice versa.

[Varriount]

@andreaferretti Wouldn't it be possible for the compiler to automatically create an object variant to hold the different values?

[andreaferretti]

I guess it would be possible, but then you would have to deal with a value of this generated type. I am not sure this buys you anything

[cooldome]

@Araq, does new proposed exception handling will still provide stacktraces?

[Araq]

Yeah, but we also already have system.writeStackTrace and system.getStackTrace and that's independent from the exception mechanism. (Yes, I know you need to query the stack before the exception unwinds the stack.)

[Varriount]

@andreaferretti The compiler already knows exactly what kind of errors a procedure can throw.
That means, given 2 enum types a function can throw, the compiler turn this:

try:
  let res = exceptionalProc()
except AValue:
   echo getCurrentException(), " A"
except BValue:
   echo getCurrentException(), " B"

Into, roughly,

let (errVariant, res) = exceptionalProc()
if errVariant.errType == 1 and errVariant.err1Value == AValue:
   echo getCurrentException(), " A"
if errVariant.errType == 2 and errVariant.err2Value == BValue:
   echo getCurrentException(), " B"

The datatype used behind the scenes would be something like

type ErrValue = object
  case errTypeKind: enum # Dynamically generated based on possible thrown types
  of etk1:
    errValue1: enum
  # Repeat for other possible thrown types

[andreaferretti]

@Varriount yeah, that could be done, at the cost of potentially generating a new type for each proc that throws.

That may be more convenient, but declaring your variant type has some advantages too: you are kind of forced to declare all errors that your library can throw, and convert external errors in your type. This is a pattern that is not too bad.

[mratsim]

To complement the distinction between bugs and recoverable errors and @zah post, here are relevant snippets from https://gist.github.com/zah/d2d729b39d95a1dfedf8183ca35043b3 (it should probably be public btw) and the corresponding blog post http://joeduffyblog.com/2016/02/07/the-error-model/

Also this RFC is strongly linke to the strict mode RFC https://github.com/nim-lang/Nim/issues/7826

---

From the blog post

Principles

As we set out on this journey, we called out several requirements of a good Error Model:

• Usable. It must be easy for developers to do the “right” thing in the face of error, almost as if by accident. A friend and colleague famously called this falling into the The Pit of Success. The model should not impose excessive ceremony in order to write idiomatic code. Ideally it is cognitively familiar to our target audience.

• Reliable. The Error Model is the foundation of the entire system’s reliability. We were building an operating system, after all, so reliability was paramount. You might even have accused us as obsessively pursuing extreme levels of it. Our mantra guiding much of the programming model development was “correct by construction.”

• Performant. The common case needs to be extremely fast. That means as close to zero overhead as possible for success paths. Any added costs for failure paths must be entirely “pay-for-play.” And unlike many modern systems that are willing to overly penalize error paths, we had several performance-critical components for which this wasn’t acceptable, so errors had to be reasonably fast too.

• Concurrent. Our entire system was distributed and highly concurrent. This raises concerns that are usually afterthoughts in other Error Models. They needed to be front-and-center in ours.

• Diagnosable. Debugging failures, either interactively or after-the-fact, needs to be productive and easy.

• Composable. At the core, the Error Model is a programming language feature, sitting at the center of a developer’s expression of code. As such, it had to provide familiar orthogonality and composability with other features of the system. Integrating separately authored components had to be natural, reliable, and predictable.

Overview of error models

Bugs Aren’t Recoverable Errors!

A critical distinction we made early on is the difference between recoverable errors and bugs:

• A _recoverable error_ is usually the result of programmatic data validation. Some code has examined the state of the world and deemed the situation unacceptable for progress. Maybe it’s some markup text being parsed, user input from a website, or a transient network connection failure. In these cases, programs are expected to recover. The developer who wrote this code must think about what to do in the event of failure because it will happen in well-constructed programs no matter what you do. The response might be to communicate the situation to an end-user, retry, or abandon the operation entirely, however it is a predictable and, frequently, planned situation, despite being called an “error.”

• A _bug_ is a kind of error the programmer didn’t expect. Inputs weren’t validated correctly, logic was written wrong, or any host of problems have arisen. Such problems often aren’t even detected promptly; it takes a while until “secondary effects” are observed indirectly, at which point significant damage to the program’s state might have occurred. Because the developer didn’t expect this to happen, all bets are off. All data structures reachable by this code are now suspect. And because these problems aren’t necessarily detected promptly, in fact, a whole lot more is suspect. Depending on the isolation guarantees of your language, perhaps the entire process is tainted.

---

[Varriount]

Just a situation that I ran into today, which I think should be kept in mind for whatever system is decided upon: I'm currently writing a command shell (like, bash, cmd.exe, etc). Part of the for statement involves declaring a regular expression, like for <regex> in <command>, so that the output of the command can be split into pieces.

The code looks roughly like this:

var regex: Regex
try:
  regex = toPattern( arguments[0])
except RegexError:
  echo getCurrentException().msg # Prints out the type of error *and* its place within the input

When arguments[0] contains "a[", the following is printed out:

Invalid set. Missing `]`
a[
 ^

Note that in this case, the error contains information about both the class of error, and the place the error occurred.

[Skrylar]

As long as "not much" changes on the written code side (ex. @zah pointing out the category/code numbers, mapped through macros) it seems okay.

@Varriount I would argue you should read data like that from an exception's parameters. Error systems that only supply data via a string (and I'm not accusing Regex of this per se) are horrible.

Arguing that because average programmers don't wield exception types in a semantically pleasing way is the same bogus "average idiots do it wrong so take away everyone's cake" argument that has the macro system suppressed from, well, almost every language.

[nitely]

@Varriount I would argue you should read data like that from an exception's parameters.

Well, I guess that can be useful for presenting the data in other formats. That does not mean the message should be removed from the traceback. Having a custom error within the traceback is beyond useful, I do that everywhere. Handling custom error messages in a different way for every library wouldn't be pretty, to say the least. Most users consuming an API just want the error to be shown within the traceback. I don't want to deal with "exception's parameters" anywhere (not in code I write nor in code I use), that's an edge case.

[rayman22201]

I finally had some time to read the C++ proposal in more detail. I have some thoughts.

The C++ proposal allows both old style ref exceptions and new style variant exceptions to coexist.
In the paper this is done by wrapping blocks that use variant exceptions with a "throws" keyword.

A similar thing could be implemented with Nim pragmas to identify functions that throw variant exceptions.

In the C++ proposal, catch can accept both types of exceptions, and can transparently convert from one type to the other. We could do something similar in Nim with converters.

Using the effect system we could track which functions throw which type of exception, and annotate the catching function with what type of exception it is willing to catch. Or perhaps also globally with a compiler flag?

The major cost of this from a language standpoint is that catch now has to be much smarter, as it has to handle two different classes of exceptions.

This results in two classes of exceptions, which isn't ideal, but there is some precedent for allowing multiple systems to co-exist: the multiple GC options and the destructor system, and more directly related the {.gcsafe.} pragma. I realize this is not an "apples to apples" comparison though.

[rayman22201]

Regarding the discussion between @andreaferretti and @Varriount about the combinatorial explosion of variants.
I am personally a big fan of @andreaferretti's proposal.
It allows for a large amount of expressiveness in error handling, while still achieving the desired efficiency properties.

There is also strong arguments that forcing a library author to handle lower level exceptions and translate them into more contextually useful exceptions as you go higher up the stack is a good practice.
Currently wide spread exception systems don't encourage this behavior, and it causes real world issues.
@see this quote from section 4.1.9 of the C++ proposal:

Propagating arbitrarily typed exceptions is not composable, and is often lossy.
It is already true that intermediate code should translate lower-level exception types to higher-level types as the exception moves across a semantic boundary where the error reporting type changes, so as to preserve correct semantic meaning at each level. In practice, however, programmers almost never do that consistently, and this has been observed in every language with typed propagated exceptions that I know of, including Java, C#, and Objective-C (see [Squires 2017] for entertaining discussion). This proposal actively supports and automates this existing best practice by embracing throwing values that can be propagated with less loss of specific information....

Propagating arbitrarily typed exceptions breaks encapsulation by leaking implementation detail types from lower levels. (“Huh? What’s a bsafe::invalid_key? All I did was call PrintText!”) As a direct result...

That being said, maybe there is a middle ground between libraries having to manually define and translate all error variants and the compiler auto-generating a combined variant error type that is a giant Frankenstein combination of all possible error types.

Perhaps a macro could be used to generate an error type that combines the error types the programmer knows they want to throw. A pseudo-automated approach.
@see straw man example below:

  myLibraryError* = enum
    errA,
    errB

type combinedError* = combinedEnum
    myLibraryError
    stdLibError,
    someOtherLibraryError

This is more work from the library author, but less work than having to manually translate all the error types across module boundries. It also prevents the problem of a bunch of "similar but not quite the same" error types that @Varriount raised.

In theory this looks like a more labor intensive equivalent to the completely automated approach (the compiler auto-generating a bunch of combinatorial error types).

I predict that in practice the library author knows better than the compiler what exceptions they want to handle / throw: This is a programmer guided manual pruning of the possible error combinations.

If the program throws an exception that the library author didn't expect, that is arguably a bug, and should fail fast inside the library code anyway. The compiler could even detect and error if the program doesn't handle all the necessary variant types.

I'm not sure if this idea is good or not, I'm throwing it out there for consideration and critique :-)

[Bulat-Ziganshin]

straw man example below

your example just mimics hierarchy of exception classes - technique that is supported by all major languages (C++, Java, C#, Python), but not supported directly by errcodes approach as implemented in C/Rust/Go/Zig

[rayman22201]

@Bulat-Ziganshin

your example just mimics hierarchy of exception classes

That is exactly the point.
Many of the concerns raised on this thread are about keeping the exception hierarchy with this new system. I'm attempting to find a compromise that gives errcode like performance while also letting people have exception hierarchies.

[Bulat-Ziganshin]

I'm attempting to find a compromise that gives errcode like performance while also letting people have exception hierarchies.

But then why you make this citation? It says that the exception hierarchy is broken, not what it's great and we want to find a cheap way to emulate it, in particular:

There is also strong arguments that forcing a library author to handle lower level exceptions and translate them into more contextually useful exceptions as you go higher up the stack is a good practice. Currently wide spread exception systems don't encourage this behavior,

What this mean? Exception classes allows you to make hierarchies, but they cannot force you. The same is for errcode approach - you can develop an hierarchy of errcodes, but you can't force all users to do it well. So, what's the difference?

So, please make a choice:
[ ] exception class hierarchy doesn't have some features you are proposing
[ ] you propose cheaper implementation that provides some of the features of the class hierarchy

But please don't mix arguments of the first with proposal of the second. You idea doesn't give us any benefits OVER class hierarchy, so the arguments you cited can't be used to support it.

[rayman22201]

You are forcing a false dichotomy. It is not one or the other.
This proposal does have benefits over the class hierarchy. It is much cheaper! That is a very real benefit.

so the arguments you cited can't be used to support it.

You are not understanding my argument. I am personally in favor of errcode or variant style error handling. Which is what I argue at the top of the comment.
Immediately below that I say:

That being said, maybe there is a middle ground...

In other words. I prefer variant type style, but many in this thread disagree and I am attempting to design a compromise.

The referenced C++ paper, gives similar arguments around a compromise, which is why they propose the system of two different exception types.

This proposal is not solely mine. It has been discussed by @Araq, @andreaferretti and @Varriount, and I reference their comments.

My contributions are specifically the idea of allowing both types of exceptions, and making the auto-generation of variant types as proposed by @Varriount more explicit. You are calling me out as if I made some radical departure from the discussion.

I argue that variant type exceptions are better for runtime performance and arguably better for library design, and cite arguments to support that claim.

I then acknowledge the earlier discussion that provide arguments against such a system, and attempt to provide a compromise.

[Bulat-Ziganshin]

Sorry, I don't mean that you differ from Araq proposal, or his arguments. I just can't sum up together everything already said, so I argued over your concrete post. If that contradicts Araq's arguments too, it's OK for me.

This proposal does have benefits over the class hierarchy. It is much cheaper!

Yes, here I agree. You can say that errcode approach is much cheaper. It's OK.

But if you think that it has other benefits, please give me details of these benefits. In particular, I don't think that errcodes will improve over classes in "encouraging" exception hierarchies. Do you agree with me here?

PS: Yes, I'm against the Araq proposal. I just try to dissolve my arguments into smaller, discrete pieces. I will be glad to see answers about "other benefits" from everyone, including Araq.

PPS: yeah, I'm going to read the C++ paper...

[rayman22201]

But if you think that it has other benefits, please give me details of these benefits. In particular, I don't think that errcodes will improve over classes in "encouraging" exception hierarchies. Do you agree with me here?

I agree with you. It does not improve over exception hierarchies.

I personally think exception hierarchies are not as useful as people say they are, which is why I am in favor of Araq's proposal. Exceptions are crap, let me have something simpler with better performance so that I can write a Nim kernel module and actually use some std lib functions and types. But I know I am in the minority on this.

If we can design a solution that is equivalent (or at least close feature parity) to exception hierarchies (the current state of Nim), but has better performance, shouldn't we do that?

No matter what your opinion is, the paper is very good. I definitely recommend it! 😄

[dom96]

To be clear: I'm largely happy with this proposal, but not for Nim v1.0. I know that at least @rayman22201 doesn't think this is being suggested for Nim pre-v1.0 but as far as I know it is. This is why I am pushing back against this.

[Bulat-Ziganshin]

For me, the main giveaway from the C++ paper is the following citation:

The primary design goal is conceptual integrity [Brooks "Mythical man-month"], which means that the design is coherent and reliably does what the user expects it to do. Conceptual integrity’s major supporting principles are:
• Be consistent: Don’t make similar things different, including in spelling, behavior, or capability. Don’t make different things appear similar when they have different behavior or capability.
• Be orthogonal: Avoid arbitrary coupling. Let features be used freely in combination.
• Be general: Don’t restrict what is inherent. Don’t arbitrarily restrict a complete set of uses. Avoid special cases and partial features.

We must use those as principles for the Nim development.

[endragor]

TL;DR I agree with where this and #7826 RFCs go.

I recently added a bit of manually-managed data structures into a relatively complex multi-threaded framework, and faced plenty of problems with exceptions, so it's nice to see there are RFCs to address these problems. Even when not dealing with manual memory allocations/deallocations, I feel exceptions do a lot of harm in absence of RAII semantics, since every line of code you write may raise, and you have to make sure it never leaves you up in an inconsistent state (simple examples: having a mutex locked or a counter not decremented). For simple cases inserting try/finally or defer helps, but is still something you have to keep in mind and something that affects performance because of setjmp. For complex cases defer may not work as you may want to unlock/free a resource conditionally depending on what happens within the procedure. I noticed libraries that accept callbacks sometimes do not consider that the callbacks may raise and end up in a bad state if they do. This is expected, since it's easy to forget about handling exceptions when the compiler doesn't force you to.

As a solution, you can try adding {.raises: []} to procedures where it is critical, but with the current state of Nim and the stdlib it will turn your code into a mess, because you'd have to insert try/except everywhere, as even array access may raise IndexError. Even if you call alloc with --gc:none, then you suddenly have to add try/except, too, since alloc may call user-defined outOfMemHook and it may potentially raise, so effects system asks you to handle that.

Overall, I support the idea of making writing {.raises: [].} code easier, which this RFC would help a lot. However, one very useful property of exceptions is that you can see where certain problem occurred. This is almost always needed in debug mode, and in production mode, too, unless you cannot allocate space to keep the stack trace in (the embedded use case?). So if Nim goes with the RFC, a must-have is a simple built-in way to obtain stack traces of exceptions (this isn't directly mentioned in the original post, but @Araq confirmed above this will be added).

Ideally, {.raises: []} should be the default, and it's not hard to enforce that if FatalErrors just kill the program, reporting where the error occurred. The rest would be conscious decisions made by the programmer. It would help:
1) Ensure that if something within your procedure raises, you will know about it and do something about it - either handle the exception or add raises pragma to the proc.
2) Reduce possibility that a procedure may raise a whole bunch of unclear exceptions, like asyncdispatch.poll does now (raises: [Exception, ValueError, OSError, FutureError, IndexError]). When a programmer sees this, he or she would be encouraged to instead provide more clear exceptions and document them.

The downside of above is that it will make Nim less ideal for quick prototyping. The solution suggested in #7826 seems to provide good of both worlds - if you care about robustness in your project, you could use func everywhere. If you need to throw something together fast, just use proc.

[dom96]

Ideally, {.raises: []} should be the default, and it's not hard to enforce that if FatalErrors just kill the program, reporting where the error occurred.

This is how it works in Java AFAIK and I think that's the reason people hate checked exceptions.

[endragor]

People may hate being forced to be disciplined, but it pays off when you develop something big that needs to be robust. The only two robust ways of handling (recoverable) errors I know are checked exceptions and returning errors, e.g. Result[T], like it's usually done in functional languages. But Result[T] doesn't work well without algebraic types and pattern matching, since you may still potentially access the value without checking for error.

You usually don't care about robustness when you are throwing together a quick prototype, and that's what my last paragraph was about.

I programmed on Java in the past, and checked exceptions were definitely a good thing for anything over 1k LoC - as I described earlier, they resulted in cleaner APIs where you know what to expect and make sure you handle errors. What most people hate about checked exceptions, as I understand, is that if you want to write a simple program that, say, counts lines in a file, you'll have to add exception handling boilerplate, while in Nim you can do it in 1 line: echo readFile("file.txt").splitLines().len.

The problem is that by allowing non-robust way and keeping it the default, Nim ensures that most of the ecosystem will be written that way. I'm sure that if Java didn't have checked exceptions, the code created with it would be much worse than it already is.

[Araq]

Java's mistake is that the default in interface declarations (and everywhere else, but it's most problematic for interfaces) is "throws nothing" and so people are forced to handle the error where they cannot handle it so they "log" the error away and everything depends on a logging framework. Alternatively unchecked exceptions are used. We can learn from this mistake and still have a more disciplined approach to error handling.

[drslump]

Really interesting discussion guys. After reading it and the related material I think that @Araq proposal can be tuned to gain a lot of benefits as others have pointed out, and not only solve the technical problem of supporting embedded systems.

Personally I think a key concept for handling errors in a way that scales from rapid prototyping to complex projects is to ensure that they are properly encapsulated at clear boundaries.

In the case of Nim there is no concept of packages so the only way I see as a natural boundary would be module exports, where each function exported must handle recoverable errors from the stuff it uses and only raise or forward its own error definitions (and perhaps a very well known set of core ones). That means that when prototyping you don't need to care but if you're creating a module then it's your responsibility to handle errors, it's just not fair to only do the funny part of implementing the success branch and push boring error handling to the module user 😄

I'm not familiar enough with Nim though to know with any certainty if a module export is a broad enough boundary for this to work right. It might be a bit of a pain if you want to refactor a library into separate modules for example. Perhaps that could be solved with a pragma that makes that module "private"?

My point though is that once the locality of the error handling is enforced, exception class hierarchies are seldom required, you can use a module and know exactly what errors you can expect from it, instead of some generic catch all that basically logs the error and wishes the program can continue running by sheer luck.