Problem-solving: Expressing the idea that a function successfully returned nothing is difficult

[codesections]

Most functions in Raku return values, but a few don't. In for those functions, we have Nil and its subtype, Failure. Here are a few examples of functions that return something that .isa(Nil):

#`(1) sleep(0).isa(Nil);
#`(2) ‘foo’.index('b').isa(Nil);
#`(3) ‘’.unival.isa(Nil);
#`(4) sub f {}; &f.WHY.isa(Nil);
#`(5) (+'tree').isa(Nil);
#`(6) dd.isa(Nil);
#`(7) EVAL('42', :check).isa(Nil);

In each case, the function does not return a value, and so returning the special value Nil to signal an absence is 100% appropriate. However, even though Nil always means "there wasn't a value", these functions really fall into three different categories:

• Nil signals an error – we didn't get a value, but should have (ex. 5)
• Nil signals successful completion – just like on the command line, no news is good news (ex.s 1, 6, and 7)
• Nil doesn't signals success or failure, it just means "no value found". (ex.s 2, 3, and 4)

I could have given more error examples, but I didn't bother because Raku already has Failure to distinguish between that category from the others. However, we _don't_ currently have a good way to distinguish between the other two categories of Nil. I believe that having a way to do so would help make our type system both slightly easier to understand for new users and slightly more powerful for more experienced users.

[codesections]

My proposal for addressing this issue is very simple: Add a new Ok or Success type that is the exact inverse of a Failure – that is, it is a subtype of Nil but signals successful completion of an operation. A minimal implementation of this type could be as simple as:

class Ok is Nil { 
    method Bool(--> True) {} 
    method gist() { self.^name } 
}

Advantages

Adding this class (and, where appropriate, changing return values) would have two advantages:

More readable code, (especially in combination with try or other Failures)

For example, both of these snippets would become possible:

#`( perform actions) if try EVAL($foo, :check); 

and

my %h is BagHash;
(try %h.add: +$str ) ?? #`(handle success) !! #`(handle failure);

More self-documenting method/subroutine signatures

Right now, many functions have signatures that look a lot like

sub sleep($seconds = Inf --> Nil)

That's _fairly_ obvious, once you think a bit about what Nil must mean in that context. But (imo), the following would be much clearer, especially to new users:

sub sleep($seconds = Inf --> Ok)

This difference may seem – and, indeed, be – pretty minor in the context of simple subs like sleep. However, in more complex function signatures (which will be increasingly prevalent as Raku begins to be used in production for larger and larger projects), the added clarity of returning Ok instead of Nil seems like it would be even more helpful.

Potential downsides

The only downside I can think of is that this change adds one more core type, and thus slightly increases the complexity of the Raku type system, which risks making the language more difficult to learn. (Thanks to @lizmat for pointing this issue out on IRC). However, I believe that two factors mitigate against this consideration. First, Raku currently has a very rich type system with over 300 built-in types; adding one more with a self-explanatory name does not seem likely to increase the complexity significantly.

Second and more importantly, splitting out a separate Ok class could _remove_ a significant source of confusion for new Rakoons. I have observed that many new Rakoons think that Nil always indicates a failure. Indeed, the docs (correctly!) note that "[a Nil] may also be used as a cheaper and less explosive alternative to a Failure" and (correct!) statements like "Nil is really a type of generic soft failure" are pretty common. While these statements are, as noted, correct, they could confuse users into thinking that a Nil always represents a Failure or at least an unexpectedly missing value. Adding an Ok (or similar) type could help avoid this confusion and, hopefully, simplify learning Raku.

Past discussions

I previously discussed this suggestion on the #raku-dev channel. I tried to summarize the points raised in that discussion, but the logs are linked above if you want the full discussion.

[vrurg]

This needs some careful thinking. So far, not sure if I really like it, even though I well understand the reasoning behind it.

The first thing which would confuse me is a type object being True. Correct me if I'm wrong, but so far we have no class being True without instantiation. That's definitely will be a confusion for a newby.

Aside of that, semantics of Ok will put additional burden on optimizer because it will have to treat it same way as Nil in many but not all situations.

What is related to commonplace meaning of Nil, I wonder why people choose the second part of its description over the first part? I mean _Absence of a value or a benign failure_ from https://docs.raku.org/type/Nil.

[FCO]

For returning nothing I usually use Empty, not Nil…

Sent from my iPhone

On 8 Mar 2021, at 21:57, Vadim Belman [e-mail] wrote:


This needs some careful thinking. So far, not sure if I really like it, even though I well understand the reasoning behind it.

The first thing which would confuse me is a type object being True. Correct me if I'm wrong, but so far we have no class being True without instantiation. That's definitely will be a confusion for a newby.

Aside of that, semantics of Ok will put additional burden on optimizer because it will have to treat it same way as Nil in many but not all situations.

What is related to commonplace meaning of Nil, I wonder why people choose the second part of its description over the first part? I mean Absence of a value or a benign failure from https://docs.raku.org/type/Nil.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[vrurg]

For returning nothing I usually use Empty, not Nil…

Empty is a positional (Slip) which doesn't make it reasonable for non-positional return types:

sub foo(--> Int) { Empty };
foo; # Type check failed for return value; expected Int but got Slip (Empty)

[codesections]

The first thing which would confuse me is a type object being True. Correct me if I'm wrong, but so far we have no class being True without instantiation. That's definitely will be a confusion for a newby.

That's a good point, and one I hadn't considered.

Ok, sight revision to my Ok suggestion, which has the added benefit of more closely paralleling fail/Failure:

class Ok is Nil {
    method new() { self.bless }
    method Bool(Ok:D:--> True) {}
    method gist() { self.^name }
}

sub ok { Ok.new }

With this tweak, Ok is falsy, just like every other type class. Ok.new returns a truthy value, but it's not the only .new method to do so. And just like it's rare to call Failure.new() instead of fail, the intention would be that most users would call ok rather than Ok.new() – and, as an independent routine, there'd be absolutely nothing odd about ok returning a truthy value.

I actually like this setup better than my original suggestion, even aside from solving the issue you brought up; as I said, it's a closer parallel to fail. So thanks for bringing that up! (I suppose we could go even further in paralleling Failures, and let Ok take a payload. This would basically turn Ok into a success wrapper type (like Haskell's Just or Rust's, um, Ok). I don't think we should go that far, but thought I'd mention it as a possibility.)

Ok will put additional burden on optimizer because it will have to treat it same way as Nil in many but not all situations.

I don't know the optimizer all that well, so I'll defer to you on that one. But whenever Ok isn't in a boolean context, it _can_ be treated like a Nil, so that doesn't seem _too_ horrible. But, in any event, I'd put all that into the "torture the implementer" category (words I may come to regret, as I shift into doing more implementation work…)

[tony-o]

What’s the issue with just changing the return types of those?

+str should return a not a number failure
sleep should return the number of seconds slept
EVAL should return the last value like a block

having a type that’s the exact opposite of failure without more meaning can already be expressed using what’s there and a !.

[codesections]

EVAL should return the last value like a block

Oops, that was a typo on my part (now fixed). EVAL($str) _does_ return the last value; it's EVAL($str, :check) that returns Nil, to indicate that it checked the syntax without encountering any errors.

+str should return a not a number failure

It does. That was my example of a Failure (but Failures are a subspecies of Nil, which is the same treatment I was sugesting for Ok)

[vrurg]

I think @tony-o approach works best here. The concept of _meaningful nothing_ really sounds weird and over-complicates things. The need to instantiate Ok to get true out of it – why not just return True then? Perhaps there is a need to re-consider return values of some of the core subs and methods. But otherwise I'm personally still not convinced that Ok will bring more profit than harm.

As to the return values – EVAL(:check) returning True worth considering. Luckily, it's not specced yet.

And, BTW, ok is already occupied by Test and we better don't touch it. ;)

[FCO]

I meant when not specifying the return type

On 9 Mar 2021, at 00:40, Vadim Belman [e-mail] wrote:

For returning nothing I usually use Empty, not Nil…

Empty is a positional (Slip) which doesn't make it reasonable for non-positional return types:

sub foo(--> Int) { Empty };
foo; # Type check failed for return value; expected Int but got Slip (Empty)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/Raku/problem-solving/issues/275#issuecomment-793206202, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAYL6UAM47OSK3NQVYLQJ3TCVVBRANCNFSM4YZ62IKA.

[JJ]

I think this is similar to type None, used in Haskell and other languages,
right?

[niner]

The whole idea of an Ok or Success return value seems to be predicated on the
assumption that it's actually necessary to check every single method or
function call for success or failure. I don't see that assumption holding. If
I call a function, I actually assume that it will do what it ought to do. If
something really unexpected happens and it cannot perform its job, I expect it
to throw an exception.

Now exceptions are sometimes a bit cumbersome to use and there are cases when
exceptions are more likely than "it would be really odd if this happened but
that's life I guess", e.g. IO failing due to permission errors or the like.
For these cases we have Failure which can be checked for and defused by simple
boolification as would happen in an if clause or boolean chain.

That should really cover all cases. Assume success, catch exceptions if
there's a good way to handle them and use Failures when failure is an option
but you still need to ensure that failures get handled somehow.

Because that's the whole point: to not silently ignore failing calls and have
something else blow up later when it's much harder to discern what happened.
An explicit "Success" return type would not help with that.

What else could it help with? Distinguishing between "did produce a value" and
"could not produce a value" cases was brought up. But we already have a
perfectly fine mechanism for that: defined vs. undefined. And conveniently Nil
does become an undefined value of the appropriate type when assigned.

As to the examples that a Success type would help with:

I fear that EVAL(:check) is just not a well designed API anyway. A function or
method should have one and only one purpose and complete exactly one task.
With the :check argument, EVAL's purpose changes completely. Instead of
executing some code and returning what the code evaluates to, it now checks
the code for syntactical and grammatical correctness and returns nothing but
throws an exception if the code is not correct.

I know that the implementation of both is quite similar, but that does not say
anything about whether it's good API design to have them share a name. I
completely understand the temptation to give the function additional
functionality with such minor tweaks. Unfortunately that also leads to the
second issue besides having a function called EVAL that doesn't actually
always evaluate: we have a function that throws exceptions in normal use.

The fact that someone would want to actively check some code already implies
that it's to be expected that not all code would pass this check (otherwise an
exception during a normal EVAL would be enough to handle the...well
exception). But exceptions are not meant to be thrown during normal
processing. They are as the name states quite explicitly for the proverbial
exception to the rule. As we'd expect EVAL(:check) to report errors in the
checked code, the proper way to do so would be a return value. So it should be
EVAL(:check --> Bool) instead of EVAL(:check --> Any).

Fixing the inappropriate use of exceptions as "return values" directly exposes
the design issue: that an argument of the function changes the function's
signature. That's confusing and clearly not good design. Of course in Raku we
have a kind of escape hatch with multis, so we could have a multi EVAL($code,
:$check! --> Bool) candidate, but that still smells and at that point we could
as well give it a different name.

So long story short, tony-o is right that this is not really an example of the
need for a Success type but for fixing a problem with the function itself.

I have to admit that I don't really understand the second example.

my %h is BagHash;
(try %h.add: +$str ) ?? #`(handle success) !! #`(handle failure);

This is already perfectly possible with Failures or exceptions and that seems
to be what this example presumes (otherwise why have the try there?):

rakudo -e '(try BagHash.new.add: +"f") ?? say "success" !! say "failure"'
failure

It seems to me that instead of adding yet more ways to distinguish between
successful and failing operations, we would benefit of good documentation of
the options we already have and where they are appropriate.

[codesections]

I call a function, I actually assume that it will do what it ought to do. If something really unexpected happens and it cannot perform its job, I expect it to throw an exception.

I agree with this.

But sometimes what a function ought to to is signal the absence of a value that could have been present. E.g., when I run 'foo'.index('b'), the Nil it returns is _not_ signalling an error; it's signalling that it looked for "b", and successfully determined that it's absent from the string "foo". That Nil carries information: a value that might have been present was in fact absent.

Conversely, the Nils returned by, e.g., sleep, BagHash.add, BagHash.remove, SetHash.set, SetHash.unset, and dd do _not_ carry any information (other that the function did what it was supposed to which, as you say, is already conveyed by the lack of an exception.) There's no value that could have been present but wasn't; the function just returned. I believe that we ought to have some way to distinguish the absent-value Nil from the successful-completion Nil.

As a note, sub f() {}; dd &f.WHY prints Nil+{Mu::Suggestion[Str]}, which points to a different way we could distinguish between these two types of Nils: we could have the Nils that don't signal the absence of a value return Nil but Ok, or Nil but Expected, or even just Nil but True.

---

In any event, this issue has now gotten replies from several people, and approximately zero of them seem to agree that this change (or one along these lines) is worth pursuing. I believe it's fair to say that no consensus has emerged in favor of this solution :D

I'll leave this issue open for a bit longer, in case this response changes anyone's mind / someone new adds that they feel strongly that the issue should be addressed. Assuming neither of those happen, I'll close this and see whether I can find specific APIs to improve to address the same underlying issue (especial ones like EVAL(:check) that aren't spec'ed in their current implementation. has changed anyone

[lizmat]

FWIW, I take --> Nil to be the Raku equivalent of C's void.