Linter: Lint for undocumented exceptions.

[bsutton]

The dart language supports exceptions but not checked exceptions.

The lack of check exceptions means that developer must rely on documentation to determine what exceptions a method will throw.

If you look at a significant portions of code published on pub.dev, developers are simply not documenting the exceptions that are thrown.
Documenting exceptions is particularly error prone as it requires the developer to explore each called api to determine the set of exceptions the api throws.

Of course given that most api's are not documenting their exceptions the result is that a developer must examine thousands of lines of code to determine what exceptions are being thrown.
This of course results in the developer not documenting their own api as its simply too much effort and worse they simply are not doing the necessary work to handle exceptions appropriately.
The lack of documentation then cascades up into the next level of code and so on.

The end result is that much of the code published on pub.dev is of questionable quality.

This problem could largely be resolved by providing a lint rule for pedantic that warns if an api throws an exception.

By adding this link to pedantic we would see a very rapid improvement on the quality of code on pub.dev for the following reasons.
1) as a user of a pub.dev package I can see what exceptions are being thrown and handle them correctly.
2) as a publisher on pub.dev its now easy to document my exceptions as the linter generates a list of thrown exceptions.
3) if I make a code change which changes the set of thrown exceptions the linter will notify my to fix the doco.
4) ide's can be modified to provide a 'quick fix' for this new lint rule which will make keeping doco up to date simple.
5) with exceptions now documented developers will be force to actually consider what exceptions are going to be thrown and take appropriate action rather than just ignoring them as happens now.
6) when third party projects change the set of thrown exceptions (as happened recently with firebase) I get a lint rather than having to wait for a runtime exception.

Essentially we end up with a cascade affect and everyone benefits.

Here is an example of how I would see the workflow.

/// Throws NotARealException
void dowork()
{
   actuallyDoTheWork();
}

void actuallyDoTheWork()
{
   throws PosixException();
}

Linter:

The method dowork throws an non-existant exception NotARealException. Please remove.
The method actuallyDoTheWork throws PosixException. Please document the throws statement.

Once I correct the above lints I should then see:

void dowork()
{
   actuallyDoTheWork();
}

/// Throws PosixException when something goes wrong.
void actuallyDoTheWork()
{
   throws PosixException();
}

Lint error:

The method dowork throws PosixException. Please document the throws statement.

I see two alternatives for how the linter inspects the code.

1) the linter just looks at the body of a method for any direct exceptions and the documentation of all invoked methods (as per the above example)
2) the linter looks at the body of the method and the body of any direct api calls.

The 2nd alternative would see a much faster improvement in code on pub.dev as it would:
1) allow users of a library to at least see any 1st level exceptions third party packages throw
2) would encourage developers to raise issues against the third party project to improve their exception documentation.

The result is that the lint would essentially cascade both 'up' into the developers own code and 'down' into third party projects resulting in an improvement in quality of all dart code.

This lint looks (in my ignorant opinion) fairly easy to implement.

The benefits to the community however will be huge.

[bwilkerson]

Thanks for the suggestion.

If I remember correctly, the original rationale for not supporting checked exceptions was a belief that having any unchecked exceptions effectively negates the value of supporting checked exceptions, and the assumption that unchecked exceptions were unavoidable. I tend to agree, but I mention it not to start a discussion about the value of checked exceptions but because I believe a similar principle is at work here and should be considered.

If we did implement a lint for this, then the lint would only be effective if all of the packages that a given package depends on have also enabled the lint. Packages whose authors did not choose to enable the lint have effectively turned all exceptions into unchecked exceptions. The fact that it requires wide-spread adoption of the lint to give the lint value really does significantly diminish the value of having the lint.

I suspect that you realize that and that that's why you suggested adding the lint to pedantic: as a forcing function for adoption. Unfortunately, adding it to pedantic isn't a trivial process. The pedantic rule set is the set of lints that are used by internal developers at Google. In order to add a lint to that set we'd first need to convince internal developers to adopt the lint, and that doesn't seem like a likely event. So, I don't think we can force adoption that way. Adoption would need to happen more organically.

In the absence of widespread demand for the lint, I don't see much value in adding it. I'll leave this issue open for now so that community members can voice their opinions; if there's enough demand then we'd certainly consider adding it.

[bsutton]

My first statement would be that, regardless of the checked vs unchecked
debate, documentation on what exceptions an api throws would help improve
code quality and make the life of many developers easier.

`If we did implement a lint for this, then the lint would only be effective
if all of the packages that a given package depends'

I wouldn't agree with this statement.

Whilst the mechanism would certainly work better if dependencies
implemented it, the lint would provide intrinsic value without requiring
other packages to use it.

If we could implement my preferred recommendation (look inside the first
level of called code) then we get a certain level of documentation being
'pulled' from the dependencies even if the package doesn't implement the
lint.

As a package developer who wants to provide quality documentation the lint
would make the process of documenting exceptions easier.
I would suggest that the core reason exceptions are not currently
documented is because it's hard to do so.
This lint would essentially automate the hardest piece
of documenting exceptions (discovering what exceptions are thrown).

Whilst the inclusion in pedantic would be ideal I understand that it might
be difficult to get the google team to adopt it internally.

This however doesn't undermine the intrinsic value of the lint and I would
expect over time that anyone interested in generating quality code would
adopt the lint.

Discovering what exceptions are thrown is a constant friction point when
coding in dart so I believe that developers would be happy to adopt it as
it solves a daily problem with their own libraries.

We could also encourage the adoptions by adding it to effective dart.

The core tenant in the lint system is that it should improve code quality,
I would suggest that this lint will have a significant impact on code
quality.

I would ask that you reconsider this lint.

On Wed, 23 Sep 2020 at 01:31, Brian Wilkerson [e-mail]
wrote:

Thanks for the suggestion.

If I remember correctly, the original rationale for not supporting checked
exceptions was a belief that having any unchecked exceptions effectively
negates the value of supporting checked exceptions, and the assumption that
unchecked exceptions were unavoidable. I tend to agree, but I mention it
not to start a discussion about the value of checked exceptions but because
I believe a similar principle is at work here and should be considered.

If we did implement a lint for this, then the lint would only be effective
if all of the packages that a given package depends on have also enabled
the lint. Packages whose authors did not choose to enable the lint have
effectively turned all exceptions into unchecked exceptions. The fact that
it requires wide-spread adoption of the lint to give the lint value really
does significantly diminish the value of having the lint.

I suspect that you realize that and that that's why you suggested adding
the lint to pedantic: as a forcing function for adoption. Unfortunately,
adding it to pedantic isn't a trivial process. The pedantic rule set is
the set of lints that are used by internal developers at Google. In order
to add a lint to that set we'd first need to convince internal developers
to adopt the lint, and that doesn't seem like a likely event. So, I don't
think we can force adoption that way. Adoption would need to happen more
organically.

In the absence of widespread demand for the lint, I don't see much value
in adding it. I'll leave this issue open for now so that community members
can voice their opinions; if there's enough demand then we'd certainly
consider adding it.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/linter/issues/2246#issuecomment-696798219,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAG32OEZHTQFAJSC3DR3BGTSHC7OVANCNFSM4RVC6GPQ
.

[pq]

Thanks for the thoughtful conversation!

I see where Brian is going with the potential limitations if such an analysis were not universally enforced but I'm pretty compelled by @bsutton's observations and the general motivation. If nothing else, it seems like this could improve package documentation.

I'm in favor.

👍

[lrhn]

If the analyzer can actually analyze which methods throw which exceptions, then it would be possible to only warn about undocumented exceptions on API boundaries.

• If your package exposes a method or function which might throw FooException, but isn't documented as such (it's immediate or inherited documentation doesn't have a paragraph starting with Throws which contains [FooException] or FooException), then the lint would warn.
• If your package uses a method or function from another package which might throw a FooException, and that method doesn't document it, even if they haven't enabled the lint, we can give a hint at the use-site that this method might throw a FooException, at least until it's handled in some way.
• If a virtual method implementation throws an exception, and it's super-interface signature isn't documented as doing the same, then we should warn that it's a potentially incompatible override. Someone calling the method at the super-interface type won't know that there might be an exception.

I wouldn't worry about code that is internal to a package, only documentation on API boundaries is really important.
Otherwise we force people to add documentation to internal implementation details, where the author is the only one using it anyway. That rabbit hole is what makes people dislike Java's checked exceptions (or to catch and rethrow as unchecked to make it through your helper functions, then maybe recatch and rethrow as checked again at the API boundary).
Alternatively, have a second lint for internal consistency as well, for people who want to document all their internal functions.

The big problem here is obviously correctly detecting which exceptions can be thrown. Static functions are probably reasonably easy, but virtual methods are harder. With the third item above, it might be sufficient to look at the actual interface documentation. It's too much of a false-positive generator to claim that an interface method throws an exception just because one subclass somewhere does, that one class might just be for testing and won't ever affect the real program.

This can work for async functions too, but it's going to be hard to analyze direct uses of Future.error.

It would require analyzing the entire program, to follow call chains to the end, not just looking at signatures and documentation of what you are calling. If that's too much, then it's true that it only works for exceptions that are already documented as throwing.

[bsutton]

I would want to document internal method but can understand that might not be everyone's preference (but I still get to think they are wrong:)

[OlegAlexander]

Perhaps the lint for internal functions can be optional for those who want it. But the lint for public APIs should arguably be on by default.

[natebosch]

Overall I do expect this would be infeasible to handle for a sufficient number of cases as to make it trustworthy.

• If a method is documented as /// Throws FooException if the Bar is not initialized;, but I know I'm initializing Bar before the call, how do I prove to the analyzer that I don't need to document FooException as an exception my method might throw?
• How would we treat something in a method body like doSomething(() { mightThrowException(); });. Do I need to document the same exceptions or not?

Adding more in depth structure to DartDoc would be a departure from our current idioms and user expectations. This is still pretty readable, but I would strongly oppose any solutions which lead us towards @throws or something similar.
https://dart.dev/guides/language/effective-dart/documentation#do-use-prose-to-explain-parameters-return-values-and-exceptions

The transitive behavior here has potential implications for evolving packages in the ecosystem - the dependencies present when we analyze a package to produce the lints may not match the dependencies present for a downstream users. This pushes towards a place where we need to model any change in exception as a major version bump (which one could certainly argue is a good thing). As a consequence we could have higher ecosystem churn - if one of my dependencies changes a thrown exception, even if that doesn't impact me directly, I can't have a wide constraint on that dependency, because _my_ documentation is forcibly tied to a single major version of my dependency's documentation.

If we do decide to pursue this, we should treat subtypes of Error as implementation details. It shouldn't matter _what_ error I throw, and callers should never catch it. I would be strongly opposed to a model where change from one error to another needs to be a major version bump.

[bsutton]

@natebosch thats for the response

If a method is documented as /// Throws FooException if the Bar is not initialized;, but I know I'm initializing Bar before the call, how do I prove to the analyzer that I don't need to document FooException as an exception my method might throw?

It would seem that this scenario could be handled via the existing 'ignore lint' mechanism.
I don't know if you can pass an argument to the ignore mechanism but this sort of concept might work:

void a(int b)
{
    if (b == null) throw Bad;
}

void c()
{
   // ignore:  undocumented_throws Bad
    a(1);
}

As to the prose I supposed the following might work:

/// If you don't pass [b] then we will throw [Bad].
void a(int b)
{
    if (b == null) throw Bad;
}

Mentioning the exception by name in square brackets could be considered sufficient for the linter to be happy.

The transitive behavior here has potential implications for evolving packages in the ecosystem

I agree that this is needs some thought.

any change in exception as a major version bump (which one could certainly argue is a good thing).

This very issue is what brought me here. Firebase made a breaking change as to what exceptions were being thrown and I had no idea that I had a problem. Fortunately the code didn't make it into production.

As a consequence we could have higher ecosystem churn - if one of my dependencies changes a thrown exception, even if that doesn't impact me directly,

This is really no different to a change to an api that you aren't using.

I doubt that exceptions change any more frequently than apis (in fact I suspect they change less often) as such I wouldn't expect this to create any great amount of churn and the churn it does create will be valid and necessary.

we should treat subtypes of Error as implementation details.

I think I agree with this. If users felt otherwise it could be implemented as an alternate lint.

[bsutton]

I think we should also allow documentation to target a base exception class.

class BaseException implements Exception {}

class DerivedException extends BaseException {}

/// There is no escaping the fact that we throw [BaseException]
void main()
{
throws DerivedException();
}

I would consider this lazy documentation but we probably need to cater for everyone.

We could have a lint that controls whether this is allowed or explicit declarations of each exception is required.

Explicit declarations of every exception can become very noisy.

[safield]

This is a pain point for me. Not sure what the best solution is for dart, but I personally am missing my checked exceptions I am used to.

[rubenferreira97]

After some Flutter and Dart experimenting I can feel empathy with this issue, caused mostly by poor documentation. I miss checked exceptions, however I know they don't belong to dart.

Given that I would like to ask if tracing exceptions by analyzing the throw keyword is feasible in Dart Lint or Dart itself.

Example:

We know that functionThatMightThrow throws SomeException

// Linter (Information): This function throws the following exceptions: SomeException.
void functionThatMightThrow()
{
   if(someErrorOccurred) throws SomeException;
}

We know that functionThatThrows throws SomeException and SomeException2 (since this function itself do not catch SomeException)

// Linter (Information): This function throws the following exceptions: SomeException, SomeException2.
// Linter (Warning): Unhandled exceptions on line functionThatMightThrow: SomeException (Will rethrow)
void functionThatThrows()
{
   functionThatMightThrow(); 
   throws SomeException2;
}

//Linter (Warning): Unhandled exceptions on line functionThatThrows: SomeException, SomeException2 (Will rethrow)
void doStuff()
{
   functionThatThrows();
}

This might be an optional rule. Since Dart seems to differentiate between Exceptions and Errors maybe take that into account and only check for throws Exceptions . Exceptions normally need to be catch as they represent run-time problems.

This could be a warning or just an informative list of exceptions that each function throws, like mouse hovering a function and giving the list.

I don't know if this could be ever be achieved however I think that this issue really needs some care or else what will happen is: programmers will not handle exceptions for a foolish reason, they don't know which exceptions to catch. Even if we might not know why those exceptions are being throw, since that's the documentation role, at least we are aware that the program might throw X, Y, Z exception.