Vavr: Annotate for nullness?

[victornoel]

Hi,

This is a lot of work, but it would be great to have javaslang annotated for nullness of method's parameters and return value.

I'm not exactly sure which annotation library should be used, but any should be ok (they are not all equals in features and expressivity…) I guess because I think all analysers that exploit them understand most… I hope :)

[danieldietrich]

Generally I agree that this might be useful but I'm afraid, Javaslang can't depend on any 3rd party library. That is one of the few constraints. Javaslang might only depend on JDK 8.

[victornoel]

hm yes, that makes sense!

To complete the discussion, I think there are JSR (I'm thinking of jsr305) that defines annotations for that (but I also think they were kind of abandoned… not sure if they plan to introduce one in jdk9, they wanted to for jdk8 then never did it…).

Another point is that these annotations does not have to be there at runtime, so I don't know if it makes any difference to you… :)

Another option would be to ship some equivalent annotations and hope analysis tools base their analysis on annotation simple names instead of qualified names. I think findbugs does this, I know in eclipse you can configure many annotations for each concept (nullable, nonnul, etc), so it could be acceptable.

Last option is not to include them and let people like me manually annotate them using external annotation mechanisms such as Eclipse's.

[danieldietrich]

Thank you for your investigation. I also think that it is not priority one of Oracle to include these into JDK. There is an Oracle document "Java 8's new Type Annotations" but the link of the (default?) implementation "Checker Framework" does not exist any more.

Let's keep this open for a while. I don't think it will make it into 2.1.0 but it is definitely worth to wait for JDK 9's take on it.

Because these annotations are so basic/essential, our goal should be to use the (yet non-existing) Java standard.

--

However, I think in most cases null is accepted in Javaslang.

For example

List<String> list = List(null); // is OK!

but

String min = list.min(); // throws NPE!

An additional check will help in cases we pass higher-order functions which of course may never be null.

Btw - Javaslang offers the new TODO function, which helps the developer not to use null for rapid prototyping. Some examples:

import static javaslang.API.*;

final String s = TODO("load user name from DB");

int calculateOverallAmount(int basketId) {
    return TODO("sum overall amount of items in user basket");
}

These will fail with a NotImplementedError containing a descriptive message instead of passing null around (which might be overseen and go to production o_O).

[talios]

From memory - most of the checker frameworks ( and at least IntelliJ ) will work with an @Nullable or @NotNull annotation from ANY package ( since there were so many ), so in theory, javaslang could provide it's own ones.

[victornoel]

The checker framework is actually quite active, it is available at this URL: http://types.cs.washington.edu/checker-framework/

For nullness checking in javaslang, I was thinking more about stuffs like the static function provided by Either, Option, List, and so on: they all return non-null values (even if their content could be null in some cases). The same is true for modification operations such as List.append and others.
Usually, you simply annotate your packages as non null by default and then fixes all the warning of the analysis tool until every special case is annotated.

The TODO function is quite nice! I'm going to use it :)

[danieldietrich]

I also think that it is possible but I'm still thinking... My _feeling_ about it is that it is an additional layer of information about the _reality_. This layer needs to be synchronized with the reality, so it most probably will eventually contain false or missing information.

Because null is almost nowhere used in Javaslang other than in container.getOrElse(defaultValue), container.forEach(maybeNull -> { ... }), container.peek(maybeNull -> { ... }) et.al. I don't know if it is really worth to add that complexity.

[victornoel]

The only interest of introducing nullness annotations is to play nice with analysers and IDEs so that they can better check user's code: clearly it won't bring anything to javaslang code for sure.

[io7m]

Which nullability annotations to use is still quite a serious problem in Java...

http://io7m.github.io/jnull/documentation/p2s1.xhtml

(This is in no way intended to suggest that mine should be used!)

[victornoel]

@io7m FYI, be careful not to mix annotations meant to be used at runtime for constraints validation on beans or user-originated stuffs like that (as the j2e ones or even those in hibernate validation) and those for static analysis used to find bugs in the code (all the others you cite).
Still, what you observe is really a problem… even though I'm not really sure your yet another module is going to make things simpler ^^
I think it would be best to choose one from the existing ones and I'm pretty sure what yours permits to express is covered by at least jsr305 and even maybe the eclipse's one, since it is possible to negate NonNullByDefault behaviour.

[io7m]

FYI, be careful not to mix annotations meant to be used at runtime for constraints validation on beans or user-originated stuffs like that

Yep, I did make that point in the documentation. None of the annotations have consistent retention semantics.

… even though I'm not really sure your yet another module is going to make things simpler

It absolutely isn't, but it at least gives me something consistent to use across all of my own packages.

jsr305

Still has issues in Intellij IDEA, unfortunately. You can't use a custom NonNullByDefault annotation and have to use Intellij's "proprietary" annotations instead to get the same effect.

As I said, I'm absolutely not suggesting to use the io7m annotations. I pointed at that page in the documentation because I know of nowhere else that's listed the issues.

[victornoel]

FYI, be careful not to mix annotations meant to be used at runtime for constraints validation on beans or user-originated stuffs like that

Yep, I did make that point in the documentation. None of the annotations have consistent retention semantics.

What I meant by that is that you shouldn't even consider the validation annotations, and your annotations shouldn't be retained at runtime, it makes no sense in the context of static analysis.

Still has issues in Intellij IDEA, unfortunately. You can't use a custom NonNullByDefault annotation and have to use Intellij's "proprietary" annotations instead to get the same effect.

Ah yes, the problem with the meta annotations from jsr305, there is the same problem with the Eclipse way of manipulating annotations and the need for using eclipse's annotations.. pfff, they are terrible :)

Another place where this a lot of discussion on the matter: http://stackoverflow.com/questions/4963300/which-notnull-java-annotation-should-i-use

[danieldietrich]

The nullability problem in Java is not solvable, even with an additional layer of annotations. Some people might use static checkers, others might not. The annotations do not save us from NullPointerExceptions at runtime.

As you mentioned, to date there are several competing implementations. Adding another one does feel a bit like:

This library is about object-functional programming with algebraic data types. There are other ways to make it explicit that there might occur null in the sense of 'not present', e.g. using the Option type.

From the perspective of API maintenance it complicates things for us. All signs suggest not to introduce that additional layer above the existing API.

Even Scala deprecated the @NotNull type for removal:

For now, we will not introduce it in Javaslang. It is the wrong way - to the dark side of the moon. We will stay on the light side.

[io7m]

Value types will almost certainly introduce a concept of non-nullability at the VM level, anyway. Maybe they'll extend that to reference types some day.

[victornoel]

I don't really agree with your reasoning @danieldietrich :)

I believe the need exists for annotating methods with nullness information, whatever is the library, as long as users are using nullness analysers and IDE with such feedbacks (you should try such facilities, it's really useful, but terrible to use when you get warnings everywhere because the libraries you use are not annotated).
I agree that solving this need is problematic, especially with the "15 standards" situation ^^ so I agree with the conclusion that it's too much hassle to do it for javaslang for now.
But if there was a unique standardized way of annotating nullness provided by the JDK, I really think it would be useful to have javaslang annotated!

FYI I personally started using the external annotations facilities of Eclipse to annotate what I use of javaslang for my needs.

And btw, they didn't deprecate the annotation in scala, but a trait (like an interface with defaults). I don't even think scala has such annotations…

[lukaseder]

I believe the need exists for annotating methods with nullness information, whatever is the library, as long as users are using nullness analysers and IDE with such feedbacks

I'm following this discussion with great pleasure, and I totally agree with @danieldietrich's reluctance of adding any annotation for nullness.

Mike Ernst is a brilliant guy and the author of the checker framework. I've once met him in person at a conference, and he was quite disappointed that most people endlessly bikeshed nullness when talking about the checker framework, rather than realising what powerful and complex type systems they could be implementing instead using annotations (and then be type checking using his framework).

This is important to realise. You cannot possibly think that engaging in a "parallel universe" type system involving type annotations is a simple task, or that you will stop at abusing that language feature for "just some simple" null checks. At some point, you will want to annotate collection emptiness, string "size-ness", number "positive-ness", "single-stream-consumption-ness" etc., so this is a very strategic decision that must not be underestimated (at least in a cleanly designed, not so wishy-washy API).

The Java community (and many other communities) are extremely obsessed with this null legacy. Sure, we all wish languages (and the JVM) were designed without this burden, but we now have it and it usually isn't all that bad, unless some junior developer on the team thinks that it's a good idea to have null collections or something. In a library like Javaslang, you can in fact be very sure that you will hardly ever encounter null, because references in Javaslang are mostly one of three things:

• A collection, which is never null but empty
• An Option which replaces null (it is in fact a collection of cardinality 0..1)
• A non-null reference

Likewise, in Scala, you don't have this discussion, because the Scala language and libraries are designed in very similar ways as Javaslang, so there's no real need for such an annotation in Scala.

TL;DR: YAGNI! So, why not just disable your IDE warnings for all Javaslang API usage? :)

(and if your IDE cannot type check only parts of your application for these annotations, then it is doing it wrong)

[victornoel]

TL;DR: YAGNI! So, why not just disable your IDE warnings for all Javaslang API usage? :)

Good idea actually, didn't think of that :)

[paplorinc]

Another option nobody mentioned: let's treat null as if it were -0.0F ... or false ... or a space character ... or NaN ... or a Void[]{} ... or a transaction of 0 dollars from "Jane Doe" to "John Smith" through the Null Islands!

We could then return 0 for the average or min for an empty collection AND if it only contains a null ... we would correct so many mistakes automagically!

---

Joking aside, as @danieldietrich already witnessed several times, I don't like nulls and I would eliminate most null checks (probably even Objects.requireNonNull calls and simply state in the documentation that "null will result in undefined behavior"), and especially the above abominations, where null is treated as a default value automagically! :)

[mp911de]

Too bad I didn't see this discussion earlier. I'm referring to

the dark side of the moon. We will stay on the light side.

which I feel should be clarified a bit. I think this can cause misunderstanding so let me try to dig into that topic.

From our perspective, we sometimes see the moon entirely, partially or we don't see it at all. This is due to lunar phases. If we dig a bit more into detail, we'll discover there's no such thing as the dark side or light side of the moon. There are two things that play into the issue.

1. The moon is not a perfectly sphere with a perfect mass/gravity distribution - with every point on its surface equidistant from its center. The shape of the moon is that of an oblate spheroid, meaning it has the shape of a ball that is slightly flattened. This causes earth's gravity to pull over time on the most massive parts which slow down moon's rotation. This happened over a very long time which caused the moon to be tidally locked to the earth. Because of conservation of angular momentum, the moon tries to rotate, but earth's gravity applies a torque which causes the moon to align with the side we see right now.

1. The moon still rotates, but it does it only once per each revolution around the earth (about once every 27.3 days). The moon's orbit and rotation around the earth are causing what we call lunar phases. With the rotation around the earth, we are only able to see a certain part of the moon illuminated by the sun. One side of the moon is always illuminated, but we aren't able to see always the whole illuminated side. A full moon from our perspective is when the moon is right behind the earth (as seen from the sun or called opposition). At half moon we see the moon from the side and if we don't see the moon (new moon) the moon is right between the earth and the sun (or called inferior conjunction).
   

TL; DL:

• There are only a near side and a far side of the moon.
• Each side gets the same amount of light over the period of rotation.
• We are just not able to see it from our perspective.

References:

• https://en.wikipedia.org/wiki/Orbit_of_the_Moon
• https://en.wikipedia.org/wiki/Lunar_phase
• https://www.youtube.com/watch?v=j91XTV_p9pc

[danieldietrich]

Personally I will not dive deeper - the result stays the same, for now Javaslang will not include null-checks in form of annotations/as API addition.

This issue stays open for any community discussion. It is a hot topic. Many opinions exist. Please write yours - there is not the one and only truth. Be kind and respectful.

- Daniel

[danieldietrich]

@mp911de nice one :) will show that my 8 yo son - he is interested in space travel / planets & stars

[stephenc]

Just a quick note on JSR-305 annotations...

The oracle JRE redistribution terms expressly declare that only java and/or javax released through the JSR process may be included with the entire distribution if you are redistributing a JRE.

It is unclear whether this prevents redistribution of class files that include annotation references to annotations within the java or javax packages, so this may not affect everyone...

But TL;DR if you bundle a JRE you must ensure that the jar claiming to be JSR-305 annotations is excluded as that JSR has not had an official release

[pgpx]

I use the annotations as a nice clear and consistent form of documentation. Generally nullness (along with thread safety and immutability) is not documented (at least at the method level, e.g. getOrElse does not explicitly state that other can be null). If you specify these annotations then you are making it explicit that you (as the class author) have considered and documented nullness (in a hard-to-miss manner).

[danieldietrich]

@pgpx thx!

at least at the method level, e.g. getOrElse does not explicitly state that other can be null

The 'semantics' of the Javaslang API spec is as follows (the same as I understand the JDK Java API docs):

• if nothing special is stated, method arguments of a specific type accept every possible value of that type (including null - the anti-value in Java that has no type)
• if a method is a partial function, i.e. the method range is defined for a subset of the domain, then the javadoc describes two things:
  
  
  
  1. which values of a specific parameter (type) are valid
  
  
  2. what happens if an invalid value is passed (for a specific parameter)
  
  
• the output is specified
• possible side-effects are specified

The API describes much more than just non-nullability.

[pgpx]

Oh, that does make sense. I guess I'm too used to working with legacy code that has little to no Javadoc, in which case adding some nullness annotations is a quick way of helping my understanding (but arguably should also/instead involve better documentation and/or migrate to Optional/non-null collections, etc). That doesn't apply to Javaslang anyway, so you shouldn't have to add non-standard annotations as well (and analysis tools could/should support separately defining the nullness of 3rd party libraries anyway without modifying their code).

[danieldietrich]

@mp911de "The Dark Side of the Moon" was an album by Pink Floyd that dealt with things that "make people mad". The title is "an allusion to lunacy, rather than astronomy". _Source: Wikipedia_

listen here