Typing: Shorter syntax for Optional[...]

[JukkaL]

Multiple people have suggested a shorter syntax to replace Optional[...]. The current syntax will become more inconvenient if we don't infer optional types from None default values (see #275).

Here are various options that I remember seeing proposed:

1. x? or ?x

This would require new Python syntax.

Hack uses ?x. TypeScript uses x? for names, not types, if I've understood things correctly.

1. x | None

This would require support for | for all type objects, which would not be backward compatible.

The | operator is used by TypeScript.

1. {x}

Probably not an option as this is too cryptic. Suggested in #151 .

1. from typing import Optional as O or from typing import Optional as Opt

This is not actually a new syntax and works currently. These are arguably inelegant.

[refi64]

FWIW most modern languages that support optional/non-nullable types use ? (e.g. Crystal, VerboseScriptTypeScript, Swift, Dart, etc.).

[dmoisset]

One alternative in the line of x|None (i.e. improving the syntax for Union rather than Optional) and taking from 3, would be to make: [T1, T2, T3] a shorthand for Union[T1, T2, T3]; so instead of Optional[T] you could use [T, None]

The same could be applied with tuples of types (round brackets) or sets of types (curly brackets).

[markshannon]

I don't like the list form [T1, T2] as that implies a record type to me. The set form {T1, T2} seems more natural for a union.

[markshannon]

So here are the options proposed so far (that I am aware of).

def foo(x:Optional[int]=None): ... Status quo -- works out of the box
def foo(x:{int}=None): ... A bit cryptic -- works out of the box
def foo(x:int|None=None):... Involves major modification on int
def foo(x:int or None=None):... Works out of the box -- Loses or None when evaluated
def foo(x:{int, None}=None):... Works out of the box
def foo(x:[int, None]=None):... Works out of the box -- [int,None] != [None,int]?

Note that all but the first two are general to any union type and not specific to a union with NoneType.

[ilevkivskyi]

x? or ?x for variables is not sufficient, since optional types can appear inside other types (like Callable[..., Optional[int]] or List[Optional[str]]). Also I think that with int? syntax (like in Swift) it is easy to miss the ? at the end. I don't like the [T1, T2] syntax either, since it looks ugly in Callable, for example Callable[[[int, None]], [int, None]].

In general I don't think the "verbosity" is a problem, I have worked with --strict-optional for some time I didn't notice this. I have rather noticed that people often use None where it can be avoided (PEP 526 partially solves this problem).

[ilinum]

I can't seem to see where the x? and ?x is not sufficient for expressing any optional type. If you want to have a List[Optional[str]] it would just be List[str?]. If you want Optional[List[str]], it will be List[str]?.

Here are my two cents on the issue: the current syntax is a little more explicit, so it's harder to miss. However, if we go for a less verbose syntax, people will get used to it pretty fast and will not miss the optional types.

At the same time, I think it's okay to keep the current syntax. If you're just starting to use types, then the explicit syntax is much easier to understand whereas any other syntax takes more time getting used to.

[ambv]

I'm +1 on either str? or ?str. Very short and obvious.

[gvanrossum]

Maybe we should use ?str? or ¿str?.

[ethanhs]

I am -1 on either ?str or str?. One of the great things about Python's design in my mind is that it doesn't use !bool_var. not bool_var may be more explicit, but for many reasons, its harder to miss and much more readable. It is very good at almost phrase like constructions, eg if not check: ... instead of if !check, which can be easily missed when reading or writing.

I have a few ideas to throw out (listing Jukka's first as I like Opt, and as a reference):

def foo(a: Optional[str]): ...
def foo(a: Opt[str]): ...
def foo(a: Non[str]): ...
def foo(a: Null[str]): ...
def foo(a: Maybe[str]): ...

I think of all of these Opt seems promising, but I don't think it is exactly there. The other one that seems interesting to me is Maybe as it is shorter, but not as much as other suggestions, however it has the advantage of being intuitive, memorable, easy to understand, and shorter than Optional.

[JukkaL]

@ethanhs Maybe[str] was one of the original ideas floated around before we chose Optional, but it was rejected because if was considered too clever, I think.

[markshannon]

What is actually wrong with Optional[str]? And where is it a problem?

I think we ought to establish what problem we are solving before we attempt to solve it.

[JukkaL]

My main problem with Optional[str] is that more complex types often are quite verbose. Here is an example function return type from a recent mypy PR:

Optional[Callable[[MethodSigContext], CallableType]]

If the return type would be Optional as well, it would look like this:

Optional[Callable[[MethodSigContext], Optional[CallableType]]]

This doesn't look very readable to me, though part of this because of Callable. Using type aliases can help, but it can also make code hard to read, as you need to find the definition of the alias to understand the type.

Here's the same type using type? syntax, for example:

Callable[[MethodSigContext], CallableType?]?

I don't like this, since it's hard to see that the Callable[...] value is optional. The final ? is next to ], so I have to find the matching [ to see which type is getting modified by the ?.

Here's with ?type:

?Callable[[MethodSigContext], ?CallableType]

This is pretty readable, but the ? prefix doesn't look very clean or Pythonic, I'd say. Here the ? is associated with name of the type instead of the closing square bracket, which arguably makes this easier to read than the previous example.

Here's the example using | None:

Callable[[MethodSigContext], CallableType | None] | None

It doesn't strike me as a significant win over Optional[...] in terms of readability.

[dmoisset]

The verbosity (and nesting of brackets) is one of the drawbacks, but I also like the chance to remove the confusion on Optional vs optional arguments (that may have a default or None but something else). That gives us two unrelated but slightly overlapping concepts use the same name, so users are confused (I have had to clarify this thing to every user I have taught to use mypy).

Something that could be done from a styleguide standpoint is use the | operator but recommend to put None first. So @JukkaL 's example would be:

None | Callable[[MethodSigContext], None | CallableType]

which I think makes it more visible which parts are nullable, and IMO is more readable then the current syntax given the removed nesting

[markshannon]

It seems unavoidable that complex types will be lengthy. But longer does not necessarily mean harder to read.
Names like Optional and Callable may take a bit more space, but they are highly readable (at least for a native English speaker). ? requires me to mentally translate to "Optional".

I don't see that any of the suggestions are significantly more readable, and most aren't really any shorter apart from ?, and that is far too big a language change for a fairly unimportant feature.

[markshannon]

Regarding @dmoisset point that Optional is confusing w.r.t. to optional arguments, wouldn't a simple rename suffice?

from typing import Optional as NoneOr
x : NoneOr[str]

[dmoisset]

@markshannon I could do renames like that in my own code, but "personal" approaches are not useful on annotations, because those are essentially placed on interfaces. I would make my annotations harder to read for third parties using my library unless I'm using a "standard" alias, so that's why it makes sense to discuss this change as part of the standard.

[jamesob]

Supporting x? as a builtin in lieu of typing.Optional[x] would be a big win for ergonomics.

Incidentally oftentimes my first two imports in a file are

import typing as t
# and now
from typing import Optional as Op

[gvanrossum]

With PEP 604 now accepted and slated for Python 3.10, we can just say int | None. This is less verbose than Optional[int], and has the added advantage that if you happen to need an optional union, you can just tack on | None to the end, e.g. int | str | None -- with Optional[] and Union[] I often hesitated whether that's better spelled as Optional[Union[int, str]] or as Union[int, str, None].

If we were to adopt int? we'd have this problem back -- do you write int | str?, or int? | str?, or (int | str)?...

It's also easier to backport int | None -- it's already acceptable syntax to ast.parse(), and once the major type checkers understand the notation we can start using it in typeshed. Not so for int?: we'd have to do precision surgery on typed_ast to be able to support that.

[pauleveritt]

Is PEP 604 the kind of thing that can go in typing-extensions as a backport?

[gvanrossum]

Is PEP 604 the kind of thing that can go in typing-extensions as a backport?

There's nothing to import (it's just an overloaded operator) -- but type checkers have to support it. It makes sense to me that type checkers could support it even for older Python versions, at least in stub files and under the scope of from __future__ import annotations (PEP 563), which is supported by Python 3.7 and later.

In earlier Python versions (even Python 2) it could be supported in type comments -- but again we first have to teach the type checkers about it. (I imagine there's a type checker where you have some influence. :-)

Once type checkers support it we could start using it in typeshed.

[Ovsyanka83]

I am -1 on either ?str or str?. One of the great things about Python's design in my mind is that it doesn't use !bool_var. not bool_var may be more explicit, but for many reasons, its harder to miss and much more readable. It is very good at almost phrase like constructions, eg if not check: ... instead of if !check, which can be easily missed when reading or writing.

I agree with ethanhs here. One of the strengths of Python is that it is often read as simple phrases (like ternary expressions, not and is not operators, and list comprehensions) so the speed of both reading and writing code increases while the chance of misreading something decreases. I believe that ?type or type? would make type hints harder to read, especially when used extensively. At this point, I cannot think of a syntax that is more verbose and concise than None | type. It is so good, in fact, that I will probably stop using typing.Optional altogether.

[gvanrossum]

Let’s close this issue. Maggie sent het proposal to typing-sig, which is where we discuss these things nowadays.

[The-Compiler]

For reference, because I had no idea what "to typing-sig" meant: That's a mailinglist ([email protected]) and the thread mentioned is here: Mailman 3 Update: Work on Optional Syntax PEP - Typing-sig - python.org