Pyright: "`yield` is not defined" when using explicit type annotation with a `yield` expression

[decorator-factory]

The following code:

def f():
    x: str = yield 42   # here
    return x

Causes an error: "yield is not defined" on the # here line.
If I remove the explicit annotation (: str), no errors are reported.

[erictraut]

Thanks for the bug report.

When I wrote the parser for pyright, I used the latest grammar specification at the time, which was the Python 3.6 grammar. This grammar spec contains a bug, which was not fixed until the Python 3.8 grammar spec.

I've updated the pyright parser to reflect the corrected grammar spec. Yield expressions are now permitted on the RHS of an annotated variable assignment. So are chained (unannotated) variable assignments.

[layday]

I don't think the 3.8 grammar is quite right either (or maybe I'm reading it wrong)? Both of these are syntax errors:

In [1]: e: int = *(3, )
  File "<ipython-input-1-d50680a24c48>", line 1
    e: int = *(3, )
             ^
SyntaxError: can't use starred expression here


In [2]: f: int = g = h = 3
  File "<ipython-input-2-c1f8fd1a9b5f>", line 1
    f: int = g = h = 3
               ^
SyntaxError: invalid syntax

[erictraut]

Yep, you're correct. The Python 3.8 grammar is wrong too. I guess I should stop trusting the spec and always verify it with the Python interpreter.

[jakebailey]

The definitive source would be the grammar they use to generate the parser (it's no longer hand-written), but it's a bit confusing as to why the website wouldn't just be some transformed version of that.

[erictraut]

Here's what I've found:

The grammar specs for Python 3.6, 3.7, 3.8 and 3.9 are all wrong in different ways.

Python 3.7 and older support yield expressions on the RHS of an annotated assignment. They don't support star expressions (unpack operator) or chained assignments.

Python 3.8 added support for star expressions. (It doesn't support chained assignments, although it arguably should for consistency.)

Python 3.9 has a new parser, but the parse rules in this case appear to be the same as in Python 3.8.

[erictraut]

Oh, it's even stranger than that. The grammar (i.e. the parsers) in Python 3.8 and 3.9 accept star expressions and do not emit any error during the parse phase, but the interpreter later emits a SyntaxError if it encounters a star expression in this context.

[erictraut]

This is addressed in Pyright 1.1.84, which I just published. It will also be included in the next release of Pylance.