Pyright: `NewType` from attrs is unknown

[layday]

Describe the bug
When attempting to decorate a class with attr.attrs, Pyright reports that attrs is partially unknown. The unknown parameter corresponds to _NoOpType which is a NewType of type object.

To Reproduce
Type check the snippet below.

Expected behavior
That the unknown parameter would be of type Type[_NoOpType].

Screenshots or Code

# pyright: strict

import attr

# Type of "attrs" is partially unknown
#   Type of "attrs" is "Overload[(maybe_cls: TypeVar('_C'), these: Dict[str, Any] | None = ..., repr_ns: str | None = ..., repr: bool = ..., cmp: bool | None = ..., hash: bool | None = ..., init: bool = ..., slots: bool = ..., frozen: bool = ..., weakref_slot: bool = ..., str: bool = ..., auto_attribs: bool = ..., kw_only: bool = ..., cache_hash: bool = ..., auto_exc: bool = ..., eq: bool | None = ..., order: bool | None = ..., auto_detect: bool = ..., getstate_setstate: bool | None = ..., on_setattr: _OnSetAttrType | List[_OnSetAttrType] | Unknown | None = ...) -> TypeVar('_C'), (maybe_cls: None = ..., these: Dict[str, Any] | None = ..., repr_ns: str | None = ..., repr: bool = ..., cmp: bool | None = ..., hash: bool | None = ..., init: bool = ..., slots: bool = ..., frozen: bool = ..., weakref_slot: bool = ..., str: bool = ..., auto_attribs: bool = ..., kw_only: bool = ..., cache_hash: bool = ..., auto_exc: bool = ..., eq: bool | None = ..., order: bool | None = ..., auto_detect: bool = ..., getstate_setstate: bool | None = ..., on_setattr: _OnSetAttrType | List[_OnSetAttrType] | Unknown | None = ...) -> (p0: TypeVar('_C')) -> TypeVar('_C')]"
@attr.attrs
class Foo:
    pass

VS Code extension or command-line
Extension, 1.1.72

[erictraut]

I'm not able to repro this problem with the code you've provided above. Could you please double-check that you didn't leave out something?

[layday]

Do you have the latest version of attrs? I'm not able to repro it with v19 in a clean environment:

$ nix-shell --pure -p nodejs-14_x -p python38 --command '
  python -m venv venv
  source venv/bin/activate
  python -m pip install attrs==20.*
  npx pyright --verbose test.py
  rm -rf venv
  '
Collecting attrs==20.*
  Using cached attrs-20.2.0-py2.py3-none-any.whl (48 kB)
Installing collected packages: attrs
Successfully installed attrs-20.2.0
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/.../venv/bin/python -m pip install --upgrade pip' command.
npx: installed 1 in 1.9s
No configuration file found.
Search paths found for configured python interpreter:
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/site-packages
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/lib-dynload
  /Users/.../venv/lib/python3.8/site-packages
When attempting to get search paths from python interpreter:
  Executing interpreter: 'python3'
  Skipping '/nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python38.zip' because it is not a valid directory
  Received 4 paths from interpreter
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/site-packages
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/lib-dynload
    /Users/.../venv/lib/python3.8/site-packages
stubPath /Users/.../typings is not a valid directory.
Assuming Python platform Darwin
Searching for source files
Found 1 source file
/Users/.../test.py
  7:2 - error: Type of "attrs" is partially unknown
  Type of "attrs" is "Overload[(maybe_cls: TypeVar('_C'), these: Dict[str, Any] | None = ..., repr_ns: str | None = ..., repr: bool = ..., cmp: bool | None = ..., hash: bool | None = ..., init: bool = ..., slots: bool = ..., frozen: bool = ..., weakref_slot: bool = ..., str: bool = ..., auto_attribs: bool = ..., kw_only: bool = ..., cache_hash: bool = ..., auto_exc: bool = ..., eq: bool | None = ..., order: bool | None = ..., auto_detect: bool = ..., getstate_setstate: bool | None = ..., on_setattr: _OnSetAttrType | List[_OnSetAttrType] | Unknown | None = ...) -> TypeVar('_C'), (maybe_cls: None = ..., these: Dict[str, Any] | None = ..., repr_ns: str | None = ..., repr: bool = ..., cmp: bool | None = ..., hash: bool | None = ..., init: bool = ..., slots: bool = ..., frozen: bool = ..., weakref_slot: bool = ..., str: bool = ..., auto_attribs: bool = ..., kw_only: bool = ..., cache_hash: bool = ..., auto_exc: bool = ..., eq: bool | None = ..., order: bool | None = ..., auto_detect: bool = ..., getstate_setstate: bool | None = ..., on_setattr: _OnSetAttrType | List[_OnSetAttrType] | Unknown | None = ...) -> (p0: TypeVar('_C')) -> TypeVar('_C')]" (reportUnknownMemberType)
1 error, 0 warnings
Completed in 0.666sec

$ nix-shell --pure -p nodejs-14_x -p python38 --command '
  python -m venv venv
  source venv/bin/activate
  python -m pip install attrs==19.*
  npx pyright --verbose test.py
  rm -rf venv
  '
Collecting attrs==19.*
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Installing collected packages: attrs
Successfully installed attrs-19.3.0
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.
You should consider upgrading via the '/Users/.../venv/bin/python -m pip install --upgrade pip' command.
npx: installed 1 in 1.831s
No configuration file found.
Search paths found for configured python interpreter:
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/site-packages
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8
  /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/lib-dynload
  /Users/.../venv/lib/python3.8/site-packages
When attempting to get search paths from python interpreter:
  Executing interpreter: 'python3'
  Skipping '/nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python38.zip' because it is not a valid directory
  Received 4 paths from interpreter
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/site-packages
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8
    /nix/store/np0b7p04kbhl244yv65d2sc37kk7wcqf-python3-3.8.5/lib/python3.8/lib-dynload
    /Users/.../venv/lib/python3.8/site-packages
stubPath /Users/.../typings is not a valid directory.
Assuming Python platform Darwin
Searching for source files
Found 1 source file
0 errors, 0 warnings
Completed in 0.624sec

[erictraut]

Ah yes, I had version 19.* installed. I'm able to repro with 20.*

This is caused by the same regression that you reported in #938. It will be fixed in the next release. I'll probably do a quick update in the next 24 hours to address this issue. This week's Pylance release will contain the fix as well.

[layday]

I'm seeing more errors in my codebase that I'm assuming are related so I'll wait for the new release :) Thank you.

[erictraut]

This is addressed in Pyright 1.1.73, which I just published.

[layday]

It looks like it's still not able to resolve type variables from py.typed packages with inline annotations in some circumstances:

# pyright: strict

import pydantic

# Type of "parse_obj" is partially unknown
#   Type of "parse_obj" is "(obj: Any) -> Unknown"
pydantic.BaseModel.parse_obj({})

parse_obj is annotated as follows:

class BaseModel(Representation, metaclass=ModelMetaclass):
    @classmethod
    def parse_obj(cls: Type['Model'], obj: Any) -> 'Model': ...

where Model = TypeVar('Model', bound='BaseModel').

If I try to emulate this locally the type variable is resolved correctly:

# pyright: strict

from typing import TYPE_CHECKING, Any, Type, TypeVar

if TYPE_CHECKING:
    Model = TypeVar('Model', bound='BaseModel')

class BaseModel:
    @classmethod
    def parse_obj(cls: Type['Model'], obj: Any) -> 'Model':
        ...

BaseModel.parse_obj({})

If I remove py.typed from Pydantic it is also resolved correctly.

[erictraut]

Thanks for the bug report. Yes, this was another unintended side effect of recent changes I made for py.typed packages that contain inlined stubs. Thanks for your patience as we work through these issues. This problem will be fixed in the next release.

[layday]

I believe there are still some issues with py.typed packages. With Pyright 1.1.74:

# pyright: strict

import attr


@attr.s(frozen=True, slots=True)
class TraceRequestChunkSentParams:
    """ Parameters sent by the `on_request_chunk_sent` signal"""
    chunk = attr.ib(type=bytes)

# Type of "TraceRequestChunkSentParams().chunk" is "str"
reveal_type(TraceRequestChunkSentParams().chunk)


import aiohttp

# Type of "aiohttp.TraceRequestEndParams().chunk" is "Unknown"
reveal_type(aiohttp.TraceRequestChunkSentParams().chunk)

If I remove py.typed from aiohttp and flick on useLibraryCodeForTypes the type of aiohttp.TraceRequestChunkSentParams().chunk is revealed to be str.

[erictraut]

The problem here is that the "aiohttp" package claims that it's typed (as evidenced by the fact that it has a "py.typed" file). Pyright is going to assume in that case that type annotations are present and that it doesn't need to fall back on type inference to understand types.

However, the "chunk" class variable in this case is missing a type annotation.

If a library author claims that their library is typed, it's important for them to actually provide types! Until now, there hasn't been adequate tooling to help them find missing types. I'm working on new functionality in the CLI version of pyright that finds and reports missing types within a "py.typed" library.

[layday]

Ah right, I forgot that useLibraryCodeForTypes is automatically disabled for py.typed packages. The CLI command is an exciting development.

[layday]

Sorry to add another comment here, but it is kind of semi-relevant. prompt-toolkit is another py.typed package, with Enum members which are not annotated. However, the Enum subclasses str (see here). Would all the Enum members need to be annotated in this instance or should Enums be special-cased?

[erictraut]

That's a good point about values within an enum class. Those are effectively constants and should be treated as such. I'll fix that.