Mypy: Distinguish between attributes of a class and attributes of its instances

[mrwright]

The following typechecks:

class A(object):
    def __init__(self):
        # type: () -> None
        self.X = 0 # type: int

x = A.X

even though X is not an attribute of the class A, just an attribute of its instances, and the x = A.X line causes an AttributeError when executed.

[JukkaL]

This is how mypy currently behaves by design, but the behavior is clearly a compromise. To distinguish between class and instance attributes, we could do something like this:

• Attributes only initialized via self get an instance attribute flag. These attributes can't be accessed via the class object (neither read or set).
• The instance attribute flags are normally inherited in derived classes.
• It's okay to assign to an inherited instance attribute in a class body. In this case the attribute loses the instance attribute flag in the derived class.
• Attributes without the flag (i.e. those assigned to in the class body) can be used as either instance or class attributes. They behave identically to how all attributes are currently treated by mypy.

[oleiade]

I very often hit this limitation. Has been any advancement made on the topic? How can I help making this move forward?

Thanks :-)

[gvanrossum]

One thing you can do to help would be to support PEP 526, in its tracker item at python/typing#258 and its draft at https://github.com/phouse512/peps/commits/pep-0526. And maybe ClassVar proposed there can be backported? But I guess first and foremost mypy's internals need to be redesigned to be able to make the distinction.

[JukkaL]

There's a proposed new way of annotation class variables: https://github.com/python/typing/issues/258

On pre-3.6 Python it would look like this:

from typing import ClassVar

class A:
    x = 0  # type: ClassVar[int]   # Class variable
    y = 0  # Instance variable with default in class body

A().x = 3  # Error: "x' is a class variable

This wouldn't immediately help with the original issue, though, and the original proposal that I discussed above is still relevant. We've made no progress on that.

@oleiade If you are interested in trying to implement instance-only attribute detection, I'm happy to give some ideas about how to do it -- or if you want to bounce ideas off me, please go ahead :-)

[enomado]

Historically classvars was used to to define runtime types of instances with the same name in ORMs, forms, validatiors.

As an example SQLAlchemy DeclarativeBase mentioned in #974

class User(Base):
    name = ...

User().name: str but
User.name: InstrumentedAttribute and it is used as filter:

session.query(User).filter(User.name.like('...'))

I don't know a details of how hard it could be implemented but at first glance it could be expressed as type without new syntax.
name: ClassInstanceVar[classvartype, instancevartype]

UPD:
Actually according PEP mypy should not treat User.name as str if it's defined as name: str because its instance definition anyway, and currently it does.

[tuukkamustonen]

@enomado I was actually just recently asking something similar for peewee ORM in https://github.com/coleifer/peewee/issues/1559.

name: ClassInstanceVar[classvartype, instancevartype]

Maybe classvartype could be even inferred as in:

class User:
    name: ClassInstanceVar[str] = CharField(...)

User.name  # CharField
User().name  # str

Actually according PEP mypy should not treat User.name as str if it's defined as name: str because its instance definition anyway, and currently it does.

So you mean with just:

class User:
    name: str

mypy will infer Any type for User().name?

[gvanrossum]

No:

class User:
  name: str

reveal_type(User().name)

Gives

_.py:4: error: Revealed type is 'builtins.str'

[enomado]

mypy will infer Any type for User().name?

All this issue is about that typing system should feel a difference, but currently it does not.

```from typing import ClassVar

class User:
name: str

reveal_type(User().name)
reveal_type(User.name)

class XUser:
name: ClassVar[str]

reveal_type(XUser().name)
reveal_type(XUser.name)
```

buildins.str in all cases

[gvanrossum]

Well if you just put name: str in the class I think mypy is right in inferring both User.name and User().name as type str. It is quite similar to putting name: str = '' in the class -- mypy doesn't care whether the value is initialized. And this is a common use case that we don't want to break.

For the actual example using InstrumentedAttribute maybe the solution is to make that class implement the descriptor protocol (__get__), at least in the stub. Then mypy uses the __get__ return type for the instance variable type.

[kamahen]

FWIW, Kythe takes a similar attitude to instance and class variables for cross-referencing source files (and not just for Python) -- both instance and class variables are treated as the same.

[enomado]

@tuukkamustonen looks like i've found the workaround for now.

class Base:  # this should be BaseModel or smth
    pass


class D(Generic[InstT]):

    @overload
    def __get__(self, inst: None, own: Type[Base]) -> Any: pass

    @overload
    def __get__(self, inst: Base, own: Type[Base]) -> InstT: pass

    def __get__(self, inst, own):
        pass

    def __set__(self, obj: Base, value: InstT) -> None: pass


class A(Base):
    f: D[int] = D()


reveal_type(A.f) # any
reveal_type(A().f) # int

This could be Generic with ClassT too, to act exact as ClassInstanceVar should.

Probably we can also mix this with

from typing import TYPE_CHECKING
if TYPE_CHECKING:
  class YourModel...
else:
  class YourModel...

Btw i'm not sure how to deal with __set__ A.f = 123
error:Incompatible types in assignment (expression has type "int", variable has type "D[int]")

@gvanrossum, thanks!

[JukkaL]

Closing as a duplicate of #240.