Mypy: Announcement issue for plugin API changes

[msullivan]

The mypy plugin interface is experimental, unstable, and prone to change. In particular, there are no guarantees about backwards compatibility. Backwards incompatible changes may be made without a deprecation period.

We will, however, attempt to announce breaking changes in this issue, so that plugin developers can subscribe to this issue and be notified.

Breaking changes fall into three broad categories:

1. Changes to the actual plugin API itself
2. Changes to parts of the "implicit plugin API"---that is, internals that plugins are likely to use (representation of types, etc).
3. Changes to things that really aren't plausibly part of the plugin API (but, of course, that some plugins might be using anyway...)

Issues in category 1 will be consistently announced here, issues in category 3 will probably be announced here only if problems are reported, and issues in category 2 will be somewhere in the middle.

[msullivan]

And we'll start this off with a belated category 3 announcement:

The default wheels for mypy 0.700 are compiled with mypyc. This breaks monkey-patching of mypy internals.

If you are the author of a mypy plugin that relies on monkey-patching mypy internals, get in touch with us and we can probably find a better approach. (For example, #6598 added hooks needed by the django plugin.)

[gvanrossum]

(Sorry for the accidental unpin. Fixed now.)

[JukkaL]

The new semantic analyzer requires changes to some plugins, especially those that modify classes. In particular, hooks may be executed multiple times for the same definitions. PR #7135 added documentation about how to support the new semantic analyzer.

Note that mypy 0.720 (to be released soon) will enable the semantic analyzer by default, and the next release after that will remove the old semantic analyzer.

PRs #7136, #7132, #7096, #6987, #6984, #6724 and #6515 contain examples of changes that may be needed to plugins.

To test that a plugin works with the semantic analyzer, you should have test cases that cause mypy to analyze things twice. The easiest way to achieve is to add a forward reference to a type at module top level:

forwardref: C   # Forward reference to C causes deferral
class C: pass

# ... followed by whatever you want to test

[msullivan]

PR #7397 moves around some functions as part of untangling the cyclic imports in mypy.

The most prominent change and the most likely to impact plugins is:

• mypy.types.UnionType.make_simplified_union -> mypy.typeops.make_simplified_union

Additionally:

• mypy.types.TypeVarDef.erase_to_union_or_bound -> mypy.types.typeops.erase_def_to_union_or_bound
• mypy.types.TypeVarType.erase_to_union_or_bound -> mypy.types.typeops.erase_to_union_or_bound
• mypy.types.{true_only, false_only, true_or_false} -> mypy.typeops
• mypy.types.CallableType.corresponding_argument -> mypy.types.typeops.corresponding_argument
• Many functions from mypy.checkmember have been moved to mypy.typeops. The most prominent here is bind_self.

[msullivan]

PR #7829 makes all name and fullname methods in mypy into properties. This will unfortunately require changes to many plugins. We've decided that it is worth removing a long-standing pain point and that it is better to do it sooner than later.

sed can be used to update code to the new version with something like sed -i -e 's/\.name()/.name/g' -e 's/\.fullname()/.fullname/g'

If your plugin wishes to support older and newer versions during a transition period, this can be done with these helper functions:

from typing import Union
from mypy.nodes import FuncBase, SymbolNode


def fullname(x: Union[FuncBase, SymbolNode]) -> str:
    fn = x.fullname
    if callable(fn):
        return fn()
    return fn


def name(x: Union[FuncBase, SymbolNode]) -> str:
    fn = x.name
    if callable(fn):
        return fn()
    return fn

I don't have an automated way to convert code to use these, but if somebody produces one and sends it to me I will update this post.

Sorry for the inconvenience!

[ilevkivskyi]

PR https://github.com/python/mypy/pull/7923 changed the internal representation of type aliases in mypy. Previously, type aliases were always eagerly expanded. For example, in this case:

Alias = List[int]
x: Alias

the type of the Var node associated with x was Instance, now it will be a TypeAliasType. This change can cause subtle bugs in plugins that make decisions using calls like if isinstance(typ, Instance): ... as such calls will now return False for type aliases.

There are two helper functions mypy.types.get_proper_type() and mypy.types.get_proper_types() that return expansions for type aliases. Note: if after making the decision on the isinstance() call you pass on the original type (and not one of its component) it is recommended to always pass on the unexpanded alias.

There is also a mypy plugin to type-check your mypy plugins, see misc/proper_plugin.py, it will flag all "dangerous" isinstance() calls.

Sorry for the inconvenience!

[ilevkivskyi]

(An additional small reminder related to last two comments: don't forget that a plugin entry point gets the mypy version string, you can use it for more flexibility.)

[bazilio43]

The mypy plugin interface is experimental, unstable, and prone to change. In particular, there are no guarantees about backwards compatibility. Backwards incompatible changes may be made without a deprecation period.

We will, however, attempt to announce breaking changes in this issue, so that plugin developers can subscribe to this issue and be notified.

Breaking changes fall into three broad categories:

1. Changes to the actual plugin API itself
2. Changes to parts of the "implicit plugin API"---that is, internals that plugins are likely to use (representation of types, etc).
3. Changes to things that really aren't plausibly part of the plugin API (but, of course, that some plugins might be using anyway...)

Issues in category 1 will be consistently announced here, issues in category 3 will probably be announced here only if problems are reported, and issues in category 2 will be somewhere in the middle.