Mypy: Function returning a generic function

Created on 18 May 2016 · 13Comments · Source: python/mypy

Mypy doesn't support a function that tries to return a generic callable. I wonder if mypy could make this code work properly:

from typing import TypeVar, Callable, Any

T = TypeVar('T', bound=Callable[..., Any])

def deco() -> Callable[[T], T]:
    def inner(x):
        return x
    return inner

@deco()
def g(x: str) -> int: ...

g('') # should be fine? currently "Argument 1 to "g" has incompatible type "str"; expected None"
g(1) # should be an error

bug priority-1-normal

Source

JukkaL

Most helpful comment

As a workaround, rewriting nested functions to a class seem to work:

from functools import wraps
from typing import TypeVar, Callable, Any, cast

TFun = TypeVar('TFun', bound=Callable[..., Any])

class log_calls:
    def __init__(self, prefix: str) -> None:
        self.prefix = prefix

    def __call__(self, f: TFun) -> TFun:
        prefix = self.prefix
        @wraps(f)
        def wrapper(*args, **kwargs) -> Any:
            print('{}: {!r} {!r}'.format(prefix, args, kwargs))
            return f(*args, **kwargs)
        return cast(TFun, wrapper)

@log_calls('Calling foo')
def foo(x: str) -> str:
    return x

reveal_type(foo)

roganov on 15 Oct 2016

👍10

All 13 comments

This is related to my comments at #1317, but maybe worth keeping open as a separate issue.

rwbarton on 18 May 2016

As Jukka and I discussed at PyCon, this situation where a type variable appears only in the result type of a generic callable could be handled by "floating in" the type variable to the result type: instead of

def [T] () -> def (T) -> T

give deco the type

def () -> def [T] (T) -> T

I wonder how often this arises in practice?

rwbarton on 13 Jun 2016

The only case where I remember having seen this is in a decorator that is used like this:

@memoize(cache_size=200)
def do_stuff(): ...

The decorator, when called, returns a generic function T -> T that preserves the signature of the decorated function.

JukkaL on 13 Jun 2016

At Zulip, we're using such decorator-returning functions a lot. One of these decorator-returning functions, named cache_with_keys, is applied to dozens of functions, many of which are called hundreds of times (see https://github.com/zulip/zulip/issues/1348). It'll be great to see this issue fixed since a lot of our code is not properly type checked because of this.

sharmaeklavya2 on 22 Jul 2016

I have another example from our codebase:

@app.route('/')
@rate_limit(30, timedelta(seconds=60))
def index() -> Response:
    # ...

Both of the decorators returned by app.route() and rate_limit() at lest formally preserve the signature of the callable being decorated.

Sadly I don't have enough knowledge to help with this myself.

jstasiak on 10 Aug 2016

This seems like something we'll really want to fix, but it's not immediately clear yet what the fix looks like.

gnprice on 11 Aug 2016

As a workaround, rewriting nested functions to a class seem to work:

from functools import wraps
from typing import TypeVar, Callable, Any, cast

TFun = TypeVar('TFun', bound=Callable[..., Any])

class log_calls:
    def __init__(self, prefix: str) -> None:
        self.prefix = prefix

    def __call__(self, f: TFun) -> TFun:
        prefix = self.prefix
        @wraps(f)
        def wrapper(*args, **kwargs) -> Any:
            print('{}: {!r} {!r}'.format(prefix, args, kwargs))
            return f(*args, **kwargs)
        return cast(TFun, wrapper)

@log_calls('Calling foo')
def foo(x: str) -> str:
    return x

reveal_type(foo)

roganov on 15 Oct 2016

👍10

Oooh, very clever! I can't say I personally understand how it works, and we
should still fix the original error, but this looks like a useful
workaround. Thanks!

On Sat, Oct 15, 2016 at 4:28 AM, Yegor Roganov [email protected]
wrote:

As a workaround, rewriting nested functions to a class seem to work:

from functools import wrapsfrom typing import TypeVar, Callable, Any, cast

TFun = TypeVar('TFun', bound=Callable[..., Any])
class log_calls:
def init(self, prefix: str) -> None:
self.prefix = prefix
def __call__(self, f: TFun) -> TFun:
    prefix = self.prefix
    @wraps(f)
    def wrapper(*args, **kwargs) -> Any:
        print('{}: {!r} {!r}'.format(prefix, args, kwargs))
        return f(*args, **kwargs)
    return cast(TFun, wrapper)
@log_calls('Calling foo')def foo(x: str) -> str:
return x

reveal_type(foo)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/python/mypy/issues/1551#issuecomment-253978622, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACwrMntwUxGKXBJXiFRnuVKRAye7x5Rxks5q0LjDgaJpZM4IhSDU
.