Mypy: Make any callable compatible with (*args: Any, **kwargs: Any)?

[JukkaL]

Perhaps a callable that accepts arbitrary arguments by having (*args: Any, **kwargs: Any) could be treated similarly to Callable[..., X] (with explicit ...), i.e. a callable with arbitrary arguments would be compatible with it.

The reason for this is that there's no easy way to annotate a function so that the type would be Callable[..., X], even though this seems to be a somewhat common need. Users sometimes assume that using (*args: Any, **kwargs: Any) will work for this purpose and are confused when it doesn't. https://github.com/python/typing/issues/264#issuecomment-436525286 is a recent example.

Example:

from typing import Any

class A:
    # A method that accepts unspecified arguments and returns int
    def f(self, *args: Any, **kwargs: Any) -> int: pass

def B(A):
    def f(self, x: int) -> int: pass  # Should be okay

We could perhaps extend this to handle decorators as well:

from typing import Any, Callable, TypeVar

T = TypeVar('T', bound=Callable[..., Any])

def deco(f: T) -> T:
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        print('called')
        return f(*args, **kwargs)
    return wrapper  # Maybe this should be okay

[ilevkivskyi]

I actually like this idea, I have seen this confusion several times, and although it is a bit unsafe, most of the time when people write (*args, **kwargs) it means "don't care", rather than "should work for all calls".

[gvanrossum]

Agreed, this is a case where practicality beats purity.

[jonapich]

I just stumbled upon this trying to build a Protocol to require a callable that takes an Event as the first parameter:

class SubprocessTarget(Protocol):
    def __call__(self, quit_event: MultiprocessEvent, *args: Any, **kwargs: Any) -> int: ...

Just throwing this out here, but maybe a variant of Any could be useful for defining callback signatures:

...*args: AnyOrMissing, **kwargs: AnyOrMissing (sorry for the poor choice of name)

Or maybe the __call__ could be decorated with something like @OptionalVarArgs to indicate that we really don't care about args/kwargs? Or the other way around, @PartialSignature to indicate that the method may include additional arguments, then the protocol signature only lists the required arguments (it doesn't have to include args/kwargs anymore).

Another similar (but different) use case would be to require the method to include a specific kwarg (or **kwargs), regardless of its position. I feel these will be hard to implement without the help of some decorators?

[ckyoog]

I do read the following words in mypy docs

Mypy recognizes a special form Callable[..., T] (with a literal ...) which can be used in less typical cases. It is compatible with arbitrary callable objects that return a type compatible with T, independent of the number, types or kinds of arguments. Mypy lets you call such callable values with arbitrary arguments, without any checking – in this respect they are treated similar to a (*args: Any, **kwargs: Any) function signature.

Does it mean this issue has been resolved? Correct me if I am wrong.

[jonapich]

@ckyogg: no, the issue here is to be able to enforce partial signatures to improve duck typing possibilities. Your excerpt is asking for "any callback whatsoever" but i want to define "a callback with a quit flag as first argument and anything else after"

[jlaine]

Has anyone found a good solution for this? My usecase is exactly the same as @jonapich I want to enforce the first argument and allow anything afterwards.

[chadrik]

+1. It seems like the only solution right now is to create a union type alias covering all cases, which is obviously something I'd like to avoid doing!

class CallbackWithKwargs1(Protocol[T_contra, T_co]):
    def __call__(self, __element: T_contra, __arg1=...) -> T_co:
        pass

class CallbackWithKwargs2(Protocol[T_contra, T_co]):
    def __call__(self, __element: T_contra, __arg1=..., __arg2=...) -> T_co:
        pass

...

CallbackWithKwargs=Union[
    CallbackWithKwargs1, CallbackWithKwargs2, ...]

[antonagestam]

Another common real-world example is trying to create a type for a functional view in Django, where the first argument is always a Request and there might be more arguments for URL parameters.

[rmorshea]

@antonagestam perhaps https://github.com/python/mypy/issues/8263 could help resolve this issue?

[jmehnle]

I really just care about typing a function (in a prototype) with arbitrary arguments (except for perhaps self), and def foo(self, *args: Any, **kwargs: Any) -> Any: ... just doesn't work. What am I to do?

[JelleZijlstra]

@jmehnle does something like foo: Callable[..., Any] work?

[rmorshea]

Tip for achieving this at present:

from typing import Callable, Any


class Parent:
    method: Callable[..., int]
    def method(  # type: ignore
        self, *args: Any, **kwargs: Any,
    ) -> int:
        ...

class Child(Parent):
    def method(self, x: str) -> int:
        ...

Try it: https://mypy-play.net/?mypy=latest&python=3.8&gist=ba254920ee62608c5696c29610b2a379

[jmehnle]

@jmehnle does something like foo: Callable[..., Any] work?

Alas, no. I get: note: Protocol member Connection.cursor expected settable variable, got read-only attribute.

And unfortunately @rmorshea's trick still generates that very same message (see also https://github.com/python/mypy/issues/9560).

[hauntsaninja]

Doesn't Guido's trick in #9560 work?

[jmehnle]

It does. I saw that only just now. I'm going to use that for the time being, but of course that still doesn't make any callable match the (*args: Any, **kwargs: Any) type (this issue).