Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Overload ambiguity is ignored in self-annotated methods #11347

Open
BvB93 opened this issue Oct 17, 2021 · 5 comments · May be fixed by #11366 or #17239
Open

Overload ambiguity is ignored in self-annotated methods #11347

BvB93 opened this issue Oct 17, 2021 · 5 comments · May be fixed by #11366 or #17239
Labels
bug mypy got something wrong topic-overloads

Comments

@BvB93
Copy link
Contributor

BvB93 commented Oct 17, 2021

Bug Report

Per the title: mypy does not check for overload ambiguity in self-annotated methods.
Mypy will instead, effectively, ignore self and then return the first matching overload,
thus circumventing the usual Any that is returned when overload ambiguity is involved.
This has lead to a number of false positives in numpy/numpy#20099 and (in part) numpy/numpy#20072.

The conditions for triggering this bug are as following:

  • The callable is a method (things work fine for functions).
  • The method is overloaded.
  • self must be annotated with a parametrized generic.

To Reproduce

from typing import Any, Generic, overload, TypeVar, TYPE_CHECKING
import numpy as np

_CharType = TypeVar("_CharType", np.str_, np.bytes_)


class CharArray(Generic[_CharType]):
    @overload
    def strip(self: CharArray[np.str_], chars: str = ...) -> CharArray[np.str_]: ...
    @overload
    def strip(self: CharArray[np.bytes_], chars: bytes = ...) -> CharArray[np.bytes_]: ...


ar_any: CharArray[Any]
ar_str: CharArray[np.str_]
ar_bytes: CharArray[np.bytes_]

if TYPE_CHECKING:
    # The good
    reveal_type(ar_str.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"
    reveal_type(ar_bytes.strip())  # E: Revealed type is "__main__.CharArray[numpy.bytes_]"

    # The bad
    # "__main__.CharArray[Any]" would be expected here due to overload ambiguity
    reveal_type(ar_any.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"

    # Calling the method directly from the class does do the trick
    reveal_type(CharArray.strip(ar_any))  # E: Revealed type is "test.CharArray[Any]"

Your Environment

  • Mypy version used: 0.910
  • Mypy command-line flags: n.a.
  • Mypy configuration options from mypy.ini (and other config files): n.a.
  • Python version used: 3.9.6
  • Operating system and version: macOS 11.4
@BvB93 BvB93 added the bug mypy got something wrong label Oct 17, 2021
@BvB93 BvB93 mentioned this issue Oct 17, 2021
Open
@sobolevn
Copy link
Member

Simplified reproduction

Without self annotation

Let's see what will happen when we use regular @overload with Any

from typing import Generic, TypeVar, overload, Any

T = TypeVar('T')

class Some:
    @overload
    def method(self, arg: int) -> bool: ...
    @overload
    def method(self, arg: str) -> float: ...
    def method(self): ...

# ok
s1: Some
s2: Some
reveal_type(s1.method(1))  # N: Revealed type is "builtins.bool"
reveal_type(s2.method('a')) # N: Revealed type is "builtins.float"

# also ok
s3: Some
a: Any
reveal_type(s3.method(a))  # N: Revealed type is "Any"

With self annotation

from typing import Generic, TypeVar, overload, Any

T = TypeVar('T')

class Some(Generic[T]):
    @overload
    def method(self: Some[int]) -> bool: ...
    @overload
    def method(self: Some[str]) -> float: ...
    def method(self): ...

# ok
s1: Some[int]
s2: Some[str]
reveal_type(s1.method())  # N: Revealed type is "builtins.bool"
reveal_type(s2.method())  # N: Revealed type is "builtins.float"

# error
s3: Some[Any]
reveal_type(s3.method()) # N: Revealed type is "builtins.bool"
# Should be: Revealed type is "Any"

Proposed solution

Wrong @overload cases should also be Any even with self annotations.

I will send a PR with the fix.

@sobolevn
Copy link
Member

On the second thought: why s3.method() should return Any? While we can be sure that method returns Union[bool, float].

s3: Some[Any]
reveal_type(s3.method()) # N: Revealed type is "builtins.bool"
# Should be: Revealed type is "Union[bool, float]"

@BvB93
Copy link
Contributor Author

BvB93 commented Oct 20, 2021

On the second thought: why s3.method() should return Any? While we can be sure that method returns Union[bool, float].

I imagine this would be a great addition if we had a dedicated unsafe-union-type (e.g. python/typing#566),
both for methods as well as functions in general. Without it, you're very quickly going to run into the typical
issues associated with unsafe unions (#1693).

@sobolevn sobolevn linked a pull request Oct 20, 2021 that will close this issue
Open
@sobolevn
Copy link
Member

Yeap, @BvB93! I've decided that this is out of scope 👍

@vnmabus
Copy link

vnmabus commented Oct 22, 2021

BTW: this issue is why the solution #10207 (comment) does not work in Mypy.

vnmabus added a commit to GAA-UAM/scikit-fda that referenced this issue Oct 23, 2021
@vnmabus
Future typing of Basis Smoother.
15b9497 
Currently not working in Mypy because of
python/mypy#11347
@BvB93 BvB93 mentioned this issue Oct 25, 2021
Open
This was referenced Nov 1, 2021
Closed
Closed
Open
@Dr-Irv Dr-Irv mentioned this issue May 3, 2022
Closed
@BvB93 BvB93 mentioned this issue Jul 1, 2022
Closed
@BvB93 BvB93 mentioned this issue Jun 7, 2023
Merged
4 tasks
flaeppe pushed a commit to flaeppe/mypy that referenced this issue May 13, 2024
@sobolevn @flaeppe
Now overloads with ambiguous self are handled properly, refs python…
dfdcd0e 
…#11347
@flaeppe flaeppe linked a pull request May 13, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug mypy got something wrong topic-overloads
Projects
None yet
Milestone
No milestone
4 participants
@vnmabus @sobolevn @BvB93 @AlexWaygood