Can I identify the namedtuple type of an object from another package using isinstance() or some other call? Consider the following code for checking the type of a namedtuple:
>>> vercingatorix=namedtuple('vercingatorix','x')
>>> v=vercingatorix(1)
>>> w=v
>>> isinstance(w,vercingatorix)
True
So far, so good. But what if the namedtuple is created by another package, e.g. pkg1.pkg2.pkg3.vercingatorix, as reported by type()? I tried isinstance(w, pkg1.pkg2.pkg3.vercingatorix) but I get an AttributeError. The namedtuple is not globally accessible (e.g. isinstance(w, vercingatorix)- -- NameError).
This is a simplified case of my problem. I have a namedtuple object (representing a message) that, depending on the kind of message, has a different subclass type. I need to be able to identify the kind of message, and the simplest way to do this seems to be to look at its type. But I can't formulate a way to do this.
I can grab the class's name (which reveals the message type) and use that for comparison:
>>> w.__class__.__name__
'vercingatorix'
Related
The following example of accessing python Enum members by slice comes from its documentation (where Period happens to be an Enum):
list(Period)[:2]
When subclassing Enum, I thought it might make sense to permit a syntax that skips the wrapping in list (i.e. make the syntax for using my subclass more consistent with the syntax for accessing members of a list or tuple):
import enum
class MyEnumMeta(enum.EnumMeta):
def __getitem__(self, indexOrSlice):
if isinstance(indexOrSlice, (int, slice)):
return list(self)[indexOrSlice]
else:
return enum.EnumMeta.__getitem__(self, indexOrSlice)
class MyEnum(enum.Enum, metaclass=MyEnumMeta): pass
class Period(MyEnum):
A = "a"
B = "b"
C = "c"
print(Period["A"])
print(Period[1]) # type: ignore[misc]
print(Period[:2]) # type: ignore[misc]
Although this seems to execute fine, mypy is raising the following error unless I append # type: ignore[misc] to every line that attempts to use the "more condensed" syntax (which makes the syntax actually less condensed):
Enum index should be a string
(You can search for this in the mypy code, to see the relevant section)
I am uncomfortable with adding a statement that will suppress any legitimate [misc] errors that happen to occur in the same line. Furthermore, the fact that mypy went to the effort to raise this error (and that Enum doesn't permit the condensed syntax directly) makes me wonder whether there is some good reason why it should be impossible to access Enum members via integer or slice even in the way demonstrated in the Enum documentation.
If I shouldn't be doing what I am trying to do, can someone please explain why? Otherwise, can someone please suggest a better way to handle mypy? If mypy has made an error of failing to anticipate that the Enums it encounters might actually be subclasses of Enum that override __getitem__ as I propose, how do I report that error?
There is nothing wrong with your changes. Mypy has it's enum support hard-coded, though, so it won't recognize them.
If I have an enum Foo, like so:
from enum import Enum
class Foo(Enum):
BAR = 1
BAZ = 2
Then, one of the many wonderful things about enums is that there are already multiple ways of accessing members of the enum. The following example is in the interactive console:
>>> Foo(1)
<Foo.BAR: 1>
>>> Foo['BAR']
<Foo.BAR: 1>
>>> Foo.BAR
<Foo.BAR: 1>
>>> Foo(Foo.BAR)
<Foo.BAR: 1>
As such, it may be unwise for you to override EnumMeta.__getitem__ in the way you're doing, as it may not be what others reading your code will expect. The inbuilt feature of enums is that you can access members by passing the member's name — a string — into the square brackets. Doing otherwise will violate the expectations of users of your code, and, as the MyPy error shows, it will also violate the expectations of a type checker. Explicitly converting the enum to a list outside of the class is somewhat different — the type checker, and any human readers of your code, are all well aware that lists can be indexed using integers or slices.
In Python's typing module, it seems that anything that is not a generic type does not support subscripting:
>>> import typing
>>> from yarl import URL
>>> StrOrURL = typing.Union[URL, str]
>>> typing.List[StrOrURL] # OK; List is generic
typing.List[typing.Union[yarl.URL, str]]
>>> SetOrDict = typing.Union[set, dict]
>>> SetOrDict[StrOrURL] # Raises
TypeError: typing.Union[set, dict] is not a generic class
This will be the case whether SetOrDict is defined with typing.Union or typing.TypeVar. (In Python 3.7, at least.)
It looks like there is a way to subclass Generic, presumably like:
class SetOrDict(typing.Generic[StrOrURL]):
# ...
But this will immediately raise TypeError: Parameters to Generic[...] must all be type variables.
Is there a way to accommodate the above? I.e., what is the recommended practice for subscripting user-defined (non-generic) typedefs?
You can't use a Union[...], alias or otherwise, as a generic type, no. Your union contents are also not generic, you can't state what types a dict or set object can contain by using dict or set directly, you'd use Set or Dict.
You have a new union:
URLorStrSetOrDict = typing.Union[Set[StrOrUrl], Dict[StrOrUrl, Any]]
Note that a dictionary has keys and values, you need to provide information on both. If this used as the input for an API, then consider using the more generic and immutable AbstractSet and Mapping types; this makes it clear that your API only needs to read information.
Personally, I'd look hard at refactoring a codebase that is this muddy about what kinds of objects it can accept. Requiring that an API accepts only sets and existing code has to pass in set(dictionary) instead of dictionary is not an arduous requirement, or perhaps the API really would work with any iterable of StrOrUrl.
I have a function that accepts a class that derives from NamedTuple and converts it into a schema. However when I run MyPy on the following code it fails with Argument 1 to "to_schema" has incompatible type "Type[Foo]"; expected "Type[NamedTuple]"
from typing import NamedTuple, Type
def to_schema(named_tuple: Type[NamedTuple]):
pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Is there a way to properly type the code so that it typechecks with MyPy?
Edit:
Python documentation states that Type[Foo] accepts any subclasses of Foo (https://docs.python.org/3/library/typing.html#typing.Type). I have multiple subclasses of NamedTuple, for entities in our data model, so I'm looking for a way to annotate the function in a way that would typecheck.
The root issue with your code is that NamedTuple is not an actual type -- it's actually just a special sort of "type constructor" that synthesizes an entirely new class and type. E.g. if you try printing out the value of Foo.__mro__, you'll see (<class '__main__.Foo'>, <class 'tuple'>, <class 'object'>) -- NamedTuple is not present there at all.
That means that NamedTuple isn't actually a valid type to use at all -- in that regard, it's actually a little surprising to me that mypy just silently lets you construct Type[NamedTuple] to begin with.
To work around this, you have several potential approaches:
Rather then using Type[NamedTuple], use either Type[tuple] or Type[Tuple[Any]].
Your Foo genuinely is a subtype of a tuple, after all.
If you need methods or fields that are specifically present only in namedtuples, use a custom protocol. For example, if you particularly need the _asdict method in namedtuples, you could do:
from typing_extensions import Protocol
class NamedTupleProto(Protocol):
def _asdict(self) -> Dict[str, Any]: ...
def to_schema(x: Type[NamedTupleProto]) -> None: pass
class Foo(NamedTuple):
pass
to_schema(Foo)
Note that you will need to install the typing_extensions third party library to use this, though there are plans to formalize Protocols and add it to Python at some point. (I forget if the plan was Python 3.7 or 3.8).
Add a type ignore or a cast on the call to to_schema to silence mypy. This isn't the greatest solution, but is also the quickest.
For related discussion, see this issue. Basically, there's consensus on the mypy team that somebody ought to do something about this NamedTuple thing, whether it's by adding an error message or by adding an officially sanctioned protocol, but I think people are too busy with other tasks/bugs to push this forward. (So if you're bored and looking for something to do...)
Assume that class MyClass is sometimes, but not always, defined. I have a function foo(a=None) in which argument a can be None, a string, or an object of MyClass.
My question is: If MyClass is not defined in my Python session, how can I check the type of argument a in a fashion similar to isinstance without getting a NameError?
Note on duck-typing: I am deliberately limiting the function.
I'm using Python 2.6.x and Updating is not an option. A forward-compatible solution (especially for 2.7.x) is highly appreciated.
I would suggest a different approach: polyfill the class so all code that wants to refer to it can simply do so:
try:
from foo import Bar # load the native class
except ImportError:
class Bar:
pass # implement necessary parts here
You can put this into your own module and then from mymodule import Bar everywhere it's needed. That allows all your code to use Bar regardless of whether it's defined natively or not.
Even if redefining the class isn't your preferred way to handle this, handling the ImportError is still the way to handle this situation, since you will have to import the class either way and that's where the error will occur. Instead of defining the class, you may instead want to set a class_exists = False flag or something.
If MyClass isn't defined then you have no way to reference its type.
Therefore you can have no way to verify that type(a) has the correct value.
I workarounded the problem by overriding a method in MyClass and doing nothing in it (pass). After that I no longer needed to check its type.
Different workarounds may exist for different cases. Catching the NameError could be another one.
t = 'asdfas'
print(isinstance(t, str))
try:
print(isinstance(t, MyClass))
except NameError:
print(False)
Seems to me, that such a construct may appear in future python. Like typed python, which is quite new. And in typed python we have a possibility to use future types, in apos.
I'm trying to mimic methods.grep from Ruby which simply returns a list of available methods for any object (class or instance) called upon, filtered by regexp pattern passed to grep.
Very handy for investigating objects in an interactive prompt.
def methods_grep(self, pattern):
""" returns list of object's method by a regexp pattern """
from re import search
return [meth_name for meth_name in dir(self) \
if search(pattern, meth_name)]
Because of Python's limitation not quite clear to me it unfortunately can't be simply inserted in the object class ancestor:
object.mgrep = classmethod(methods_grep)
# TypeError: can't set attributes of built-in/extension type 'object'
Is there some workaround how to inject all classes or do I have to stick with a global function like dir ?
There is a module called forbiddenfruit that enables you to patch built-in objects. It also allows you to reverse the changes. You can find it here https://pypi.python.org/pypi/forbiddenfruit/0.1.1
from forbiddenfruit import curse
curse(object, "methods_grep", classmethod(methods_grep))
Of course, using this in production code is likely a bad idea.
There is no workaround AFAIK. I find it quite annoying that you can't alter built-in classes. Personal opinion though.
One way would be to create a base object and force all your objects to inherit from it.
But I don't see the problem to be honest. You can simply use methods_grep(object, pattern), right? You don't have to insert it anywhere.