I just wrote a simple #autowired decorator for Python that instantiate classes based on type annotations.
To enable lazy initialization of the class, the package provides a lazy(type_annotation: (Type, str)) function so that the caller can use it like this:
#autowired
def foo(bla, *, dep: lazy(MyClass)):
...
This works very well, under the hood this lazy function just returns a function that returns the actual type and that has a lazy_init property set to True. Also this does not break IDEs' (e.g., PyCharm) code completion feature.
But I want to enable the use of a subscriptable Lazy type use instead of the lazy function.
Like this:
#autowired
def foo(bla, *, dep: Lazy[MyClass]):
...
This would behave very much like typing.Union. And while I'm able to implement the subscriptable type, IDEs' code completion feature will be rendered useless as it will present suggestions for attributes in the Lazy class, not MyClass.
I've been working with this code:
class LazyMetaclass(type):
def __getitem__(lazy_type, type_annotation):
return lazy_type(type_annotation)
class Lazy(metaclass=LazyMetaclass):
def __init__(self, type_annotation):
self.type_annotation = type_annotation
I tried redefining Lazy.__dict__ as a property to forward to the subscripted type's __dict__ but this seems to have no effect on the code completion feature of PyCharm.
I strongly believe that what I'm trying to achieve is possible as typing.Union works well with IDEs' code completion. I've been trying to decipher what in the source code of typing.Union makes it to behave well with code completion features but with no success so far.
For the Container[Type] notation to work you would want to create a user-defined generic type:
from typing import TypeVar, Generic
T = TypeVar('T')
class Lazy(Generic[T]):
pass
You then use
def foo(bla, *, dep: Lazy[MyClass]):
and Lazy is seen as a container that holds the class.
Note: this still means the IDE sees dep as an object of type Lazy. Lazy is a container type here, holding an object of type MyClass. Your IDE won't auto-complete for the MyClass type, you can't use it that way.
The notation also doesn't create an instance of the Lazy class; it creates a subclass instead, via the GenericMeta metaclass. The subclass has a special attribute __args__ to let you introspect the subscription arguments:
>>> a = Lazy[str]
>>> issubclass(a, Lazy)
True
>>> a.__args__
(<class 'str'>,)
If all you wanted was to reach into the type annotations at runtime but resolve the name lazily, you could just support a string value:
def foo(bla, *, dep: 'MyClass'):
This is valid type annotation, and your decorator could resolve the name at runtime by using the typing.get_type_hints() function (at a deferred time, not at decoration time), or by wrapping strings in your lazy() callable at decoration time.
If lazy() is meant to flag a type to be treated differently from other type hints, then you are trying to overload the type hint annotations with some other meaning, and type hinting simply doesn't support such use cases, and using a Lazy[...] containing can't make it work.
Related
I'm wondering whether we can use the typing package to produce the definition of an "interface", that is, a class/object in Python 3.
It seems that the usual way to define an "interface" in Python is to use an abstract classdefined using ABC, and use that as your type parameter. However, since Python is dynamically typed, a fully abstract type is an interface that is nothing more than a typing hint for python. In runtime, I would expect to have zero impact from said interface. A base class can have methods that are inherited, and that's not what I want.
I'm based a lot of this on my experience with TypeScript - it enables us to very easily define object types through interface or the type keyword, but those are only used by the type checker.
Let me make my use case clearer with an example:
Let's say I'm defining a function foo as below:
def foo(bar):
nums = [i for i in range(10)]
result = bar.oogle(nums)
return result
foo is, therefore, a method that expects to receive an instance of an object that must have a method oogle that accepts a list of integers. I want to make it clear to callers that this is what foo expects from bar, but bar can be of any type.
PEP544 introduced Protocol classes, which can be used to define interfaces.
from typing import Any, List, Protocol
class Bar(Protocol):
def oogle(self, quz: List[int]) -> Any:
...
def foo(bar: Bar):
nums = [i for i in range(10)]
result = bar.oogle(nums)
return result
If you execute your script using Python you will not see any difference though. You need to run your scripts with Mypy, which is a static type checker that supports protocol classes.
I want to define a function. It has one argument whose value is a class. But not all classes are expected, only some subclass of a certain class can be passed.
I don't know if it is possible to use type hint to achieve this expectation.
If it does, how can I do that?
As far as I know, the type hint can only help when requiring value is a instance of certain type. And the relevant documentation is really poor.
class MyClass(object):
pass
def my_function(arg):
# arg should be the subclass of MyClass.
return
I just want the definition of my_function produce the same effect of what type hint do (the only difference is that normal type hint show value is a instance of certain class, here I want it show value is a subclass of certain class).
If you are trying to say that arg must be an instance of either MyClass or some subtype of MyClass, just do this:
class MyClass(object): pass
class Child(MyClass): pass
def my_function(arg: MyClass) -> None:
pass
my_function(MyClass()) # Type checks
my_function(Child()) # Type checks
my_function(3.14) # Does not type check
If you are trying to say that arg must be literally the MyClass class object or a subtype, use typing.Type:
from typing import Type
# ...snip...
def my_function_2(arg: Type[MyClass]) -> None:
pass
my_function_2(MyClass) # Type checks
my_function_2(Child) # Type checks
my_function_2(int) # Does not type check
my_function_2(MyClass()) # Does not type check
Of course, the type checker won't be able to tell exactly which subtype you happen to be using within the functions themselves.
the type hint can only help when requiring value is a instance of certain type.
This is actually not something you can do. It is impossible to construct a type hint that asserts some value must be exactly one type and never any subtypes of that type (though you can often simulate something close-ish by using generics).
A type hint always indicates that some value is either an instance of that type or some subtype.
And the relevant documentation is really poor.
You may find reading the mypy docs to be useful, if you haven't found them already.
Mypy is a PEP 484 compliant static type checker, and was developed in parallel with PEP 484 itself. (E.g. the core mypy devs were the ones who wrote PEP 484 and several other typing-related PEPs.)
The upshot is that the docs for mypy are generally more helpful than the official Python docs for typing.
The typing module documentation says that the two code snippets below are equivalent.
from typing import NamedTuple
class Employee(NamedTuple):
name: str
id: int
and
from collections import namedtuple
Employee = namedtuple('Employee', ['name', 'id'])
Are they the exact same thing or, if not, what are the differences between the two implementations?
The type generated by subclassing typing.NamedTuple is equivalent to a collections.namedtuple, but with __annotations__, _field_types and _field_defaults attributes added. The generated code will behave the same, for all practical purposes, since nothing in Python currently acts on those typing related attributes (your IDE might use them, though).
As a developer, using the typing module for your namedtuples allows a more natural declarative interface:
You can easily specify default values for the fields (edit: in Python 3.7, collections.namedtuple got a new defaults keyword so this is no longer an advantage)
You don't need to repeat the type name twice ("Employee")
You can customize the type directly (e.g. adding a docstring or some methods)
As before, your class will be a subclass of tuple, and instances will be instances of tuple as usual. Interestingly, your class will not be a subclass of NamedTuple. If you want to know why, read on for more info about the implementation detail.
from typing import NamedTuple
class Employee(NamedTuple):
name: str
id: int
Behaviour in Python <= 3.8
>>> issubclass(Employee, NamedTuple)
False
>>> isinstance(Employee(name='guido', id=1), NamedTuple)
False
typing.NamedTuple is a class, it uses metaclasses and a custom __new__ to handle the annotations, and then it delegates to collections.namedtuple to build and return the type. As you may have guessed from the lowercased name convention, collections.namedtuple is not a type/class - it's a factory function. It works by building up a string of Python source code, and then calling exec on this string. The generated constructor is plucked out of a namespace and included in a 3-argument invocation of the metaclass type to build and return your class. This explains the weird inheritance breakage seen above, NamedTuple uses a metaclass in order to use a different metaclass to instantiate the class object.
Behaviour in Python >= 3.9
typing.NamedTuple is changed from a type (class) to a function (def)
>>> issubclass(Employee, NamedTuple)
TypeError: issubclass() arg 2 must be a class or tuple of classes
>>> isinstance(Employee(name="guido", id=1), NamedTuple)
TypeError: isinstance() arg 2 must be a type or tuple of types
Multiple inheritance using NamedTuple is now disallowed (it did not work properly in the first place).
See bpo40185 / GH-19371 for the change.
Please consider this snippet of python 3.5 code:
class Foo:
pass
class Bar(Foo):
pass
class AbstractSomething:
def get_foobinator_type(self):
return Foo
I'd like to annotate (using PEP-0484 annotations) return value of get_foobinator_type method to say: "It returns a type, that is either a Foo or any subtype of it".
I din't find any sensible way to do it in Python. Here are approaches that are obviously wrong:
Following: def get_foobinator_type(self) -> Foo means that this method returns an instance of Foo.
Following: def get_foobinator_type(self) -> type means that this method returns a type, but sadly, there is no information about that this is needs to be a subtype of Foo.
In Java terms I'd like to have method with signature like: Class<Foo> getFoobinatorType().
As far as I understand, you really cannot. You're looking for a way to indicate the return type of a class; to check based on what the type of the class is, i.e its metaclass.
The problem with that is that a metaclass doesn't help a type checker evaluate what the inheritance of an object might be, if it's of type type it's alright.
Apart from that, and, not being sure what type-checker you use, mypy for example doesn't have support yet for custom metaclasses which you might use to group your objects in a more custom group.
The way I see it, you either don't annotate at all all, or, you change the implementation and annotate with Foo.
Lately (in Python 3.5.2) a generic Type was introduced, so solution is:
class AbstractSomething:
def get_foobinator_type(self) -> typing.Type[Foo]:
return Bar
See: python docs.
I think what you need is TypeVar from the typing module.
from typing import TypeVar
class Foo:
pass
class Bar(Foo):
pass
T = TypeVar('T', bound=Foo)
class AbstractSomething:
def get_foobinator_type(self) -> T:
return Foo
From the documentation of typing:
Alternatively, a type variable may specify an upper bound using
bound=. This means that an actual type substituted (explicitly
or implicitly) for the type variable must be a subclass of the
boundary type, see PEP 484
I'm aware that I can use 'Find Usages' to find what's calling a method in a class.
Is there a way of doing this for all methods on a given class? (or indeed all methods in file)
Use Case: I'm trying to refactor a god class, that should almost certainly be several classes. It would be nice to be able to see what subset of god class methods, the classes that interact with it use. It seems like PyCharm has done the hard bit of this, but doesn't let me scale it up.
I'm using PyCharm 2016.1.2
https://intellij-support.jetbrains.com/hc/en-us/community/posts/206666319-See-all-callers-of-all-methods-of-a-class
This is possible, but you have to deal with abstraction, otherwise Pycharm doesn't know the method in question belongs to your specific class. AKA - Type Hinting
Any instance of that method being called in an abstraction layer which does not have type hinting will not be found.
Example:
#The class which has the method you're searching for.
class Inst(object):
def mymethod(self):
return
#not the class your looking for, but it too has a method of the same name.
class SomethingElse(object):
def mymethod(self):
return
#Option 1 -- Assert hinting
def foo(inst):
assert isinstance(inst, Inst)
inst.mymethod()
#Option 2 -- docstring hinting
def bar(inst):
"""
:param inst:
:type inst: Inst
:return:
:rtype:
"""
inst.mymethod()
Nowadays it would be rather easy for Pycharm to use Python 3.6 type hints and match function calls "correctly", as type hints are part of Python 3.5/3.6 language. Of course partial type hints in a big software cause some compromises when resolving the targets of the method calls.
Here is an example, how type hints makes it very easy to do type inferencing logic and resolve the correct target of the call.
def an_example():
a: SoftagramAnalysisAction = SoftagramAnalysisAction(
analysis_context=analysis_context,
preprocessors=list(preprocessors),
analyzers=list(analyzers),
analysis_control_params=analysis_control_params)
output = a.run()
In above example, local variable a is specially marked to have type SoftagramAnalysisAction which makes it clear that run() call below targets to the run method of that class (or any of its possible subclasses).
The current version (2018.1) does not resolve these kind of calls correctly but I hope that will change in the future.