Python Adding Methods at runtime - python

I'm trying to dynamically add methods to classes at runtime, and am seeing some issues:
#Here we define a set of symbols within an exec statement and put them into the dictionary d
d = {}
exec "def get_%s(self): return self.%s" % (attr_name, attr) in d
#Now, we bind the get method stored in d['get_%s'] to our object (class)
func = d['get_%s' % (attr_name)].__get__(d['get_%s' % (attr_name)], class)
setattr(class_instance, func.__name__, func)
When I try to call the generated get method, I see the below:
Traceback (most recent call last):
File "Textere_AdvancedExample.py", line 77, in <module>
count = fact.get_counter()
File "<string>", line 1, in get_counter
AttributeError: 'function' object has no attribute '_counter'
Edit
Based on some of the exceptional responses given so far, I think I need to clarify why I'm doing things this way.
I'm trying to build an annotation like the below example:
#getters
#singleton
class A() {
def __init__(self):
self._a = "a"
self._b = "b"
}
Based on the names present in the class, the annotation will build getters for the private class variables at runtime and bind them to the singleton instance.
The strategy I've taken is to have an Application Context class with a set of dicts. Then, the context is passed in to the annotation, which adds the instance & class into these dicts.
On startup, the Application Context is then responsible for reading the dictionaries and then building & binding get methods to the respective singleton object.
Edit2
So this development started after some discussions with friends of mine who are Java developers regarding two libraries in particular: Spring & Lombok
I wanted to see if these particular pieces of functionality could be implemented in Python. So the application context came about originally from trying to get a functionality similar to Spring's autowire annotation. I got this working without issue.
Then, I got the generating the getters and setters and realized that I was going to have a fundamental difference in Python from the Java implementation: Lombok does this at compile time and Python is not compiled. This meant that I had to dynamically generate methods based on what's being annotated and bind them to objects manually, all at runtime. Thus, you see this sort of warping of the Java implementation.
For those interested, The full code can be found here

You can easily dynamically add static methods or class methods:
class A:
pass
#staticmethod
def foo0(x):
return x * 2
a = A()
A.foo = foo0
a.foo(3)
return 6
class A:
val = 3
#classmethod
def foo0(cls, x):
return x * cls.val
a = A()
A.foo = foo0
a.foo(2)
return 6
You can also add specific methods to an instance of a class (almost the same way)
class A:
pass
a = A()
a.foo = (lambda x: 2*x)
a.foo(3)
returns 6
You can also add an instance method to a class, through the use of the types module (in fact, this generic way can be used to create also static and class methods, as well as instance only methods):
class A:
def __init__(self, val):
self.val = val
a = A(3)
A.foo = types.MethodType((lambda self, x: self.val * x), None, A)
a.foo(2)
returns 6
But this is really monkey patching, that is a quick and dirty hack that should only be used when you need to pass slightly changed classes and you are not allowed to change the name. The nice and clean way to add functionalities to a class is inheritance
Just to make this answer better, the above is valid for Python 2.
For Python 3, only the first way to create class and static method can be used, because the types module has lost many types.
And you create an instance method as simply as:
class A:
def __init__(self, val):
self.val = val
a = A(3)
A.foo = (lambda self, x: self.val * x)
a.foo(2)
returns 6. No need for special construct here

exec("%s.__dict__['get_%s'] = lambda self: self.%s" % (class_name, attr_name, attr_name))

Related

Create a link between 2 classes

Is there any way to connect 2 classes (without merging them in 1) and thus avoiding repetition under statement if a: in class Z?
class A:
def __init__(self, a):
self.a = a
self.b = self.a + self.a
class Z:
def __init__(self, z, a=None):
self.z = z
if a: # this part seems like repetition
self.a = a.a
self.b = a.b
a = A('hello')
z = Z('world', a)
assert z.a == a.a # hello
assert z.b == a.b # hellohello
Wondering if python has some tools. I would prefer to avoid loop over instance variables and using setattr.
Something like inheriting from class A to class Z, Z(A) or such.
Here's a trivial example of class inheritance that may help you to understand:
class A:
def __init__(self, a):
self._a = a
self._b = self.a + self.a
class Z(A):
def __init__(self, z, a):
super().__init__(a)
self._z = z
clazz = Z('Hello', 'world')
print(clazz._z, clazz._a)
Conceptually, the standard techniques for associating an A instance with a Z instance are:
Using composition (and delegation)
"Composition" simply means that the A instance itself is an attribute of the Z instance. We call this a "has-a" relationship: every Z has an A that's associated with it.
In normal cases, we can simply pass the A instance to the Z constructor, and have it assign an attribute in __init__. Thus:
class A:
def __init__(self, a):
self.a = a
self.b = self.a + self.a
def action(self): # added for demonstration purposes.
pass
class Z:
def __init__(self, z, a=None):
self.z = z
self._a = a # if not None, this will be an `A` instance
Notice that the attribute for the a instance is specially named to avoid conflicting with the A class attribute names. This is to avoid ambiguity (calling it .a makes one wonder whether my_z.a should get the .a attribute from the A instance, or the entire instance), and to mark it as an implementation detail (normally, outside code won't have a good reason to get the entire A instance out of the Z; the entire point of delegation is to make it so that users of Z don't have to worry about A's interface).
One important limitation is that the composition relationship is one-way by nature: self._a = a gives the Z class access to A contents, but not the other way around. (Of course, it's also possible to build the relationship in both directions, but this will require some planning ahead.)
"Delegation" means that we use some scheme in the code, so that looking something up in a Z instance finds it in the composed A instance when necessary. There are multiple ways to achieve this in Python, at least two of which are worth mentioning:
Explicit delegation per attribute
We define a separate property in the Z class, for each attribute we want to delegate. For example:
# within the `Z` class
#property
def a(self):
return self._a.a
# The setter can also be omitted to make a read-only attribute;
# alternately, additional validation logic can be added to the function.
#a.setter
def a(self, value):
self._a.a = value
For methods, using the same property approach should work, but it may be simpler to make a wrapper function and calling it:
def action(self):
return self._a.action()
Delegation via __getattr__
The __getattr__ magic ("dunder") method allows us to provide fallback logic for looking up an attribute in a class, if it isn't found by the normal means. We can use this for the Z class, so that it will try looking within its _a if all else fails. This looks like:
def __getattr__(self, name):
return getattr(self._a, name)
Here, we use the free function getattr to look up the name dynamically within the A instance.
Using inheritance
This means that each Z instance will, conceptually, be a kind of A instance - classes represent types, and inheriting Z from A means that it will be a subtype of A.
We call this an "is-a" relationship: every Z instance is an A instance. More precisely, a Z instance should be usable anywhere that an A instance could be used, but also Z might contain additional data and/or use different implementations.
This approach looks like:
class A:
def __init__(self, a):
self.a = a
self.b = self.a + self.a
def action(self): # added for demonstration purposes.
return f'{self.z.title()}, {self.a}!'
class Z(A):
def __init__(self, z, a):
# Use `a` to do the `A`-specific initialization.
super().__init__(a)
# Then do `Z`-specific initialization.
self.z = z
The super function is magic that finds the A.__init__ function, and calls it as a method on the Z instance that's currently being initialized. (That is: self will be the same object for both __init__ calls.)
This is clearly more convenient than the delegation and composition approach. Our Z instance actually has a and b attributes as well as z, and also actually has a action method. Thus, code like my_z.action() will use the method from the A class, and accessing the a and b attributes of a Z instance works - because the Z instance actually directly contains that data.
Note in this example that the code for action now tries to use self.z. this won't work for an A instance constructed directly, but it does work when we construct a Z and call action on it:
>>> A('world').action()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in action
AttributeError: 'A' object has no attribute 'z'
>>> Z('hello', 'world').action()
'Hello, world!'
We say that such an A class, which doesn't properly function on its own, is abstract. (There are more tools we can use to prevent accidentally creating an unusable base A; these are outside the scope of this answer.)
This convenience comes with serious implications for design. It can be hard to reason about deep inheritance structures (where the A also inherits from B, which inherits from C...) and especially about multiple inheritance (Z can inherit from B as well as A). Doing these things requires careful planning and design, and a more detailed understanding of how super works - beyond the scope of this answer.
Inheritance is also less flexible. For example, when the Z instance composes an A instance, it's easy to swap that A instance out later for another one. Inheritance doesn't offer that option.
Using mixins
Essentially, using a mixin means using inheritance (generally, multiple inheritance), even though we conceptually want a "has-a" relationship, because the convenient usage patterns are more important than the time spent designing it all up front. It's a complex, but powerful design pattern that essentially lets us build a new class from component parts.
Typically, mixins will be abstract (in the sense described in the previous section). Most examples of mixins also won't contain data attributes, but only methods, because they're generally designed specifically to implement some functionality. (In some programming languages, when using multiple inheritance, only one base class is allowed to contain data. However, this restriction is not necessary and would make no sense in Python, because of how objects are implemented.)
One specific technique common with mixins is that the first base class listed will be an actual "base", while everything else is treated as "just" an abstract mixin. To keep things organized while initializing all the mixins based on the original Z constructor arguments, we use keyword arguments for everything that will be passed to the mixins, and let each mixin use what it needs from the **kwargs.
class Root:
# We use this to swallow up any arguments that were passed "too far"
def __init__(self, *args, **kwargs):
pass
class ZBase(Root):
def __init__(self, z, **kwargs):
# a common pattern is to just accept arbitrary keyword arguments
# that are passed to all the mixins, and let each one sort out
# what it needs.
super().__init__(**kwargs)
self.z = z
class AMixin(Root):
def __init__(self, **kwargs):
# This `super()` call is in case more mixins are used.
super().__init__(**kwargs)
self.a = kwargs['a']
self.b = self.a + self.a
def func(self): # This time, we'll make it do something
return f'{self.z.title()}, {self.a}!'
# We combine the base with the mixins by deriving from both.
# Normally there is no reason to add any more logic here.
class Z(ZBase, AMixin): pass
We can use this like:
>>> # we use keyword arguments for all the mixins' arguments
>>> my_z = Z('hello', a='world')
>>> # now the `Z` instance has everything defined in both base and mixin:
>>> my_z.func()
'Hello, world!'
>>> my_z.z
'hello'
>>> my_z.a
'world'
>>> my_z.b
'worldworld'
The code in AMixin can't stand on its own:
>>> AMixin(a='world').func()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 8, in func
AttributeError: 'AMixin' object has no attribute 'z'
but when the Z instance has both ZBase and AMixin as bases, and is used to call func, the z attribute can be found - because now self is a Z instance, which has that attribute.
The super logic here is a bit tricky. The details are beyond the scope of this post, but suffice to say that with mixin classes that are set up this way, super will forward to the next, sibling base of Z, as long as there is one. It will do this no matter what order the mixins appear in; the Z instance determines the order, and super calls whatever is "next in line". When all the bases have been consulted, next in line is Root, which is just there to intercept the kwargs (since the last mixin doesn't "know" it's last, and passes them on). This is necessary because otherwise, next in line would be object, and object.__init__ raises an exception if there are any arguments.
For more details, see What is a mixin and why is it useful?.

Promote instantiated class/object to a class in python?

Is there are a way in Python to store instantiated class as a class 'template' (aka promote object to a class) to create new objects of same type with same fields values, without relying on using data that was used to create original object again or on copy.deepcopy?
Like, for example I have the dictionary:
valid_date = {"date":"30 february"} # dict could have multiple items
and I have the class:
class AwesomeDate:
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val);
I create the instance of the class like:
totally_valid_date = AwesomeDate(valid_date)
print(totally_valid_date.date) # output: 30 february
and now I want to use it to create new instances of the AwesomeDate class using the totally_valid_date instance as a template, i.e. like:
how_make_it_work = totally_valid_date()
print(how_make_it_work.date) # should print: 30 february
Is there are way to do so or no? I need a generic solution, not a solution for this specific example.
I don't really see the benefit of having a class act both as a template to instances, and as the instance itself, both conceptually and coding-wise. In my opinion, you're better off using two different classes - one for the template, one for the objects it is able to create.
You can think about awesome_date as a template class that stores the valid_date attributes upon initialization. Once called, the template returns an instance of a different class that has the expected attributes.
Here's a simple implementation (names have been changed to generalize the idea):
class Thing:
pass
class Template:
def __init__(self, template_attrs):
self.template_attrs = template_attrs
def __call__(self):
instance = Thing()
for key, val in self.template_attrs.items():
setattr(instance, key, val)
return instance
attrs = {'date': '30 february'}
template = Template(template_attrs=attrs)
# Gets instance of Thing
print(template()) # output: <__main__.Thing object at 0x7ffa656f8668>
# Gets another instance of Thing and accesses the date attribute
print(template().date) # output: 30 february
Yes, there are ways to do it -
there could even be some tweaking of inheriting from type and meddling with __call__ to make all instances automatically become derived classes. But I don't think that would be very sane. Python's own enum.Enum does something along this, because it has some use for the enum values - but the price is it became hard to understand beyond the basic usage, even for seasoned Pythonistas.
However, having a custom __init_subclass__ method that can inject some code to run prior to __init__ on the derived class, and then a method that will return a new class bound with the data that the new classes should have, can suffice:
import copy
from functools import wraps
def wrap_init(init):
#wraps(init)
def wrapper(self, *args, **kwargs):
if not getattr(self, "_initalized", False):
self.__dict__.update(self._template_data or {})
self._initialized = True
return init(self, *args, **kwargs)
wrapper._template_wrapper = True
return wrapper
class TemplateBase:
_template_data = None
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
if getattr(cls.__init__, "_template_wraper", False):
return
init = cls.__init__
cls.__init__ = wrap_init(init)
def as_class(self):
cls= self.__class__
new_cls = type(cls.__name__ + "_templated", (cls,), {})
new_cls._template_data = copy.copy(self.__dict__)
return new_cls
And using it:
class AwesomeDate(TemplateBase):
def __init__(self, dates_dict):
for key, val in dates_dict.items():
setattr(self, key, val)
On the REPL we have:
In [34]: x = AwesomeDate({"x":1, "y":2})
In [35]: Y = x.as_class()
In [36]: y = Y({})
In [37]: y.x
Out[37]: 1
Actually, __init_subclass__ itself could be supressed, and decorating __init__ could be done in one shot on the as_class method. This code takes some care so that mixin classes can be used, and it will still work.
It seems like you are going for something along the lines of the prototype design pattern.
What is the prototype design pattern?
From Wikipedia: Prototype pattern
The prototype pattern is a creational design pattern in software development. It is used when the type of objects to create is determined by a prototypical instance, which is cloned to produce new objects. This pattern is used to avoid subclasses of an object creator in the client application, like the factory method pattern does and to avoid the inherent cost of creating a new object in the standard way (e.g., using the 'new' keyword) when it is prohibitively expensive for a given application.
From Refactoring.guru: Prototype
Prototype is a creational design pattern that lets you copy existing objects without making your code dependent on their classes. The Prototype pattern delegates the cloning process to the actual objects that are being cloned. The pattern declares a common interface for all objects that support cloning. This interface lets you clone an object without coupling your code to the class of that object. Usually, such an interface contains just a single clone method.
The implementation of the clone method is very similar in all classes. The method creates an object of the current class and carries over all of the field values of the old object into the new one. You can even copy private fields because most programming languages let objects access private fields of other objects that belong to the same class. An object that supports cloning is called a prototype. When your objects have dozens of fields and hundreds of possible configurations, cloning them might serve as an alternative to subclassing. Here’s how it works: you create a set of objects, configured in various ways. When you need an object like the one you’ve configured, you just clone a prototype instead of constructing a new object from scratch.
Implementing this for your problem, along with your other ideas
From your explanation, it seems like you want to:
Provide a variable containing a dictionary, which will be passed to the __init__ of some class Foo
Instantiate class Foo and pass the variable containing the dictionary as an argument.
Implement __call__ onto class Foo, allowing us to use the function call syntax on an object of class Foo.
The implementation of __call__ will COPY/CLONE the “template” object. We can then do whatever we want with this copied/cloned instance.
The Code (edited)
import copy
class Foo:
def __init__(self, *, template_attrs):
if not isinstance(template_attrs, dict):
raise TypeError("You must pass a dict to instantiate this class.")
self.template_attrs = template_attrs
def __call__(self):
return copy.copy(self)
def __repr__(self):
return f"{self.template_attrs}"
def __setitem__(self, key, value):
self.template_attrs[key] = value
def __getitem__(self, key):
if key not in self.template_attrs:
raise KeyError(f"Key {key} does not exist in '{self.template_attrs=}'.")
return self.template_attrs[key]
err = Foo(template_attrs=1) # Output: TypeError: You must pass a dict to instantiate this class.
# remove err's assignment to have code under it run
base = Foo(template_attrs={1: 2})
print(f"{base=}") # Output: base={1: 2}
base_copy = base()
base_copy["hello"] = "bye"
print(f"{base_copy=}") # Output: base_copy={1: 2, 'hello': 'bye'}
print(f"{base_copy[1]=}") # Output: base_copy[1]=2
print(f"{base_copy[10]=}") # Output: KeyError: "Key 10 does not exist in 'self.template_attrs={1: 2, 'hello': 'bye'}'."
I also added support for subscripting and item assignment through __getitem__ and __setitem__ respectively. I hope that this helped a bit with your problem! Feel free to comment on this if I missed what you were asking.
Reasons for edits (May 16th, 2022 at 8:49 PM CST | Approx. 9 hours after original answer)
Fix code based on suggestions by comment from user jsbueno
Handle, in __getitem__, if an instance of class Foo is subscripted with a key that doesn't exist in the dict.
Handle, in __init__, if the type of template_attrs isn't dict (did this based on the fact that you used a dictionary in the body of your question)

How to Inherit multiple classes in python dynamically [duplicate]

This article has a snippet showing usage of __bases__ to dynamically change the inheritance hierarchy of some Python code, by adding a class to an existing classes collection of classes from which it inherits. Ok, that's hard to read, code is probably clearer:
class Friendly:
def hello(self):
print 'Hello'
class Person: pass
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
That is, Person doesn't inherit from Friendly at the source level, but rather this inheritance relation is added dynamically at runtime by modification of the __bases__attribute of the Person class. However, if you change Friendly and Person to be new style classes (by inheriting from object), you get the following error:
TypeError: __bases__ assignment: 'Friendly' deallocator differs from 'object'
A bit of Googling on this seems to indicate some incompatibilities between new-style and old style classes in regards to changing the inheritance hierarchy at runtime. Specifically: "New-style class objects don't support assignment to their bases attribute".
My question, is it possible to make the above Friendly/Person example work using new-style classes in Python 2.7+, possibly by use of the __mro__ attribute?
Disclaimer: I fully realise that this is obscure code. I fully realize that in real production code tricks like this tend to border on unreadable, this is purely a thought experiment, and for funzies to learn something about how Python deals with issues related to multiple inheritance.
Ok, again, this is not something you should normally do, this is for informational purposes only.
Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:
setattr(Person, '__mro__', (Person, Friendly, object))
The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.
A possible workaround is to simply redefine the class:
def modify_Person_to_be_friendly():
# so that we're modifying the global identifier 'Person'
global Person
# now just redefine the class using type(), specifying that the new
# class should inherit from Friendly and have all attributes from
# our old Person class
Person = type('Person', (Friendly,), dict(Person.__dict__))
def main():
modify_Person_to_be_friendly()
p = Person()
p.hello() # works!
What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):
def main():
oldperson = Person()
ModifyPersonToBeFriendly()
p = Person()
p.hello()
# works! But:
oldperson.hello()
# does not
If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.
I've been struggling with this too, and was intrigued by your solution, but Python 3 takes it away from us:
AttributeError: attribute '__dict__' of 'type' objects is not writable
I actually have a legitimate need for a decorator that replaces the (single) superclass of the decorated class. It would require too lengthy a description to include here (I tried, but couldn't get it to a reasonably length and limited complexity -- it came up in the context of the use by many Python applications of an Python-based enterprise server where different applications needed slightly different variations of some of the code.)
The discussion on this page and others like it provided hints that the problem of assigning to __bases__ only occurs for classes with no superclass defined (i.e., whose only superclass is object). I was able to solve this problem (for both Python 2.7 and 3.2) by defining the classes whose superclass I needed to replace as being subclasses of a trivial class:
## T is used so that the other classes are not direct subclasses of object,
## since classes whose base is object don't allow assignment to their __bases__ attribute.
class T: pass
class A(T):
def __init__(self):
print('Creating instance of {}'.format(self.__class__.__name__))
## ordinary inheritance
class B(A): pass
## dynamically specified inheritance
class C(T): pass
A() # -> Creating instance of A
B() # -> Creating instance of B
C.__bases__ = (A,)
C() # -> Creating instance of C
## attempt at dynamically specified inheritance starting with a direct subclass
## of object doesn't work
class D: pass
D.__bases__ = (A,)
D()
## Result is:
## TypeError: __bases__ assignment: 'A' deallocator differs from 'object'
I can not vouch for the consequences, but that this code does what you want at py2.7.2.
class Friendly(object):
def hello(self):
print 'Hello'
class Person(object): pass
# we can't change the original classes, so we replace them
class newFriendly: pass
newFriendly.__dict__ = dict(Friendly.__dict__)
Friendly = newFriendly
class newPerson: pass
newPerson.__dict__ = dict(Person.__dict__)
Person = newPerson
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
We know that this is possible. Cool. But we'll never use it!
Right of the bat, all the caveats of messing with class hierarchy dynamically are in effect.
But if it has to be done then, apparently, there is a hack that get's around the "deallocator differs from 'object" issue when modifying the __bases__ attribute for the new style classes.
You can define a class object
class Object(object): pass
Which derives a class from the built-in metaclass type.
That's it, now your new style classes can modify the __bases__ without any problem.
In my tests this actually worked very well as all existing (before changing the inheritance) instances of it and its derived classes felt the effect of the change including their mro getting updated.
I needed a solution for this which:
Works with both Python 2 (>= 2.7) and Python 3 (>= 3.2).
Lets the class bases be changed after dynamically importing a dependency.
Lets the class bases be changed from unit test code.
Works with types that have a custom metaclass.
Still allows unittest.mock.patch to function as expected.
Here's what I came up with:
def ensure_class_bases_begin_with(namespace, class_name, base_class):
""" Ensure the named class's bases start with the base class.
:param namespace: The namespace containing the class name.
:param class_name: The name of the class to alter.
:param base_class: The type to be the first base class for the
newly created type.
:return: ``None``.
Call this function after ensuring `base_class` is
available, before using the class named by `class_name`.
"""
existing_class = namespace[class_name]
assert isinstance(existing_class, type)
bases = list(existing_class.__bases__)
if base_class is bases[0]:
# Already bound to a type with the right bases.
return
bases.insert(0, base_class)
new_class_namespace = existing_class.__dict__.copy()
# Type creation will assign the correct ‘__dict__’ attribute.
del new_class_namespace['__dict__']
metaclass = existing_class.__metaclass__
new_class = metaclass(class_name, tuple(bases), new_class_namespace)
namespace[class_name] = new_class
Used like this within the application:
# foo.py
# Type `Bar` is not available at first, so can't inherit from it yet.
class Foo(object):
__metaclass__ = type
def __init__(self):
self.frob = "spam"
def __unicode__(self): return "Foo"
# … later …
import bar
ensure_class_bases_begin_with(
namespace=globals(),
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
Use like this from within unit test code:
# test_foo.py
""" Unit test for `foo` module. """
import unittest
import mock
import foo
import bar
ensure_class_bases_begin_with(
namespace=foo.__dict__,
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
class Foo_TestCase(unittest.TestCase):
""" Test cases for `Foo` class. """
def setUp(self):
patcher_unicode = mock.patch.object(
foo.Foo, '__unicode__')
patcher_unicode.start()
self.addCleanup(patcher_unicode.stop)
self.test_instance = foo.Foo()
patcher_frob = mock.patch.object(
self.test_instance, 'frob')
patcher_frob.start()
self.addCleanup(patcher_frob.stop)
def test_instantiate(self):
""" Should create an instance of `Foo`. """
instance = foo.Foo()
The above answers are good if you need to change an existing class at runtime. However, if you are just looking to create a new class that inherits by some other class, there is a much cleaner solution. I got this idea from https://stackoverflow.com/a/21060094/3533440, but I think the example below better illustrates a legitimate use case.
def make_default(Map, default_default=None):
"""Returns a class which behaves identically to the given
Map class, except it gives a default value for unknown keys."""
class DefaultMap(Map):
def __init__(self, default=default_default, **kwargs):
self._default = default
super().__init__(**kwargs)
def __missing__(self, key):
return self._default
return DefaultMap
DefaultDict = make_default(dict, default_default='wug')
d = DefaultDict(a=1, b=2)
assert d['a'] is 1
assert d['b'] is 2
assert d['c'] is 'wug'
Correct me if I'm wrong, but this strategy seems very readable to me, and I would use it in production code. This is very similar to functors in OCaml.
This method isn't technically inheriting during runtime, since __mro__ can't be changed. But what I'm doing here is using __getattr__ to be able to access any attributes or methods from a certain class. (Read comments in order of numbers placed before the comments, it makes more sense)
class Sub:
def __init__(self, f, cls):
self.f = f
self.cls = cls
# 6) this method will pass the self parameter
# (which is the original class object we passed)
# and then it will fill in the rest of the arguments
# using *args and **kwargs
def __call__(self, *args, **kwargs):
# 7) the multiple try / except statements
# are for making sure if an attribute was
# accessed instead of a function, the __call__
# method will just return the attribute
try:
return self.f(self.cls, *args, **kwargs)
except TypeError:
try:
return self.f(*args, **kwargs)
except TypeError:
return self.f
# 1) our base class
class S:
def __init__(self, func):
self.cls = func
def __getattr__(self, item):
# 5) we are wrapping the attribute we get in the Sub class
# so we can implement the __call__ method there
# to be able to pass the parameters in the correct order
return Sub(getattr(self.cls, item), self.cls)
# 2) class we want to inherit from
class L:
def run(self, s):
print("run" + s)
# 3) we create an instance of our base class
# and then pass an instance (or just the class object)
# as a parameter to this instance
s = S(L) # 4) in this case, I'm using the class object
s.run("1")
So this sort of substitution and redirection will simulate the inheritance of the class we wanted to inherit from. And it even works with attributes or methods that don't take any parameters.

Is there a way to implement methods like __len__ or __eq__ as classmethods?

It is pretty easy to implement __len__(self) method in Python so that it handles len(inst) calls like this one:
class A(object):
def __len__(self):
return 7
a = A()
len(a) # gives us 7
And there are plenty of alike methods you can define (__eq__, __str__, __repr__ etc.).
I know that Python classes are objects as well.
My question: can I somehow define, for example, __len__ so that the following works:
len(A) # makes sense and gives some predictable result
What you're looking for is called a "metaclass"... just like a is an instance of class A, A is an instance of class as well, referred to as a metaclass. By default, Python classes are instances of the type class (the only exception is under Python 2, which has some legacy "old style" classes, which are those which don't inherit from object). You can check this by doing type(A)... it should return type itself (yes, that object has been overloaded a little bit).
Metaclasses are powerful and brain-twisting enough to deserve more than the quick explanation I was about to write... a good starting point would be this stackoverflow question: What is a Metaclass.
For your particular question, for Python 3, the following creates a metaclass which aliases len(A) to invoke a class method on A:
class LengthMetaclass(type):
def __len__(self):
return self.clslength()
class A(object, metaclass=LengthMetaclass):
#classmethod
def clslength(cls):
return 7
print(len(A))
(Note: Example above is for Python 3. The syntax is slightly different for Python 2: you would use class A(object):\n __metaclass__=LengthMetaclass instead of passing it as a parameter.)
The reason LengthMetaclass.__len__ doesn't affect instances of A is that attribute resolution in Python first checks the instance dict, then walks the class hierarchy [A, object], but it never consults the metaclasses. Whereas accessing A.__len__ first consults the instance A, then walks it's class hierarchy, which consists of [LengthMetaclass, type].
Since a class is an instance of a metaclass, one way is to use a custom metaclass:
>>> Meta = type('Meta', (type,), {'__repr__': lambda cls: 'class A'})
>>> A = Meta('A', (object,), {'__repr__': lambda self: 'instance of class A'})
>>> A
class A
>>> A()
instance of class A
I fail to see how the Syntax specifically is important, but if you really want a simple way to implement it, just is the normal len(self) that returns len(inst) but in your implementation make it return a class variable that all instances share:
class A:
my_length = 5
def __len__(self):
return self.my_length
and you can later call it like that:
len(A()) #returns 5
obviously this creates a temporary instance of your class, but length only makes sense for an instance of a class and not really for the concept of a class (a Type object).
Editing the metaclass sounds like a very bad idea and unless you are doing something for school or to just mess around I really suggest you rethink this idea..
try this:
class Lengthy:
x = 5
#classmethod
def __len__(cls):
return cls.x
The #classmethod allows you to call it directly on the class, but your len implementation won't be able to depend on any instance variables:
a = Lengthy()
len(a)

How to do Obj-C Categories in Python?

Obj-C (which I have not used for a long time) has something called categories to extend classes. Declaring a category with new methods and compiling it into your program, all instances of the class suddenly have the new methods.
Python has mixin possibilities, which I use, but mixins must be used from the bottom of the program: the class has to declare it itself.
Foreseen category use-case: Say you have a big class hierarchy that describe different ways of interacting with data, declaring polymorphic ways to get at different attributes. Now a category can help the consumer of these describing classes by implementing a convenient interface to access these methods in one place. (A category method could for example, try two different methods and return the first defined (non-None) return value.)
Any way to do this in Python?
Illustrative code
I hope this clarifies what I mean. The point is that the Category is like an aggregate interface, that the consumer of AppObj can change in its code.
class AppObj (object):
"""This is the top of a big hierarchy of subclasses that describe different data"""
def get_resource_name(self):
pass
def get_resource_location(self):
pass
# dreaming up class decorator syntax
#category(AppObj)
class AppObjCategory (object):
"""this is a category on AppObj, not a subclass"""
def get_resource(self):
name = self.get_resource_name()
if name:
return library.load_resource_name(name)
else:
return library.load_resource(self.get_resource_location())
Why not just add methods dynamically ?
>>> class Foo(object):
>>> pass
>>> def newmethod(instance):
>>> print 'Called:', instance
...
>>> Foo.newmethod = newmethod
>>> f = Foo()
>>> f.newmethod()
Called: <__main__.Foo object at 0xb7c54e0c>
I know Objective-C and this looks just like categories. The only drawback is that you can't do that to built-in or extension types.
I came up with this implementation of a class decorator. I'm using python2.5 so I haven't actually tested it with decorator syntax (which would be nice), and I'm not sure what it does is really correct. But it looks like this:
pycategories.py
"""
This module implements Obj-C-style categories for classes for Python
Copyright 2009 Ulrik Sverdrup <ulrik.sverdrup#gmail.com>
License: Public domain
"""
def Category(toclass, clobber=False):
"""Return a class decorator that implements the decorated class'
methods as a Category on the class #toclass
if #clobber is not allowed, AttributeError will be raised when
the decorated class already contains the same attribute.
"""
def decorator(cls):
skip = set(("__dict__", "__module__", "__weakref__", "__doc__"))
for attr in cls.__dict__:
if attr in toclass.__dict__:
if attr in skip:
continue
if not clobber:
raise AttributeError("Category cannot override %s" % attr)
setattr(toclass, attr, cls.__dict__[attr])
return cls
return decorator
Python's setattr function makes this easy.
# categories.py
class category(object):
def __init__(self, mainModule, override = True):
self.mainModule = mainModule
self.override = override
def __call__(self, function):
if self.override or function.__name__ not in dir(self.mainModule):
setattr(self.mainModule, function.__name__, function)
# categories_test.py
import this
from categories import category
#category(this)
def all():
print "all things are this"
this.all()
>>> all things are this

Categories