We Keep Coding

Dynamically Create Methods for Class with list of values

I have a list of values that I want to use for a Builder object implementation that is in the works.
For example:
val_list = ["abc", "def", "ghi"]
What I want to do is dynamically create methods in a class that will allow for these to be callable and retrieved in an instance.
I'm vaguely familiar with doing this with setattr(...) but the next step Im stuck at is being able to do some processing inside the method. In the example below, if I was to do this with my ever growing list, it would a WHOLE BUNCH of code that does literally the same thing. It works for now but I want this list to be dynamic, as well as the class.
For example
def abc(self, value):
self.processing1 = value + "workworkwork"
return self
def def(self, value):
self.processing1 = value + "workworkwork"
return self
def ghi(self, value):
self.processing1 = value + "workworkwork"
return self

I haven't tried this before, but I wonder if it would work using lambdas
self.my_methods = {}
val_list = []
def new_method(self,method_name):
self.my_methods[method_name] = "lambda: self.general_method(some_value)"
def general_method(self, value):
print(value)
Honestly, I'm sure that won't work as written, but hopefully you see the train of thought if it looks of possible interest. Since I can't visualize the overall concept, it's a little tough.
But since it seems that the method name seems important, I'm not sure what to do. Perhaps this is an XY type question? Getting stuck on the how instead of the results?
I would think there has to be a way to make this work:
[Class definition]
...
def method(self,secret_method_name,arg1):
# do something based on method name if necessary
# do something based on args

You can't call a non-existing method on a object without wrapping it first, e.g:
# A legacy class
class dummy:
def foo(self):
return "I'm a dummy!"
obj = dummy()
obj.a("1") # You can't
You can do it using a wrapper class first, here's just a idea of how you can get it done:
# Creates a object you can append methods to
def buildable(cls):
class fake:
# This method will receive the class of the object to build
def __init__(self, cls):
self.cls = cls
# This will simulate a constructor of the underlying class
# Return the fake class so we can call methods on it
def __call__(self, *args, **kwargs):
self.obj = self.cls(*args, **kwargs)
return self
# Will be called whenever a property (existing or non-existing)
# is called on a instance of the fake class
def __getattr__(self, attr):
# If the underlying object has the called attribute,
# just return this attribute
if hasattr(self.obj, attr):
return getattr(self.obj, attr)
# Call the respective function on globals with the provided
# arguments and return the fake class so we can add more methods
def wrapper(*args, **kwargs):
globals()[attr](self.obj, *args, **kwargs)
return self
return wrapper
return fake(cls)
So, how does this work?
Decorate your legacy class:
#buildable
class dummy:
def foo(self):
return "I'm a dummy!"
Create the build methods that'll modify dummy:
def a(self, some):
self.a = some + 'a'
def b(self, some):
self.b = some + 'b'
def c(self, some):
self.c = some + 'c'
Modify it:
obj = dummy()
obj.a("1").b("2").c("3")
See the brand new attributes (and the old ones too!):
print(obj.a) # 1a
print(obj.b) # 2b
print(obj.c) # 3c
print(obj.foo()) # I'm a dummy!
Note that this has some important drawbacks, such as:
Calling a non-existing attribute on dummy will not raise AttributeError:
print(obj.nini) # <function buildable.<locals>.fake.__getattr__.<locals>.wrapper at 0x7f4794e663a0>
You can't do it with multiple objects:
obj1 = dummy()
obj1.a("1").b("2")
print(obj1.a) # 1a
print(obj1.b) # 2b
obj2 = dummy()
obj2.c("3")
print(obj2.c) # 3c
print(obj1.a) # <function buildable.<locals>.fake.__getattr__.<locals>.wrapper at 0x7f524ae16280>
print(obj1.b) # <function buildable.<locals>.fake.__getattr__.<locals>.wrapper at 0x7f524ae16280>
The type of obj will not be dummy:
print(type(obj)) # <class '__main__.buildable.<locals>.fake'>
print(type(obj.obj)) # <class '__main__.dummy'>
You can't call a build method with the same name of an already existing method:
def foo(bar):
self.foo = 'foo' + bar
obj.foo("bar")
print(obj.foo())
# raises TypeError: foo() takes 1 positional argument but 2 were
You can't do it with built-in classes:
list = buildable(list)
obj = list()
obj.a("4").b("5").c("6")
# raises AttributeError: 'list' object has no attribute 'a'

Dynamically generate method from string?

I have a dict of different types for which I want to add a simple getter based on the name of the actual parameter.
For example, for three storage parameters, let's say:
self.storage = {'total':100,'used':88,'free':1}
I am looking now for a way (if possible?) to generate a function on the fly with some meta-programming magic.
Instead of
class spaceObj(object):
def getSize(what='total'):
return storage[what]
or hard coding
#property
def getSizeTotal():
return storage['total']
but
class spaceObj(object):
# manipulting the object's index and magic
#property
def getSize:
return ???
so that calling mySpaceObj.getSizeFree would be derived - with getSize only defined once in the object and related functions derived from it by manipulating the objects function list.
Is something like that possible?

While certainly possible to get an unknown attribute from a class as a property, this is not a pythonic approach (__getattr__ magic methods are rather rubyist)
class spaceObj(object):
storage = None
def __init__(self): # this is for testing only
self.storage = {'total':100,'used':88,'free':1}
def __getattr__(self, item):
if item[:7] == 'getSize': # check if an undefined attribute starts with this
return self.getSize(item[7:])
def getSize(self, what='total'):
return self.storage[what.lower()]
print (spaceObj().getSizeTotal) # 100

You can put the values into the object as properties:
class SpaceObj(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
storage = {'total':100,'used':88,'free':1}
o = SpaceObj(**storage)
print o.total
or
o = SpaceObj(total=100, used=88, free=1)
print o.total
or using __getattr__:
class SpaceObj(object):
def __init__(self, **kwargs):
self.storage = kwargs
def __getattr__(self,name):
return self.storage[name]
o = SpaceObj(total=100, used=88, free=1)
print o.total
The latter approach takes a bit more code but it's more safe; if you have a method foo and someone create the instance with SpaceObj(foo=1), then the method will be overwritten with the first approach.

>>> import new
>>> funcstr = "def wat(): print \"wat\";return;"
>>> funcbin = compile(funcstr,'','exec')
>>> ns = {}
>>> exec funcbin in ns
>>> watfunction = new.function(ns["wat"].func_code,globals(),"wat")
>>> globals()["wat"]=watfunction
>>> wat()
wat

How to share same method on two different classes in python

I have class:
class A(object):
def do_computing(self):
print "do_computing"
Then I have:
new_class = type('B', (object,), {'a': '#A', 'b': '#B'})
What I want to achieve is to make all methods and properties on class A a member of class B. Class A can have from 0 to N such elements. I want to make them all a member of class B.
So far I get to:
methods = {}
for el in dir(A):
if el.startswith('_'):
continue
tmp = getattr(A, el)
if isinstance(tmp, property):
methods[el] = tmp
if isinstance(tmp, types.MethodType):
methods[el] = tmp
instance_class = type('B', (object,), {'a': '#A', 'b': '#B'})
for name, func in methods.items():
new_method = types.MethodType(func, None, instance_class)
setattr(instance_class, name, new_method)
But then when I run:
instance().do_computing()
I get an error:
TypeError: unbound method do_computing() must be called with A instance as first argument (got B instance instead)
Why I had to do that? We have a lot of legacy code and I need fancy objects that will pretend they are old objects but really.
One more important thing. I cannot use inheritance, to much magic happens in the background.

If you do it like this, it will work:
import types
class A(object):
def do_computing(self):
print "do_computing"
methods = {name:value for name, value in A.__dict__.iteritems()
if not name.startswith('_')}
instance_class = type('B', (object,), {'a': '#A', 'b': '#B'})
for name, func in methods.iteritems():
new_method = types.MethodType(func, None, instance_class)
setattr(instance_class, name, new_method)
instance_class().do_computing()

Unless I'm missing something, you can do this with inheritance:
class B(A):
def __init__(self):
super(B, self).__init__()
Then:
>>> b = B()
>>> b.do_computing()
do_computing
Edit: cms_mgr said the same in the comments, also fixed indentation

are you creating a facade? maybe you want something like this:
Making a facade in Python 2.5
http://en.wikipedia.org/wiki/Facade_pattern
you could also use delegators. here's an example from the wxpython AGW:
_methods = ["GetIndent", "SetIndent", "GetSpacing", "SetSpacing", "GetImageList", "GetStateImageList",
"GetButtonsImageList", "AssignImageList", "AssignStateImageList", "AssignButtonsImageList",
"SetImageList", "SetButtonsImageList", "SetStateImageList", 'other_methods']
def create_delegator_for(method):
"""
Creates a method that forwards calls to `self._main_win` (an instance of :class:`TreeListMainWindow`).
:param `method`: one method inside the :class:`TreeListMainWindow` local scope.
"""
def delegate(self, *args, **kwargs):
return getattr(self._main_win, method)(*args, **kwargs)
return delegate
# Create methods that delegate to self._main_win. This approach allows for
# overriding these methods in possible subclasses of HyperTreeList
for method in _methods:
setattr(HyperTreeList, method, create_delegator_for(method))
Note that these wrap class methods... i.e both functions take a signature like def func(self, some, other, args) and are intended to be called like self.func(some, args). If you want to delegate a class function to a non-class function, you'll need to modify the delegator.

You can inherit from a parent class as such:
class Awesome():
def method_a():
return "blee"
class Beauty(Awesome):
def __init__(self):
self.x = self.method_a()
b = Beauty()
print(b.x)
>>> "blee"
This was freely typed, but the logic is the same none the less and should work.
You can also do fun things with setattr like so:
#as you can see this class is worthless and is nothing
class blee():
pass
b = blee()
setattr(b, "variable_1", "123456")
print(b.variable_1)
>>> 123456
essentially you can assign any object, method to a class instance with setattr.
EDIT: Just realized that you did use setattr, woops ;)
Hope this helps!

Python: how to implement __getattr__()?

My class has a dict, for example:
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
Then I want to use the dict's key with MyClass instance to access the dict, for example:
ob = MyClass()
v = ob.a # Here I expect ob.a returns 'v1'
I know this should be implemented by __getattr__, but I'm new to Python, I don't exactly know how to implement it.

class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
return self.data[attr]
>>> ob = MyClass()
>>> v = ob.a
>>> v
'v1'
Be careful when implementing __setattr__ though, you will need to make a few modifications:
class MyClass(object):
def __init__(self):
# prevents infinite recursion from self.data = {'a': 'v1', 'b': 'v2'}
# as now we have __setattr__, which will call __getattr__ when the line
# self.data[k] tries to access self.data, won't find it in the instance
# dictionary and return self.data[k] will in turn call __getattr__
# for the same reason and so on.... so we manually set data initially
super(MyClass, self).__setattr__('data', {'a': 'v1', 'b': 'v2'})
def __setattr__(self, k, v):
self.data[k] = v
def __getattr__(self, k):
# we don't need a special call to super here because getattr is only
# called when an attribute is NOT found in the instance's dictionary
try:
return self.data[k]
except KeyError:
raise AttributeError
>>> ob = MyClass()
>>> ob.c = 1
>>> ob.c
1
If you don't need to set attributes just use a namedtuple
eg.
>>> from collections import namedtuple
>>> MyClass = namedtuple("MyClass", ["a", "b"])
>>> ob = MyClass(a=1, b=2)
>>> ob.a
1
If you want the default arguments you can just write a wrapper class around it:
class MyClass(namedtuple("MyClass", ["a", "b"])):
def __new__(cls, a="v1", b="v2"):
return super(MyClass, cls).__new__(cls, a, b)
or maybe it looks nicer as a function:
def MyClass(a="v1", b="v2", cls=namedtuple("MyClass", ["a", "b"])):
return cls(a, b)
>>> ob = MyClass()
>>> ob.a
'v1'

Late to the party, but found two really good resources that explain this better (IMHO).
As explained here, you should use self.__dict__ to access fields from within __getattr__, in order to avoid infinite recursion. The example provided is:
def __getattr__(self, attrName):
if not self.__dict__.has_key(attrName):
value = self.fetchAttr(attrName) # computes the value
self.__dict__[attrName] = value
return self.__dict__[attrName]
Note: in the second line (above), a more Pythonic way would be (has_key apparently was even removed in Python 3):
if attrName not in self.__dict__:
The other resource explains that the __getattr__ is invoked only when the attribute is not found in the object, and that hasattr always returns True if there is an implementation for __getattr__. It provides the following example, to demonstrate:
class Test(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
def __getattr__(self, name):
return 123456
t = Test()
print 'object variables: %r' % t.__dict__.keys()
#=> object variables: ['a', 'b']
print t.a
#=> a
print t.b
#=> b
print t.c
#=> 123456
print getattr(t, 'd')
#=> 123456
print hasattr(t, 'x')
#=> True

class A(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
try:
return self.data[attr]
except Exception:
return "not found"
>>>a = A()
>>>print a.a
v1
>>>print a.c
not found

I like to take this therefore.
I took it from somewhere, but I don't remember where.
class A(dict):
def __init__(self, *a, **k):
super(A, self).__init__(*a, **k)
self.__dict__ = self
This makes the __dict__ of the object the same as itself, so that attribute and item access map to the same dict:
a = A()
a['a'] = 2
a.b = 5
print a.a, a['b'] # prints 2 5

I figured out an extension to #glglgl's answer that handles nested dictionaries and dictionaries insides lists that are in the original dictionary:
class d(dict):
def __init__(self, *a, **k):
super(d, self).__init__(*a, **k)
self.__dict__ = self
for k in self.__dict__:
if isinstance(self.__dict__[k], dict):
self.__dict__[k] = d(self.__dict__[k])
elif isinstance(self.__dict__[k], list):
for i in range(len(self.__dict__[k])):
if isinstance(self.__dict__[k][i], dict):
self.__dict__[k][i] = d(self.__dict__[k][i])

A simple approach to solving your __getattr__()/__setattr__() infinite recursion woes
Implementing one or the other of these magic methods can usually be easy. But when overriding them both, it becomes trickier. This post's examples apply mostly to this more difficult case.
When implementing both these magic methods, it's not uncommon to get stuck figuring out a strategy to get around recursion in the __init__() constructor of classes. This is because variables need to be initialized for the object, but every attempt to read or write those variables go through __get/set/attr__(), which could have more unset variables in them, incurring more futile recursive calls.
Up front, a key point to remember is that __getattr__() only gets called by the runtime if the attribute can't be found on the object already. The trouble is to get attributes defined without tripping these functions recursively.
Another point is __setattr__() will get called no matter what. That's an important distinction between the two functions, which is why implementing both attribute methods can be tricky.
This is one basic pattern that solves the problem.
class AnObjectProxy:
_initialized = False # *Class* variable 'constant'.
def __init__(self):
self._any_var = "Able to access instance vars like usual."
self._initialized = True # *instance* variable.
def __getattr__(self, item):
if self._initialized:
pass # Provide the caller attributes in whatever ways interest you.
else:
try:
return self.__dict__[item] # Transparent access to instance vars.
except KeyError:
raise AttributeError(item)
def __setattr__(self, key, value):
if self._initialized:
pass # Provide caller ways to set attributes in whatever ways.
else:
self.__dict__[key] = value # Transparent access.
While the class is initializing and creating it's instance vars, the code in both attribute functions permits access to the object's attributes via the __dict__ dictionary transparently - your code in __init__() can create and access instance attributes normally. When the attribute methods are called, they only access self.__dict__ which is already defined, thus avoiding recursive calls.
In the case of self._any_var, once it's assigned, __get/set/attr__() won't be called to find it again.
Stripped of extra code, these are the two pieces that are most important.
... def __getattr__(self, item):
... try:
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __setattr__(self, key, value):
... self.__dict__[key] = value
Solutions can build around these lines accessing the __dict__ dictionary. To implement an object proxy, two modes were implemented: initialization and post-initialization in the code before this - a more detailed example of the same is below.
There are other examples in answers that may have differing levels of effectiveness in dealing with all aspects of recursion. One effective approach is accessing __dict__ directly in __init__() and other places that need early access to instance vars. This works but can be a little verbose. For instance,
self.__dict__['_any_var'] = "Setting..."
would work in __init__().
My posts tend to get a little long-winded.. after this point is just extra. You should already have the idea with the examples above.
A drawback to some other approaches can be seen with debuggers in IDE's. They can be overzealous in their use of introspection and produce warning and error recovery messages as you're stepping through code. You can see this happening even with solutions that work fine standalone. When I say all aspects of recursion, this is what I'm talking about.
The examples in this post only use a single class variable to support 2-modes of operation, which is very maintainable.
But please NOTE: the proxy class required two modes of operation to set up and proxy for an internal object. You don't have to have two modes of operation.
You could simply incorporate the code to access the __dict__ as in these examples in whatever ways suit you.
If your requirements don't include two modes of operation, you may not need to declare any class variables at all. Just take the basic pattern and customize it.
Here's a closer to real-world (but by no means complete) example of a 2-mode proxy that follows the pattern:
>>> class AnObjectProxy:
... _initialized = False # This class var is important. It is always False.
... # The instances will override this with their own,
... # set to True.
... def __init__(self, obj):
... # Because __getattr__ and __setattr__ access __dict__, we can
... # Initialize instance vars without infinite recursion, and
... # refer to them normally.
... self._obj = obj
... self._foo = 123
... self._bar = 567
...
... # This instance var overrides the class var.
... self._initialized = True
...
... def __setattr__(self, key, value):
... if self._initialized:
... setattr(self._obj, key, value) # Proxying call to wrapped obj.
... else:
... # this block facilitates setting vars in __init__().
... self.__dict__[key] = value
...
... def __getattr__(self, item):
... if self._initialized:
... attr = getattr(self._obj, item) # Proxying.
... return attr
... else:
... try:
... # this block facilitates getting vars in __init__().
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __call__(self, *args, **kwargs):
... return self._obj(*args, **kwargs)
...
... def __dir__(self):
... return dir(self._obj) + list(self.__dict__.keys())
The 2-mode proxy only needs a bit of "bootstrapping" to access vars in its own scope at initialization before any of its vars are set. After initialization, the proxy has no reason to create more vars for itself, so it will fare fine by deferring all attribute calls to it's wrapped object.
Any attribute the proxy itself owns will still be accessible to itself and other callers since the magic attribute functions only get called if an attribute can't be found immediately on the object.
Hopefully this approach can be of benefit to anyone who appreciates a direct approach to resolving their __get/set/attr__() __init__() frustrations.

You can initialize your class dictionary through the constructor:
def __init__(self,**data):
And call it as follows:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
All of the instance attributes being accessed (read) in __setattr__, need to be declared using its parent (super) method, only once:
super().__setattr__('NewVarName1', InitialValue)
Or
super().__setattr__('data', dict())
Thereafter, they can be accessed or assigned to in the usual manner:
self.data = data
And instance attributes not being accessed in __setattr__, can be declared in the usual manner:
self.x = 1
The overridden __setattr__ method must now call the parent method inside itself, for new variables to be declared:
super().__setattr__(key,value)
A complete class would look as follows:
class MyClass(object):
def __init__(self, **data):
# The variable self.data is used by method __setattr__
# inside this class, so we will need to declare it
# using the parent __setattr__ method:
super().__setattr__('data', dict())
self.data = data
# These declarations will jump to
# super().__setattr__('data', dict())
# inside method __setattr__ of this class:
self.x = 1
self.y = 2
def __getattr__(self, name):
# This will callback will never be called for instance variables
# that have beed declared before being accessed.
if name in self.data:
# Return a valid dictionary item:
return self.data[name]
else:
# So when an instance variable is being accessed, and
# it has not been declared before, nor is it contained
# in dictionary 'data', an attribute exception needs to
# be raised.
raise AttributeError
def __setattr__(self, key, value):
if key in self.data:
# Assign valid dictionary items here:
self.data[key] = value
else:
# Assign anything else as an instance attribute:
super().__setattr__(key,value)
Test:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
f.a = 'c'
f.d = 'e'
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
print("f.d = ", f.d)
print("f.x = ", f.x)
print("f.y = ", f.y)
# Should raise attributed Error
print("f.g = ", f.g)
Output:
f.a = v1
f.b = v2
f.data = {'a': 'v1', 'b': 'v2'}
f.a = c
f.b = v2
f.data = {'a': 'c', 'b': 'v2'}
f.d = e
f.x = 1
f.y = 2
Traceback (most recent call last):
File "MyClass.py", line 49, in <module>
print("f.g = ", f.g)
File "MyClass.py", line 25, in __getattr__
raise AttributeError
AttributeError

I think this implement is cooler
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self,key):
return self.data.get(key,None)

create objects on runtime in python

how do i create object-instances on runtime in python?
say i have 2 classes:
class MyClassA(object):
def __init__(self, prop):
self.prop = prop
self.name = "CLASS A"
def println(self):
print self.name
class MyClassB(object):
def __init__(self, prop):
self.prop = prop
self.name = "CLASS B"
def println(self):
print self.name
and a dict
{('a': MyClassA), ('b': MyClassB)}
how can i create dynamic an instance of one of my two classes, depending of i choose 'a' or 'b'.
kind of this:
somefunc(str):
if 'a': return new MyClassA
if 'b': return new MyClassB
to get "CLASS B" on calling: somefunc('a').println
but in a more elegant and dynamic way (say i add more classes to the dict on runtime)

You might create a dispatcher, which is a dictionary with your keys mapping to classes.
dispatch = {
"a": MyClassA,
"b": MyClassB,
}
instance = dispatch[which_one]() # Notice the second pair of parens here!

You create a class instance by calling the class. Your class dict {('a': MyClassA), ('b': MyClassB)} returns classes; so you need only call the class:
classes['a']()
But I get the sense you want something more specific. Here's a subclass of dict that, when called with a key, looks up the associated item and calls it:
>>> class ClassMap(dict):
... def __call__(self, key, *args, **kwargs):
... return self.__getitem__(key)(*args, **kwargs)
...
>>> c = ClassMap()
>>> c['a'] = A
>>> c['b'] = B
>>> c('a')
<__main__.A object at 0x1004cc7d0>