We Keep Coding

Python loop over non-inherited attributes in base class

I have a baseclass that contains nummerical attributes that are simply passed to it on initialization as a dictionary and then added to the instance's dictionary:
class baseclass(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
def calcValue(self):
return sum(vars(self).values())
Now I have a derieved class from this class, that adds additional attributes to the class, e.g.;
class childclass(baseclass):
def __init__(self, stringValue, **kwargs):
super(childclass, self).__init__(kwargs)
self.name = stringValue
Now I would like to have a function in my baseclass that only iterates over all attributes that were added to the class but not the one that were added as child attributes. For example if I create an instance of child class like this:
instance = childclass("myname", a=1, b=2, c=3)
and then call the calcValue method, it should return 1+2+3 = 6
instance.calcValue()
but since vars(self) will return the full dictionary, uncluding the string from the childclass attribute, which of course can then not be added. Is there a way to only acces the attributes of the instance that belong to the respective derieved class?

You are storing all your attributes as ordinary values on the instance's __dict__. Which means that without any further hints, they are indistinguishable one from another.
Python has a couple mechanisms to treat attributes in special manners. If you would declare the attributes in your base class in the class itself, and just init their values inside the __init__ method, it would be possible to introspect the base class' __dict__ (and not the instance's __dict__), or the __annotations__ attribute in the same class.
As it is in the example, though, one easy thing is to use an special attribute to take note of the attributes that are added on the base class, and you then consult this as the attributes' name source:
class baseclass(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
self._numeric_attrs = set(kwargs.keys())
def calcValue(self):
return sum(getattr(self, attr) for attr in self._numeric_attrs)

A simple, safe and effective - but with added overhead - would be to store the base class attributes in a distinct attribute and use __getattr__ to serve them:
class BaseClass(object):
def __init__(self, **kwargs):
self._attribs = kwargs
def __getattr__(self, name):
try:
return self._attribs[name]
except KeyError:
raise AttributeError("object {} has no attribute {}".format(type(self).__name__, name))
def calcValue(self):
return sum(self._attribs.values())
I usually try to avoid __getattr__ as it makes the code harder to inspect and maintain, but since your class has already no definite API it doesn't make much difference here.

logging module with setters in class

I have a question about logging module, I'm using it in some classes and I'm wondering how I can log setters or getters. For example I have a class :
class Item(object):
def __init__(self, name):
self.name = name
item_ = Item('object')
item_.name = 'New object'
I want here a log to say name of object has been changed. Of course, I would like avoid using #property and setters functions which will override my class.

This is relatively straightforward to implement:
class Item(object):
def __setattr__(self, name, value):
super(Item, self).__setattr__(name, value)
# put logging code here
Do not use vars() for this. It will not handle #property and other data descriptors correctly. It will also return a read-only dictionary-like-thing if you happen to be implementing a metaclass. If you don't know what that means, then you're not implementing a metaclass, so don't worry.
If you have multiple classes like this, you can factor this functionality out into a mixin class:
class LoggingMixin(object):
def __setattr__(self, name, value):
super(LoggingMixin, self).__setattr__(name, value)
# put logging code here
Now, when you want to create a new class that needs attribute logging, simply inherit from LoggingMixin, along with any other base classes if necessary.

Python calling super constructor - am I doing it right?

I have a base class like so:
class Token:
def __init__(self, value):
self.value = value.strip()
self.tokens = None
def get_value(self):
return self.value
def tokenize(self):
pass # abstract stub
def __str__(self):
return type(self).__name__ + ': '+ re.sub(r'\s+', ' ', self.value)
And a ton of it's child classes:
class T_DefineDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_IncludeDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_IfdefDirective(Token):
def __init__(self, value):
super().__init__(value)
class T_Identifier(Token):
def __init__(self, value):
super().__init__(value)
class T_Rvalue(Token):
def __init__(self, value):
super().__init__(value)
def tokenize(self):
pass # stuff here
Now I'm a DRY programmer. I hate repetition. If you look at the code, the __init__ piece is copy-pasted in all the child classes.
My question, is there some way to avoid the repetition, or is this really the right way?
(note that the example is a bit shortened, so it may not make too much sense. But you can see the issue I mean).

If you do not have any additional setup work to do in the Token subclasses, then it is safe not to override __init__.
If you do have to perform some subclass-specific initialisation, then the patten that you're using is fine and 'pythonic'.
To clarify:
if __init__ is not defined on a class, then Python will use the __init__ method defined on (one of) its parent class(es), if possible
this is because there aren't any special rules for overriding 'magic' methods like __init__
even if the initialiser on a parent class is used, an instance of the subclass will be created
this is because the actual creation happens in __new__; the newly created object is then passed to __init__ for initialisation

If you really want to eliminate as much boilerplate as possible:
First, you don't need __init__ if all it does is call super(); special methods are inherited just like any other methods, as sapi's answer explains.
Second, you can dynamically create a bunch of classes:
token_classes = {
'T_{}'.format(name): type('T_{}'.format(name), (Token,), {})
for name in 'DefineDirective IncludeDirective IfdefDirective Identifier'.split()
}
And you can use them straight out of that dict, but if you really want to make them into globals you can:
globals().update(token_classes)
However, the whole goal of avoiding repetition is to make your code more readable and maintainable, and in this case, I think we're achieving the opposite. :)

On-demand creation of member methods using __getattr__()

Given a class MyClass(object), how can I programmatically define class member methods using some template and the MyClass.__getattr__ mechanism? I'm thinking of something along the lines of
class MyClass(object):
def __init__(self, defaultmembers, members):
# defaultmembers is a list
# members is a dict
self._defaultmembers = defaultmembers
self._members = members
# some magic spell creating a member function factory
def __getattr__(self):
# some more magic
def f_MemberB():
pass
C = MyClass(defaultmembers=["MemberA"], members=dict(MemberB=f_MemberB)
C.MemberA() # should be a valid statement
C.MemberB() # should also be a valid statement
C.MemberC() # should raise an AttributeError
C.MemberA should be a method automatically created from some template mechanism inside the class, and C.MemberB should be the function f_MemberB.

You don't need to redefine __getattr__ (and in fact you generally should never do that). Python is a late binding language. This means you can simply assign values to names in a class at any time (even dynamically at runtime) and they will now exist.
So, in your case, you can simply do:
class MyClass(object):
def __init__(self, defaultmembers, members):
# defaultmembers is a list
# members is a dict
for func in defaultmembers:
setattr(self, func.__name__, func)
for name, func in members.items():
setattr(self, name, func)
Note that this will not actually bind the method to the class (it will not get self as its first argument). If that is what you want, you need to use the MethodType function from types, like so:
from types import MethodType
class MyClass(object):
def __init__(self, defaultmembers, members):
# defaultmembers is a list
# members is a dict
for func in defaultmembers:
name = func.__name__
setattr(self, name , func)
setattr(self, name , MethodType(getattr(self, name), self))
for name, func in members.items():
setattr(self, name, func)
setattr(self, name , MethodType(getattr(self, name), self))
Example:
def def_member(self):
return 1
def key_member(self):
return 2
>>> test = MyClass([def_member], {'named_method':key_member})
>>> test.def_member()
1
>>> test.named_method()
2
You can also make the init method slightly less awkward by using *args and **kwargs, so that the example would just be test = MyClass(def_member, named_member = key_member) if you know there won't be any other arguments to the constructor of this class.
Obviously I've left out the template creation bit for the defaultmembers, since I've used passing a function, rather than simply a name, as the argument. But you should be able to see how you would expand that example to suit your needs, as the template creation part is a bit out of the scope of the original question, which is how to dynamically bind methods to classes.
An important note: This only affects the single instance of MyClass. Please alter the question if you want to affect all instances. Though I would think using mixin class would be better in that case.

python decorator to modify variable in current scope

Goal: Make a decorator which can modify the scope that it is used in.
If it worked:
class Blah(): # or perhaps class Blah(ParentClassWhichMakesThisPossible)
def one(self):
pass
#decorated
def two(self):
pass
>>> Blah.decorated
["two"]
Why? I essentially want to write classes which can maintain specific dictionaries of methods, so that I can retrieve lists of available methods of different types on a per class basis. errr.....
I want to do this:
class RuleClass(ParentClass):
#rule
def blah(self):
pass
#rule
def kapow(self):
pass
def shazam(self):
class OtherRuleClass(ParentClass):
#rule
def foo(self):
pass
def bar(self):
pass
>>> RuleClass.rules.keys()
["blah", "kapow"]
>>> OtherRuleClass.rules.keys()
["foo"]

You can do what you want with a class decorator (in Python 2.6) or a metaclass. The class decorator version:
def rule(f):
f.rule = True
return f
def getRules(cls):
cls.rules = {}
for attr, value in cls.__dict__.iteritems():
if getattr(value, 'rule', False):
cls.rules[attr] = value
return cls
#getRules
class RuleClass:
#rule
def foo(self):
pass
The metaclass version would be:
def rule(f):
f.rule = True
return f
class RuleType(type):
def __init__(self, name, bases, attrs):
self.rules = {}
for attr, value in attrs.iteritems():
if getattr(value, 'rule', False):
self.rules[attr] = value
super(RuleType, self).__init__(name, bases, attrs)
class RuleBase(object):
__metaclass__ = RuleType
class RuleClass(RuleBase):
#rule
def foo(self):
pass
Notice that neither of these do what you ask for (modify the calling namespace) because it's fragile, hard and often impossible. Instead they both post-process the class -- through the class decorator or the metaclass's __init__ method -- by inspecting all the attributes and filling the rules attribute. The difference between the two is that the metaclass solution works in Python 2.5 and earlier (down to 2.2), and that the metaclass is inherited. With the decorator, subclasses have to each apply the decorator individually (if they want to set the rules attribute.)
Both solutions do not take inheritance into account -- they don't look at the parent class when looking for methods marked as rules, nor do they look at the parent class rules attribute. It's not hard to extend either to do that, if that's what you want.

Problem is, at the time the decorated decorator is called, there is no object Blah yet: the class object is built after the class body finishes executing. Simplest is to have decorated stash the info "somewhere else", e.g. a function attribute, then a final pass (a class decorator or metaclass) reaps that info into the dictionary you desire.
Class decorators are simpler, but they don't get inherited (so they wouldn't come from a parent class), while metaclasses are inherited -- so if you insist on inheritance, a metaclass it will have to be. Simplest-first, with a class decorator and the "list" variant you have at the start of your Q rather than the "dict" variant you have later:
import inspect
def classdecorator(aclass):
decorated = []
for name, value in inspect.getmembers(aclass, inspect.ismethod):
if hasattr(value, '_decorated'):
decorated.append(name)
del value._decorated
aclass.decorated = decorated
return aclass
def decorated(afun):
afun._decorated = True
return afun
now,
#classdecorator
class Blah(object):
def one(self):
pass
#decorated
def two(self):
pass
gives you the Blah.decorated list you request in the first part of your Q. Building a dict instead, as you request in the second part of your Q, just means changing decorated.append(name) to decorated[name] = value in the code above, and of course initializing decorated in the class decorator to an empty dict rather than an empty list.
The metaclass variant would use the metaclass's __init__ to perform essentially the same post-processing after the class body is built -- a metaclass's __init__ gets a dict corresponding to the class body as its last argument (but you'll have to support inheritance yourself by appropriately dealing with any base class's analogous dict or list). So the metaclass approach is only "somewhat" more complex in practice than a class decorator, but conceptually it's felt to be much more difficult by most people. I'll give all the details for the metaclass if you need them, but I'd recommend sticking with the simpler class decorator if feasible.