We Keep Coding

python - register all subclasses

I have a class Step, which I want to derive by many sub-classes. I want every class deriving from Step to be "registered" by a name I choose for it (not the class's name), so I can later call Step.getStepTypeByName().
Something like this, only working :):
class Step(object):
_STEPS_BY_NAME = {}
#staticmethod
def REGISTER(cls, name):
_STEPS_BY_NAME[name] = cls
class Derive1(Step):
REGISTER(Derive1, "CustomDerive1Name")
...
class Derive2(Step):
REGISTER(Derive2, "CustomDerive2Name")
...

Your solution do not work for three reasons.
The first one is that _STEPS_BY_NAME only exists as an attribute of the Step class, so Step.REGISTER cannot access _STEPS_BY_NAME without a reference to the Step class. IOW you have to make it a classmethod (cf below)
The second one is that you need to explicitely use Step.REGISTER(cls) - the name REGISTER does not exist outside the Step class.
The third reason is that within a class statement's body, the class object has not yet been created not bound to it's name, so you cannot not reference the class itself at this point.
IOW, you'd want this instead:
class Step(object):
_STEPS_BY_NAME = {}
# NB : by convention, "ALL_UPPER" names denote pseudo-constants
#classmethod
def register(cls, name):
# here `cls` is the current class
cls._STEPS_BY_NAME[name] = stepclass
class Derive1(Step):
...
Step.register(Derive1, "CustomDerive1Name")
class Derive2(Step):
...
Step.register(Derive2, "CustomDerive2Name")
Now with a minor modification to Step.register you could use it as a class decorator, making things much clearer:
class Step(object):
_STEPS_BY_NAME = {}
#classmethod
def register(cls, name):
def _register(stepclass):
cls._STEPS_BY_NAME[name] = stepclass
return stepclass
return _register
#Step.register("CustomDerive1Name")
class Derive1(Step):
...
#Step.register("CustomDerive2Name")
class Derive2(Step):
...
As a last note: unless you have a compelling reason to register your subclasses in the base class itself, it might be better to use module-level variables and functions (a Python module is actually a kind of singleton):
# steps.py
class Step(object):
#....
_STEPS_BY_NAME = {}
def register(name):
def _register(cls):
_STEPS_BY_NAME[name] = cls
return cls
return _register
def get_step_class(name):
return _STEPS_BY_NAME[name]
And in your other modules
import steps
#steps.register("CustomDerive1Name")
class Derive1(steps.Step):
# ...
The point here is to avoid giving too many responsabilies to your Step class. I don't know your concrete use case so I can't tell which design best fits your need, but I've been using this last one on quite a few projects and it always worked fine so far.

You are close. Use this
class Step(object):
pass
class Derive1(Step):
pass
class Derive2(Step):
pass
_STEPS_BY_NAME = {
'foo': Step,
'bar': Derive1,
'bar': Derive2
}
def get_step_by_name(name):
return _STEPS_BY_NAME[name]
Warning: there might be better approaches depending on what you are trying to achieve. Such a mapping from strings to methods is a maintenance nightmare. If you want to change the name of a method, you would have to remember to change it in multiple place. You won't get any autocomplete help from your IDE either.

Refer to a property from inside the class itself

Say I have a class that looks like
class MeasurementList:
def __init__(self, measurement_list):
self.__measurements = measurement_list
#property
def measurements(self):
return self.__measurements
what is the most pythonic way to retrieve the value of self.measurements from inside the class; directly accessing the variable or going via the property (external accessor)? I.e.,
def do_something(self)
# do something
return self.measurements
or
def do_something(self)
# do something
return self.__measurements
Does any of the alternatives have any speed advantages, or easier refactoring, or other factors?

The point of properties is to add additional functionality to the process of getting/setting a field, while keeping the interface of a field.
That means you start out with a simple field, and access it as a field:
class MeasurementList:
def __init__(self, measurement_list):
self.measurements = measurement_list
def foo(self):
print("there are %d measurements" % len(self.measurements))
Then if you want/have to add additional logic to the setter/getter you convert it into a property, without having changed the interface. Thus no need to refactor accessing code.
class MeasurementList:
def __init__(self, measurement_list):
self._count = 0
self.measurements = measurement_list
#property
def measurements(self):
return self._measurements
#measurements.setter
def measurements(self value):
self._measurements = value
self._count = len(value)
def foo(self):
print("there are %d measurements" % (self._count))
def bar(self):
print(self.measurements)
Alternative reasons for using properties are readonly properties or properties that return computed (not directly stored in fields) values. In the case of read only properties you would access the backing field directly to write (from inside the class).
class MeasurementList:
def __init__(self, measurement_list):
self._measurements = measurement_list
# readonly
#property
def measurements(self):
return self._measurements
# computed property
#property
def count(self):
return len(self.measurements)
def foo(self):
print("there are %d measurements" % (self.count))
def bar(self):
print(self.measurements)
Keeping all that in mind you should not forget that there is no such thing as 'private' in python. If anyone really wants to access a private anything he can do so. It is just convention that anything starting with an underscore should be considered private and not be accessed by the caller. That is also the reason why one underscore is enough. Two underscores initiate some name mangling that is primarily used to avoid name conflicts, not prohibit access.

When you use properties in Python, you should almost always avoid accessing attribute under the property, unless it's necessary. Why?
Properties in Python are used to create getter, setter and deleter, but you probably know it.
They are usually used when you process the property data during those operation. I don't really have a good example for it right now, but consider following:
class User:
# _password stores hash object from user's password.
#property
def password(self):
return self._password.hexdigest() # Returns hash as string
#password.setter
def password(self, val):
self._password = hash(val) # Creates hash object
Here using _password and password results in quite different output. In most cases, you need simply password, both inside and outside class definition, unless you want to interact directly with object wrapped by it.
If you have the same object returned in getter and attribute, then you should follow the same practice, as you may wish sameday to add some checks or mechanics to it, and it will save you from refactoring every use of _attribute, and following that convention will also save you from errors when creating more complex descriptors.
Also, from your code, note that using __measurements (leading double underscore) results in name mangling of attribute name, so if you ever inherit from MeasurementList, you will be almost unable to access this attribute.

I presume you have seen code like this in Java. It is, however, deeply unpythonic to use methods where attribute access serves the purpose perfectly well. Your existing code would be much more simply written as
class MeasurementList:
def __init__(self, measurement_list):
self.measurements = measurement_list
Then no property is required.
The point is, presumably, to avoid allowing code external to the class to alter the value of the __measurements attribute. How necessary is this in practice?

Use setters and getters both inside and outside your class. It would make your code easier to maintain once you add some additional data processing into setters and getters:
class C(object):
_p = 1
#property
def p(self):
print 'getter'
return self._p
#p.setter
def p(self, val):
print 'setter'
self._p = val
def any_method(self):
self.p = 5
print '----'
a = self.p
myObject = C()
myObject.any_method()
From the output, you see that setter and getter are invoked:
setter
----
getter

Python: make every instance have a different superclass

In Python, I currently have instances of a class like MyClass('name1'), MyClass('name2') and so on.
I want to make it so that each instance has its own superclass, i.e., I want MyClass('name1') to be an instance of Name1MyClass and MyClass('name2') to be an instance of Name2MyClass. Name1MyClass and Name2MyClass would be dynamically generated subclasses of MyClass. I can't figure out how to do this, because it seems that Python always makes whatever is returned from __new__ an instance of that class. It isn't clear to me how to do it in a metaclass either.
The reason I want to do this is that I want to define __doc__ docstrings on the instances. But it seems that help completely ignores __doc__ on instances; it only looks on classes. So to put a different docstring on each instance, I need to make each instance have its own custom class.

I could be wrong, but I don't think you want a metaclass here. __metaclass__es are used when the class is created, not when you call the class to construct a new instance of the class (or something else).
Here's an answer using __new__ without a metaclass. It feels a bit hacky, but it seems to work:
_sentinel = Ellipsis
class MyClass(object):
def __new__(cls, name):
if name is _sentinel:
return object.__new__(cls)
else:
instance = type(name + cls.__name__, (MyClass,), {})(_sentinel)
# Initialization goes here.
return instance
print type(MyClass('name1'))
print type(MyClass('name2'))
There's a catch here -- All the business logic of initializing then new instance must be done in __new__. Since __new__ is returning a different type than the class it is bound to, __init__ won't get called.
Another option is to create a class factory:
class MyClass(object):
pass
def class_factory(name):
new_cls = type(name + MyClass.__name__, (MyClass,), {})
return new_cls() # Or pass whatever you want in here...
print type(class_factory('name1'))
print type(class_factory('name2'))
Finally, you could even create a non-__new__ class method:
class MyClass(object):
#classmethod
def class_factory(cls, name):
new_cls = type(name + cls.__name__, (cls,), {})
return new_cls() # Or pass whatever you want in here...
print type(MyClass.class_factory('name1'))
print type(MyClass.class_factory('name2'))

Using 'property' as the name of a method in a python class

Is this possible in Python?
class MyClass(object):
#property
def property(self):
return self._property
That is, I want to have a property named 'property'. It actually runs fine, but Eclipse complains with a warning. I thought the built-in #property decorator lived in a different namespace than the methods and properties within my classes.
Is it possible to rename the built-in decorator within the scope of the relevant module, so I can use the name 'property' without receiving this warning? Maybe something like the following:
attr = property
class MyClass(object):
#attr
def property(self):
return self._property
I do this, but I still get the warning, since I created an alias for the global built-in #property decorator, but the name 'property' is still a valid way to refer to it.
Any ideas?

The problem with naming a property property is the following:
class Foo(object):
#property
def property(self):
return "ham"
#property
def other_property(self):
return "spam"
The second property can't be defined since you've shadowed the name property in the class definition.
You can get around this by "renaming" property as in your example, but if I were you, I wouldn't mess with the built-ins in this way. It makes your code harder to follow.

Decorators are ordinary functions, so they live in the same namespace as other functions. Indeed your property function is inside class Foo; but it turns out that that's where python first looks for decorator names, so there's a conflict.
You can see this from the fact that the following code compiles:
class Foo(object):
def decfun(x): return "ham"
#decfun
def second(self, y): pass

Using the docstring from one method to automatically overwrite that of another method

The problem: I have a class which contains a template method execute which calls another method _execute. Subclasses are supposed to overwrite _execute to implement some specific functionality. This functionality should be documented in the docstring of _execute.
Advanced users can create their own subclasses to extend the library. However, another user dealing with such a subclass should only use execute, so he won't see the correct docstring if he uses help(execute).
Therefore it would be nice to modify the base class in such a way that in a subclass the docstring of execute is automatically replaced with that of _execute. Any ideas how this might be done?
I was thinking of metaclasses to do this, to make this completely transparent to the user.

Well, if you don't mind copying the original method in the subclass, you can use the following technique.
import new
def copyfunc(func):
return new.function(func.func_code, func.func_globals, func.func_name,
func.func_defaults, func.func_closure)
class Metaclass(type):
def __new__(meta, name, bases, attrs):
for key in attrs.keys():
if key[0] == '_':
skey = key[1:]
for base in bases:
original = getattr(base, skey, None)
if original is not None:
copy = copyfunc(original)
copy.__doc__ = attrs[key].__doc__
attrs[skey] = copy
break
return type.__new__(meta, name, bases, attrs)
class Class(object):
__metaclass__ = Metaclass
def execute(self):
'''original doc-string'''
return self._execute()
class Subclass(Class):
def _execute(self):
'''sub-class doc-string'''
pass

Is there a reason you can't override the base class's execute function directly?
class Base(object):
def execute(self):
...
class Derived(Base):
def execute(self):
"""Docstring for derived class"""
Base.execute(self)
...stuff specific to Derived...
If you don't want to do the above:
Method objects don't support writing to the __doc__ attribute, so you have to change __doc__ in the actual function object. Since you don't want to override the one in the base class, you'd have to give each subclass its own copy of execute:
class Derived(Base):
def execute(self):
return Base.execute(self)
class _execute(self):
"""Docstring for subclass"""
...
execute.__doc__= _execute.__doc__
but this is similar to a roundabout way of redefining execute...

Look at the functools.wraps() decorator; it does all of this, but I don't know offhand if you can get it to run in the right context

Well the doc-string is stored in __doc__ so it wouldn't be too hard to re-assign it based on the doc-string of _execute after the fact.
Basically:
class MyClass(object):
def execute(self):
'''original doc-string'''
self._execute()
class SubClass(MyClass):
def _execute(self):
'''sub-class doc-string'''
pass
# re-assign doc-string of execute
def execute(self,*args,**kw):
return MyClass.execute(*args,**kw)
execute.__doc__=_execute.__doc__
Execute has to be re-declared to that the doc string gets attached to the version of execute for the SubClass and not for MyClass (which would otherwise interfere with other sub-classes).
That's not a very tidy way of doing it, but from the POV of the user of a library it should give the desired result. You could then wrap this up in a meta-class to make it easier for people who are sub-classing.

I agree that the simplest, most Pythonic way of approaching this is to simply redefine execute in your subclasses and have it call the execute method of the base class:
class Sub(Base):
def execute(self):
"""New docstring goes here"""
return Base.execute(self)
This is very little code to accomplish what you want; the only downside is that you must repeat this code in every subclass that extends Base. However, this is a small price to pay for the behavior you want.
If you want a sloppy and verbose way of making sure that the docstring for execute is dynamically generated, you can use the descriptor protocol, which would be significantly less code than the other proposals here. This is annoying because you can't just set a descriptor on an existing function, which means that execute must be written as a separate class with a __call__ method.
Here's the code to do this, but keep in mind that my above example is much simpler and more Pythonic:
class Executor(object):
__doc__ = property(lambda self: self.inst._execute.__doc__)
def __call__(self):
return self.inst._execute()
class Base(object):
execute = Executor()
class Sub(Base):
def __init__(self):
self.execute.inst = self
def _execute(self):
"""Actually does something!"""
return "Hello World!"
spam = Sub()
print spam.execute.__doc__ # prints "Actually does something!"
help(spam) # the execute method says "Actually does something!"