Python naming conventions for attributes and methods meant to be overwritten - python

I have some object oriented code in Python, where some classes are meant to be extended to provide the missing custom bits of code (a la Template Method pattern, but also with variables), that will only be used by the super class, not by the client code using them.
Are there any style conventions for such abstract (or dull, because their implementation in the super class would be either pass or raise a NonImplemented exception) methods and attributes?
I've been browsing the PEP-0008 and it only mentions about prepending an underscore to private members not intended to be used by subclasses.

I usually use single underscore e.g. _myvar for protected (as in C++) methods/attributes, which can be used by derived classes and use double underscore e.g. __var when it should not be used by anybody else, and as double-underscore names at class definition level are mangled, so they can't be overridden in derived class e.g.
class A(object):
def result1(self): # public method
return self._calc1()
def result2(self): # public method
return self.__calc2()
def _calc1(self): # protected method, can be overridden in derived class
return 'a1'
def __calc2(self): # private can't be overridden
return 'a2'
class B(A):
def _calc1(self):
return 'b1'
def __calc2(self):
return 'b2'
a = A()
print a.result1(),a.result2()
b = B()
print b.result1(),b.result2()
Here it seems derived class B is overriding both _calc1 and __calc2 but __calc2 isn't overridden because its name is already mangled with class name and hence output is
a1 a2
b1 a2
instead of
a1 a2
b1 b2
but ultimately choose any convention and document it, also in above case it is not that base class can't override private, here is a way :)
class B(A):
def _calc1(self):
return 'b1'
def _A__calc2(self):
return 'b2'

First of all i think that you are mistaken when you say that:
about prepending underscore to private members not intended to be used by subclasses.
Actually prepending a method/attribute by underscore is a python convention that mean that this method/attribute shouldn't be accessed outside the class (and its subclass) and I think you forgot to read about the double underscore that is used to make a method/attribute not possible to override.
class Foo(object):
def __foo(self):
print "foo"
def main(self):
self.__foo()
class Bar(Foo):
def __foo(self):
print "bar"
bar = Bar()
bar.main()
# This will print "foo" and not "bar".
There is another way of declaring stub method by the way which is using abc.ABCMeta and abc.abstractmethod.

There isn't really a naming convention for these methods because they'll have the same name when they're overridden. Otherwise they wouldn't be overridden! I think having the method to be overridden do something trivial, and documenting it accordingly, is good enough:
class MyClass:
def aMethod():
'''Will be overridden in MyDerivedClass'''
pass
Name mangling, which you mention in your question, is useful if you have a non-trivial method that will be overridden, but you still want to be able to access the base version. See the documentation for more info and an example.

Related

Get python class's namespace parent type

Is it possible to get the the namespace parent, or encapsulating type, of a class?
class base:
class sub:
def __init__(self):
# self is "__main__.extra.sub"
# want to create object of type "__main__.extra" from this
pass
class extra(base):
class sub(base.sub):
pass
o = extra.sub()
The problem in base.sub.__init__ is getting extra from the extra.sub.
The only solutions I can think of at the moment involve having all subclasses of base provide some link to their encapsulating class type or turning the type of self in base.sub.__init__ into a string an manipulating it into a new type string. Both a bit ughly.
It's clearly possible to go the other way, type(self()).sub would give you extra.sub from inside base.sub.__init__ for a extra type object, but how do I do .. instead of .sub ? :)
The real answer is that there is no general way to do this. Python classes are normal objects, but they are created a bit differently. A class does not exist until well after its entire body has been executed. Once a class is created, it can be bound to many different names. The only reference it has to where it was created are the __module__ and __qualname__ attributes, but both of these are mutable.
In practice, it is possible to write your example like this:
class Sub:
def __init__(self):
pass
class Base:
Sub = Sub
Sub.__qualname__ = 'Base.Sub'
class Sub(Sub):
pass
class Extra(Base):
Sub = Sub
Sub.__qualname__ = 'Extra.Sub'
del Sub # Unlink from global namespace
Barring the capitalization, this behaves exactly as your original example. Hopefully this clarifies which code has access to what, and shows that the most robust way to determine the enclosing scope of a class is to explicitly assign it somewhere. You can do this in any number of ways. The trivial way is just to assign it. Going back to your original notation:
class Base:
class Sub:
def __init__(self):
print(self.enclosing)
Base.Sub.enclosing = Base
class Extra(Base):
class Sub(Base.Sub):
pass
Extra.Sub.enclosing = Extra
Notice that since Base does not exist when it body is being executed, the assignment has to happen after the classes are both created. You can bypass this by using a metaclass or a decorator. That will allow you to mess with the namespace before the class object is assigned to a name, making the change more transparent.
class NestedMeta(type):
def __init__(cls, name, bases, namespace):
for name, obj in namespace.items():
if isinstance(obj, type):
obj.enclosing = cls
class Base(metaclass=NestedMeta):
class Sub:
def __init__(self):
print(self.enclosing)
class Extra(Base):
class Sub(Base.Sub):
pass
But this is again somewhat unreliable because not all metaclasses are an instance of type, which takes us back to the first statement in this answer.
In many cases, you can use the __qualname__ and __module__ attributes to get the name of the surrounding class:
import sys
cls = type(o)
getattr(sys.modules[cls.__module__], '.'.join(cls.__qualname__.split('.')[:-1]))
This is a very literal answer to your question. It just shows one way of getting the class in the enclosing scope without addressing the probably design flaws that lead to this being necessary in the first place, or any of the many possible corner cases that this would not cover.

How can I refer to the currently being defined class? [duplicate]

For a recursive function we can do:
def f(i):
if i<0: return
print i
f(i-1)
f(10)
However is there a way to do the following thing?
class A:
# do something
some_func(A)
# ...
If I understand your question correctly, you should be able to reference class A within class A by putting the type annotation in quotes. This is called forward reference.
class A:
# do something
def some_func(self, a: 'A')
# ...
See ref below
https://github.com/python/mypy/issues/3661
https://www.youtube.com/watch?v=AJsrxBkV3kc
In Python you cannot reference the class in the class body, although in languages like Ruby you can do it.
In Python instead you can use a class decorator but that will be called once the class has initialized. Another way could be to use metaclass but it depends on what you are trying to achieve.
You can't with the specific syntax you're describing due to the time at which they are evaluated. The reason the example function given works is that the call to f(i-1) within the function body is because the name resolution of f is not performed until the function is actually called. At this point f exists within the scope of execution since the function has already been evaluated. In the case of the class example, the reference to the class name is looked up during while the class definition is still being evaluated. As such, it does not yet exist in the local scope.
Alternatively, the desired behavior can be accomplished using a metaclass like such:
class MetaA(type):
def __init__(cls):
some_func(cls)
class A(object):
__metaclass__=MetaA
# do something
# ...
Using this approach you can perform arbitrary operations on the class object at the time that the class is evaluated.
Maybe you could try calling __class__.
Right now I'm writing a code that calls a class method from within the same class.
It is working well so far.
I'm creating the class methods using something like:
#classmethod
def my_class_method(cls):
return None
And calling then by using:
x = __class__.my_class_method()
It seems most of the answers here are outdated. From python3.7:
from __future__ import annotations
Example:
$ cat rec.py
from __future__ import annotations
class MyList:
def __init__(self,e):
self.data = [e]
def add(self, e):
self.data.append(e)
return self
def score(self, other:MyList):
return len([e
for e in self.data
if e in other.data])
print(MyList(8).add(3).add(4).score(MyList(4).add(9).add(3)))
$ python3.7 rec.py
2
Nope. It works in a function because the function contents are executed at call-time. But the class contents are executed at define-time, at which point the class doesn't exist yet.
It's not normally a problem because you can hack further members into the class after defining it, so you can split up a class definition into multiple parts:
class A(object):
spam= 1
some_func(A)
A.eggs= 2
def _A_scramble(self):
self.spam=self.eggs= 0
A.scramble= _A_scramble
It is, however, pretty unusual to want to call a function on the class in the middle of its own definition. It's not clear what you're trying to do, but chances are you'd be better off with decorators (or the relatively new class decorators).
There isn't a way to do that within the class scope, not unless A was defined to be something else first (and then some_func(A) will do something entirely different from what you expect)
Unless you're doing some sort of stack inspection to add bits to the class, it seems odd why you'd want to do that. Why not just:
class A:
# do something
pass
some_func(A)
That is, run some_func on A after it's been made. Alternately, you could use a class decorator (syntax for it was added in 2.6) or metaclass if you wanted to modify class A somehow. Could you clarify your use case?
If you want to do just a little hacky thing do
class A(object):
...
some_func(A)
If you want to do something more sophisticated you can use a metaclass. A metaclass is responsible for manipulating the class object before it gets fully created. A template would be:
class AType(type):
def __new__(meta, name, bases, dct):
cls = super(AType, meta).__new__(meta, name, bases, dct)
some_func(cls)
return cls
class A(object):
__metaclass__ = AType
...
type is the default metaclass. Instances of metaclasses are classes so __new__ returns a modified instance of (in this case) A.
For more on metaclasses, see http://docs.python.org/reference/datamodel.html#customizing-class-creation.
If the goal is to call a function some_func with the class as an argument, one answer is to declare some_func as a class decorator. Note that the class decorator is called after the class is initialized. It will be passed the class that is being decorated as an argument.
def some_func(cls):
# Do something
print(f"The answer is {cls.x}")
return cls # Don't forget to return the class
#some_func
class A:
x = 1
If you want to pass additional arguments to some_func you have to return a function from the decorator:
def some_other_func(prefix, suffix):
def inner(cls):
print(f"{prefix} {cls.__name__} {suffix}")
return cls
return inner
#some_other_func("Hello", " and goodbye!")
class B:
x = 2
Class decorators can be composed, which results in them being called in the reverse order they are declared:
#some_func
#some_other_func("Hello", "and goodbye!")
class C:
x = 42
The result of which is:
# Hello C and goodbye!
# The answer is 42
What do you want to achieve? It's possible to access a class to tweak its definition using a metaclass, but it's not recommended.
Your code sample can be written simply as:
class A(object):
pass
some_func(A)
If you want to refer to the same object, just use 'self':
class A:
def some_func(self):
another_func(self)
If you want to create a new object of the same class, just do it:
class A:
def some_func(self):
foo = A()
If you want to have access to the metaclass class object (most likely not what you want), again, just do it:
class A:
def some_func(self):
another_func(A) # note that it reads A, not A()
Do remember that in Python, type hinting is just for auto-code completion therefore it helps IDE to infer types and warn user before runtime. In runtime, type hints almost never used(except in some cases) so you can do something like this:
from typing import Any, Optional, NewType
LinkListType = NewType("LinkList", object)
class LinkList:
value: Any
_next: LinkListType
def set_next(self, ll: LinkListType):
self._next = ll
if __name__ == '__main__':
r = LinkList()
r.value = 1
r.set_next(ll=LinkList())
print(r.value)
And as you can see IDE successfully infers it's type as LinkList:
Note: Since the next can be None, hinting this in the type would be better, I just didn't want to confuse OP.
class LinkList:
value: Any
next: Optional[LinkListType]
It's ok to reference the name of the class inside its body (like inside method definitions) if it's actually in scope... Which it will be if it's defined at top level. (In other cases probably not, due to Python scoping quirks!).
For on illustration of the scoping gotcha, try to instantiate Foo:
class Foo(object):
class Bar(object):
def __init__(self):
self.baz = Bar.baz
baz = 15
def __init__(self):
self.bar = Foo.Bar()
(It's going to complain about the global name 'Bar' not being defined.)
Also, something tells me you may want to look into class methods: docs on the classmethod function (to be used as a decorator), a relevant SO question. Edit: Ok, so this suggestion may not be appropriate at all... It's just that the first thing I thought about when reading your question was stuff like alternative constructors etc. If something simpler suits your needs, steer clear of #classmethod weirdness. :-)
Most code in the class will be inside method definitions, in which case you can simply use the name A.

Is there a method like '__getattribute__' for class (not instance) variables?

I have a class sysprops in which I'd like to have a number of constants. However, I'd like to pull the values for those constants from the database, so I'd like some sort of hook any time one of these class constants are accessed (something like the getattribute method for instance variables).
class sysprops(object):
SOME_CONSTANT = 'SOME_VALUE'
sysprops.SOME_CONSTANT # this statement would not return 'SOME_VALUE' but instead a dynamic value pulled from the database.
Although I think it is a very bad idea to do this, it is possible:
class GetAttributeMetaClass(type):
def __getattribute__(self, key):
print 'Getting attribute', key
class sysprops(object):
__metaclass__ = GetAttributeMetaClass
While the other two answers have a valid method. I like to take the route of 'least-magic'.
You can do something similar to the metaclass approach without actually using them. Simply by using a decorator.
def instancer(cls):
return cls()
#instancer
class SysProps(object):
def __getattribute__(self, key):
return key # dummy
This will create an instance of SysProps and then assign it back to the SysProps name. Effectively shadowing the actual class definition and allowing a constant instance.
Since decorators are more common in Python I find this way easier to grasp for other people that have to read your code.
sysprops.SOME_CONSTANT can be the return value of a function if SOME_CONSTANT were a property defined on type(sysprops).
In other words, what you are talking about is commonly done if sysprops were an instance instead of a class.
But here is the kicker -- classes are instances of metaclasses. So everything you know about controlling the behavior of instances through the use of classes applies equally well to controlling the behavior of classes through the use of metaclasses.
Usually the metaclass is type, but you are free to define other metaclasses by subclassing type. If you place a property SOME_CONSTANT in the metaclass, then the instance of that metaclass, e.g. sysprops will have the desired behavior when Python evaluates sysprops.SOME_CONSTANT.
class MetaSysProps(type):
#property
def SOME_CONSTANT(cls):
return 'SOME_VALUE'
class SysProps(object):
__metaclass__ = MetaSysProps
print(SysProps.SOME_CONSTANT)
yields
SOME_VALUE

What is the naming convention for Python class references

What is the naming convention for a variable referencing a class in Python?
class MyClass(object):
pass
# which one is correct?
reference_to_class = MyClass
# or
ReferenceToClass = MyClass
Here is another example that resembles my situation:
# cars.py
class Car(object):
pass
class Sedan(Car):
pass
class Coupe(Car):
pass
class StatonWagon(Car):
pass
class Van(Car):
pass
def get_car_class(slug, config):
return config.get(slug)
# config.py
CONFIG = {
'ford-mustang': Coupe,
'buick-riviera': Coupe,
'chevrolet-caprice': Sedan,
'chevy-wan' Van:
'ford-econoline': Van
}
# main.py
from config.py import CONFIG
from cars import get_car_class
MyCarClass = get_car_class('buick-riviera')
my_car = MyCarClass()
I would prefer ReferenceToClass, that everybody new to the code knows it's a class and not an instance. But as poplitea wrote, literature reference would be great.
On module level the second:
ReferenceToClass = MyClass
As a function argument, the first:
reference_to_class = MyClass
tl;dr: for global/public names use AllCaps like XORcist said:
class Logger:
pass
AliasLogger = Logger
For function parameters and function locals, make it clear that you are dealing with the class object with a descriptive name like this:
def some_func(logger_class):
pass
or something along the lines
def some_func(my_class_classobj):
pass
when the word "class" is actually in your classname. For classobj, see also class_ and klass.
Analysis/Motivation (long version)
No thorough reading, but at a glance PEP 8 doesn't seem to be explicit on this (neither google's python style guide for that matter).
Since a variable name is probably just yet-another name binding in python, in my opinion it doesn't really matter whether you bind that name with the definition block or later with the = equal sign to some object.
For this I agree with XORcist in that module level "alias" references should adhere to your class naming standard, probably AllCaps:
class MyClass(object):
pass
# good
ReferenceToClass = MyClass
However when it comes to parameter and variable names, supposedly lowercase_underscores should apply, right? I'm unhappy with only that, since it will push you into the instance vs class reference ambiguity. There is the potential that an all-lowercase name may be an attempt to hint the object being an instance. For that matter, I recommend postfixing your all-lowercase, class-referencing variable names with the "class" suffix, like this:
class Logger(object):
pass
def function_expecting_class_reference(logger_class):
pass
I renamed your example class MyClass to Logger because in real scenarios only a few class name contains the string "class". However in that latter case I propose to avoid the ambiguity with descriptive naming yet again. For example, you may use a classobj suffix:
class MyClass(object):
pass
def function_expecting_class_reference(another_param, my_class_classobj):
ReferenceToClass = MyClass
Another alternative I tend to take is to use the suffix klass, like my_class_klass. Not everyone seems to get the latter, but anyway I'm yet to test whether they would get the former any better.
I treat it the same as an instance variable, which PEP8 defines as using lowercase_underscore_style. (lowercase, with words separated by underscores as necessary to improve readability.)
http://www.python.org/dev/peps/pep-0008/#id34

Calling private parent class method from parent class (django)

I want to call a redefined private method from an abstract parent class. I am using django if that matters.
class Parent(models.Model):
def method1(self):
#do somthing
self.__method2()
def method2(self):
pass # I also tried calling up a prent method with super
class child(Parent):
def method1(self)
super(Child, self).method1()
def __method2(self):
#do something
I get a
AttributeError: "'Chil' object has no attribute '_Parent__method2'"
What I am doing wrong ?
Initial double underscores prevent polymorphism since both the method definition and the method call get mangled, to two different names. Replace with a single underscore to fix this.
Also, double underscores are not used for "private" attributes, and you should discard whatever reference told you that they are. They're used for MI disambiguation.

Categories