Am I missing something, or this something like this not possible?
class Outer:
def __init__(self, val):
self.__val = val
def __getVal(self):
return self.__val
def getInner(self):
return self.Inner(self)
class Inner:
def __init__(self, outer):
self.__outer = outer
def getVal(self):
return self.__outer.__getVal()
foo = Outer('foo')
inner = foo.getInner()
val = inner.getVal()
print val
I'm getting this error message:
return self.__outer.__getVal()
AttributeError: Outer instance has no attribute '_Inner__getVal'
You are trying to apply Java techniques to Python classes. Don't. Python has no privacy model like Java does. All attributes on a class and its instances are always accessible, even when using __name double-underscore names in a class (they are simply renamed to add a namespace).
As such, you don't need an inner class either, as there is no privileged access for such a class. You can just put that class outside Outer and have the exact same access levels.
You run into your error because Python renames attributes with initial double-underscore names within a class context to avoid clashing with subclasses. These are called class private because the renaming adds the class names as a namespace; this applies both to their definition and use. See the Reserved classes of identifiers section of the reference documentation:
__*
Class-private names. Names in this category, when used within the context of a class definition, are re-written to use a mangled form to help avoid name clashes between “private” attributes of base and derived classes.
All names with double underscores in Outer get renamed to _Outer prefixed, so __getVal is renamed to _Outer__getVal. The same happens to any such names in Inner, so your Inner.getVal() method will be looking for a _Inner__getVal attribute. Since Outer has no _Inner__getVal attribute, you get your error.
You could manually apply the same transformation to Inner.getVal() to 'fix' this error:
def getVal(self):
return self.__outer._Outer__getVal()
But you are not using double-underscore names as intended anyway, so move to single underscores instead, and don't use a nested class:
class Outer:
def __init__(self, val):
self._val = val
def _getVal(self):
return self._val
def getInner(self):
return _Inner(self)
class _Inner:
def __init__(self, outer):
self._outer = outer
def getVal(self):
return self._outer._getVal()
I renamed Inner to _Inner to document the type is an internal implementation detail.
While we are on the subject, there really is no need to use accessors either. In Python you can switch between property objects and plain attributes at any time. There is no need to code defensively like you have to in Java, where switching between attributes and accessors carries a huge switching cost. In Python, don't use obj.getAttribute() and obj.setAttribute(val) methods. Just use obj.attribute and obj.attribute = val, and use property if you need to do more work to produce or set the value. Switch to or away from property objects at will during your development cycles.
As such, you can simplify the above further to:
class Outer(object):
def __init__(self, val):
self._val = val
#property
def inner(self):
return _Inner(self)
class _Inner(object):
def __init__(self, outer):
self._outer = outer
#property
def val(self):
return self._outer._val
Here outer.inner produces a new _Inner() instance as needed, and the Inner.val property proxies to the stored self._outer reference. The user of the instance never need know either attribute is handled by a property object:
>>> outer = Outer(42)
>>> print outer.inner.val
42
Note that for property to work properly in Python 2, you must use new-style classes; inherit from object to do this; on old-style classes on property getters are supported (meaning setting is not prevented either!). This is the default in Python 3.
The leading-double-underscore naming convention in Python is supported with "name mangling." This is implemented by inserting the name of the current class in the name, as you have seen.
What this means for you is that names of the form __getVal can only be accessed from within the exact same class. If you have a nested class, it will be subject to different name mangling. Thus:
class Outer:
def foo(self):
print(self.__bar)
class Inner:
def foo2(self):
print(self.__bar)
In the two nested classes, the names will be mangled to _Outer__bar and _Inner__bar respectively.
This is not Java's notion of private. It's "lexical privacy" (akin to "lexical scope" ;-).
If you want Inner to be able to access the Outer value, you will have to provide a non-mangled API. Perhaps a single underscore: _getVal, or perhaps a public method: getVal.
Related
Is it possible to get the the namespace parent, or encapsulating type, of a class?
class base:
class sub:
def __init__(self):
# self is "__main__.extra.sub"
# want to create object of type "__main__.extra" from this
pass
class extra(base):
class sub(base.sub):
pass
o = extra.sub()
The problem in base.sub.__init__ is getting extra from the extra.sub.
The only solutions I can think of at the moment involve having all subclasses of base provide some link to their encapsulating class type or turning the type of self in base.sub.__init__ into a string an manipulating it into a new type string. Both a bit ughly.
It's clearly possible to go the other way, type(self()).sub would give you extra.sub from inside base.sub.__init__ for a extra type object, but how do I do .. instead of .sub ? :)
The real answer is that there is no general way to do this. Python classes are normal objects, but they are created a bit differently. A class does not exist until well after its entire body has been executed. Once a class is created, it can be bound to many different names. The only reference it has to where it was created are the __module__ and __qualname__ attributes, but both of these are mutable.
In practice, it is possible to write your example like this:
class Sub:
def __init__(self):
pass
class Base:
Sub = Sub
Sub.__qualname__ = 'Base.Sub'
class Sub(Sub):
pass
class Extra(Base):
Sub = Sub
Sub.__qualname__ = 'Extra.Sub'
del Sub # Unlink from global namespace
Barring the capitalization, this behaves exactly as your original example. Hopefully this clarifies which code has access to what, and shows that the most robust way to determine the enclosing scope of a class is to explicitly assign it somewhere. You can do this in any number of ways. The trivial way is just to assign it. Going back to your original notation:
class Base:
class Sub:
def __init__(self):
print(self.enclosing)
Base.Sub.enclosing = Base
class Extra(Base):
class Sub(Base.Sub):
pass
Extra.Sub.enclosing = Extra
Notice that since Base does not exist when it body is being executed, the assignment has to happen after the classes are both created. You can bypass this by using a metaclass or a decorator. That will allow you to mess with the namespace before the class object is assigned to a name, making the change more transparent.
class NestedMeta(type):
def __init__(cls, name, bases, namespace):
for name, obj in namespace.items():
if isinstance(obj, type):
obj.enclosing = cls
class Base(metaclass=NestedMeta):
class Sub:
def __init__(self):
print(self.enclosing)
class Extra(Base):
class Sub(Base.Sub):
pass
But this is again somewhat unreliable because not all metaclasses are an instance of type, which takes us back to the first statement in this answer.
In many cases, you can use the __qualname__ and __module__ attributes to get the name of the surrounding class:
import sys
cls = type(o)
getattr(sys.modules[cls.__module__], '.'.join(cls.__qualname__.split('.')[:-1]))
This is a very literal answer to your question. It just shows one way of getting the class in the enclosing scope without addressing the probably design flaws that lead to this being necessary in the first place, or any of the many possible corner cases that this would not cover.
I have a class Step, which I want to derive by many sub-classes. I want every class deriving from Step to be "registered" by a name I choose for it (not the class's name), so I can later call Step.getStepTypeByName().
Something like this, only working :):
class Step(object):
_STEPS_BY_NAME = {}
#staticmethod
def REGISTER(cls, name):
_STEPS_BY_NAME[name] = cls
class Derive1(Step):
REGISTER(Derive1, "CustomDerive1Name")
...
class Derive2(Step):
REGISTER(Derive2, "CustomDerive2Name")
...
Your solution do not work for three reasons.
The first one is that _STEPS_BY_NAME only exists as an attribute of the Step class, so Step.REGISTER cannot access _STEPS_BY_NAME without a reference to the Step class. IOW you have to make it a classmethod (cf below)
The second one is that you need to explicitely use Step.REGISTER(cls) - the name REGISTER does not exist outside the Step class.
The third reason is that within a class statement's body, the class object has not yet been created not bound to it's name, so you cannot not reference the class itself at this point.
IOW, you'd want this instead:
class Step(object):
_STEPS_BY_NAME = {}
# NB : by convention, "ALL_UPPER" names denote pseudo-constants
#classmethod
def register(cls, name):
# here `cls` is the current class
cls._STEPS_BY_NAME[name] = stepclass
class Derive1(Step):
...
Step.register(Derive1, "CustomDerive1Name")
class Derive2(Step):
...
Step.register(Derive2, "CustomDerive2Name")
Now with a minor modification to Step.register you could use it as a class decorator, making things much clearer:
class Step(object):
_STEPS_BY_NAME = {}
#classmethod
def register(cls, name):
def _register(stepclass):
cls._STEPS_BY_NAME[name] = stepclass
return stepclass
return _register
#Step.register("CustomDerive1Name")
class Derive1(Step):
...
#Step.register("CustomDerive2Name")
class Derive2(Step):
...
As a last note: unless you have a compelling reason to register your subclasses in the base class itself, it might be better to use module-level variables and functions (a Python module is actually a kind of singleton):
# steps.py
class Step(object):
#....
_STEPS_BY_NAME = {}
def register(name):
def _register(cls):
_STEPS_BY_NAME[name] = cls
return cls
return _register
def get_step_class(name):
return _STEPS_BY_NAME[name]
And in your other modules
import steps
#steps.register("CustomDerive1Name")
class Derive1(steps.Step):
# ...
The point here is to avoid giving too many responsabilies to your Step class. I don't know your concrete use case so I can't tell which design best fits your need, but I've been using this last one on quite a few projects and it always worked fine so far.
You are close. Use this
class Step(object):
pass
class Derive1(Step):
pass
class Derive2(Step):
pass
_STEPS_BY_NAME = {
'foo': Step,
'bar': Derive1,
'bar': Derive2
}
def get_step_by_name(name):
return _STEPS_BY_NAME[name]
Warning: there might be better approaches depending on what you are trying to achieve. Such a mapping from strings to methods is a maintenance nightmare. If you want to change the name of a method, you would have to remember to change it in multiple place. You won't get any autocomplete help from your IDE either.
I have a class called A:
>>> class A:
def __init__(self):
self.register = {}
>>>
class A will be sub-classed by class B. class B however, contains methods that need to be registered as a name: function pair in instances of class A's dictionary. This is so class A can do work using the methods.
Here is an example of what I mean:
>>> class B(A):
def foo(self):
pass
def bar(self):
pass
>>> b = B()
>>> b.register # foo and bar were registered
{'key': <foo function>, 'key2': <bar function>}
>>>
Is there an idiomatic way to solve this? Or is something like this not possible, and it would be better to change my codes structure.
Note this is not a duplicate of Auto-register class methods using decorator because my register is not global, it is an instance variable of a class. This means using a meta-class like shown in the selected answer would not work.
I think the link to this question you mentioned in your post really can be used to solve your problem, if I understand it correctly.
The information you're trying to register is global information. While you want each instance to have a register containing this global information, all you really need to do is have __init__ copy the global register into the instance register.
If you will declare all classes you need, and after that worry about instance registers have references to all declared methods in all subclasses, you just need to performa a "register" information when declaring the subclasses themselves. That is easy to do in Python 3.6 (but not 3.5) with the new __init_subclass__ mechanism. In Python 3.5 and before that, it is easier performed using a metaclass.
class FallbackDict(dict):
def __init__(self, fallback):
self.fallback = fallback
def __missing__(self, key):
value = self.fallback[key]
self[key] = value
return value
class A:
register = {}
def __init__(self):
# instance register shadows global register
# for access via "self."
self.register = FallbackDict(__class__.register)
def __init_subclass__(cls):
for attrname, value in cls.__dict__.items():
if callable(value):
__class__.register[attrname] = value
The code here is meant to be simple - the custom dict class will "copy on read" values of the A.regiser dictionary into the instance dictionary. If you need a more consistent dictionary that include this behavior (for example, one that will iterate correctly the keys, values and items of both itself and the fallback dictionary) you'd better implement the "FallbackDict" class as an instance of collections.abc.MutableMapping instead (and just use an aggregate dictionary to actually keep the data)
You don't need the custom dict at all if you plan to create all your classes that register new methods before creating any instance of "A" - or if newly created classes don't have to update the existing instances - but in that case, you should copy A.register to the instance's "self.register" inside __init__.
If you can't change to Python 3.6 you will also need a custom metaclass to trigger the __init_subclass__ method above. Just keep it as is, so that your code is ready to move to Python 3.6 and eliminate the metaclass, and add a metaclass something like:
class MetaA(type):
def __init__(cls, name, bases, namespace):
super().__init__(name, bases, namespace)
cls.__init_subclass__()
class A:
...
#clasmethod
def __init_subclass__(cls):
# as above
...
I am reading up on how we ensure data encapsulation in python.One of the blog says
"Data Encapsulation means, that we should only be able to access private attributes via getters and setters"
Consider the following snippets from the blog:
class Robot:
def __init__(self, name=None, build_year=None):
self.name = name
self.build_year = build_year
Now, if i create the object of the class as below:
obj1=Robot()
obj1.name('Robo1")
obj1.build_year("1978")
Currently, i can access the attributes directly as i have defined them public(without the __notation)
Now to ensure data encapsulation, i need to define the attributes as privates
using the __ notation and access private attributes via getters and setters.
So the new class definition is as follows:
class Robot:
def __init__(self, name=None, build_year=2000):
self.__name = name
self.__build_year = build_year
def set_name(self, name):
self.__name = name
def get_name(self):
return self.__name
def set_build_year(self, by):
self.__build_year = by
def get_build_year(self):
return self.__build_year
Now i instantiate the class as below:
x = Robot("Marvin", 1979)
x.set_build_year(1993)
This way, i achive data encapsulation as private data members are no longer accessed directly and they can only be accessed via the class methods.
Q1:Why are we doing this? Who are we protecting the code from? Who is outside world?Anyone who has the source code can tweak it as per their requirement, so why at all do we add extra methods(get/set) to modify/tweak the attributes?
Q2:Is the above example considered data encapsulation?
Data encapsulation is slightly more general than access protection. name and build_year are encapsulated by the class Robot regardless of how you define the attributes. Python takes the position that getters and setters that do nothing more than access or assign to the underlying attribute are unnecessary.
Even using the double-underscore prefix is just advisory, and is more concerned with preventing name collisions in subclasses. If you really wanted to get to the __build_year attribute directly, you still could with
# Prefix attribute name with _Robot
x._Robot__build_year = 1993
A better design in Python is to use a property, which causes Python to invoke a defined getter and/or setter whenever an attribute is defined directly. For example:
class Robot(object):
def __init__(self, name, by):
self.name = name
self.build_year = by
#property
def name(self):
return self._name
#name.setter
def name(self, newname):
self._name = newname
#property
def build_year(self):
return self._build_year
#build_year.setter
def build_year(self, newby):
self._build_year = newby
You wouldn't actually define these property functions so simply, but a big benefit is that you can start by allowing direct access to a name attribute, and if you decide later that there should be more logic involved in getting/setting the value and you want to switch to properties, you can do so without affecting existing code. Code like
x = Robot("bob", 1993)
x.build_year = 1993
will work the same whether or not x.build_year = 1993 assigns to build_year directly or if it really triggers a call to the property setter.
About source code: sometimes you supply others with compiled python files that does not present the source, and you don't want people to get in mess with direct attribute assignments.
Now, consider data encapsulation as safe guards, last point before assigning or supplying values:
You may want to validate or process assignments using the sets, to make sure the assignment is valid for your needs or enters to the variable in the right format, (e.g. you want to check that attribute __build_year is higher than 1800, or that the name is a string). Very important in dynamic languages like python where a variable is not declared with a specific type.
Same goes for gets. You might want to return the year as a decimal, but use it as an integer in the class.
Yes, your example is a basic data encapsulation.
Suppose you have a python method that gets a type as parameter; is it possible to determine if the given type is a nested class?
E.g. in this example:
def show_type_info(t):
print t.__name__
# print outer class name (if any) ...
class SomeClass:
pass
class OuterClass:
class InnerClass:
pass
show_type_info(SomeClass)
show_type_info(OuterClass.InnerClass)
I would like the call to show_type_info(OuterClass.InnerClass) to show also that InnerClass is defined inside OuterClass.
AFAIK, given a class and no other information, you can't tell whether or not it's a nested class. However, see here for how you might use a decorator to determine this.
The problem is that a nested class is simply a normal class that's an attribute of its outer class. Other solutions that you might expect to work probably won't -- inspect.getmro, for example, only gives you base classes, not outer classes.
Also, nested classes are rarely needed. I would strongly reconsider whether that's a good approach in each particular case where you feel tempted to use one.
An inner class offers no particular special features in Python. It's only a property of the class object, no different from an integer or string property. Your OuterClass/InnerClass example can be rewritten exactly as:
class OuterClass(): pass
class InnerClass(): pass
OuterClass.InnerClass= InnerClass
InnerClass can't know whether it was declared inside another class, because that's just a plain variable binding. The magic that makes bound methods know about their owner ‘self’ doesn't apply here.
The innerclass decorator magic in the link John posted is an interesting approach but I would not use it as-is. It doesn't cache the classes it creates for each outer object, so you get a new InnerClass every time you call outerinstance.InnerClass:
>>> o= OuterClass()
>>> i= o.InnerClass()
>>> isinstance(i, o.InnerClass)
False # huh?
>>> o.InnerClass is o.InnerClass
False # oh, whoops...
Also the way it tries to replicate the Java behaviour of making outer class variables available on the inner class with getattr/setattr is very dodgy, and unnecessary really (since the more Pythonic way would be to call i.__outer__.attr explicitly).
Really a nested class is no different from any other class - it just happens to be defined somewhere else than the top-level namespace (inside another class instead). If we modify the description from "nested" to "non-top-level", then you may be able to come close enough to what you need.
eg:
import inspect
def not_toplevel(cls):
m = inspect.getmodule(cls)
return not (getattr(m, cls.__name__, []) is cls)
This will work for common cases, but it may not do what you want in situations where classes are renamed or otherwise manipulated after definition. For example:
class C: # not_toplevel(C) = False
class B: pass # not_toplevel(C.B) = True
B=C.B # not_toplevel(B) = True
D=C # D is defined at the top, but...
del C # not_toplevel(D) = True
def getclass(): # not_toplevel(getclass()) = True
class C: pass
Thank you all for your answers.
I've found this possible solution using metaclasses; I've done it more for obstination than real need, and it's done in a way that will not be applicable to python 3.
I want to share this solution anyway, so I'm posting it here.
#!/usr/bin/env python
class ScopeInfo(type): # stores scope information
__outers={} # outer classes
def __init__(cls, name, bases, dict):
super(ScopeInfo, cls).__init__(name, bases, dict)
ScopeInfo.__outers[cls] = None
for v in dict.values(): # iterate objects in the class's dictionary
for t in ScopeInfo.__outers:
if (v == t): # is the object an already registered type?
ScopeInfo.__outers[t] = cls
break;
def FullyQualifiedName(cls):
c = ScopeInfo.__outers[cls]
if c is None:
return "%s::%s" % (cls.__module__,cls.__name__)
else:
return "%s.%s" % (c.FullyQualifiedName(),cls.__name__)
__metaclass__ = ScopeInfo
class Outer:
class Inner:
class EvenMoreInner:
pass
print Outer.FullyQualifiedName()
print Outer.Inner.FullyQualifiedName()
print Outer.Inner.EvenMoreInner.FullyQualifiedName()
X = Outer.Inner
del Outer.Inner
print X.FullyQualifiedName()
If you do not set it yourself, I do not believe that there is any way to determine if the class is nested. As anyway a Python class cannot be used as a namespace (or at least not easily), I would say that the best thing to do is simply use different files.
Beginning with Python 3.3 there is a new attribute __qualname__, which provides not only the class name, but also the names of the outer classes:
In your sample this would result in:
assert SomeClass.__qualname__ == 'SomeClass'
assert OuterClass.InnerClass.__qualname__ == 'OuterClass.InnerClass'
If __qualname__ of a class does not contains a '.' it is an outer class!