We Keep Coding

Why is super() not behaving like I expected when assigning to a class variable of the base class?

I am attempting to experiment with classes so I can better understand what they do. I wanted to build a counter which records the number of instances of a class (MyClass):
class ObjectCounter: # I want this to count the number of objects in each class
myclass_obj_count = 0
class MyClass(ObjectCounter):
def __init__(self):
super().myclass_obj_count += 1 # AttributeError: 'super' object has no attribute 'myclass_obj_count'
m1 = MyClass()
m2 = MyClass()
m3 = MyClass()
print(ObjectCounter.myclass_obj_count)
Since that didn't work, I looked online for someone trying to do the same thing. Here is some code I found online. This works as expected, and I feel like I have a basic understanding of how this works. This is a better solution to the task I was attempting, but I'm not satisfied because I want to know how super() works.
class geeks:
counter = 0
def __init__(self):
geeks.counter += 1
g1 = geeks()
g2 = geeks()
g3 = geeks()
print(geeks.counter) # this gives an expected result
Therefore, I tried this instead:
class ObjectCounter: # I want this to count the number of objects in each class
myclass_obj_count = 0
def add_myclass(self):
self.myclass_obj_count += 1
class MyClass(ObjectCounter):
def __init__(self):
super().add_myclass()
my_class_1 = MyClass()
my_class_2 = MyClass()
my_class_3 = MyClass()
print(ObjectCounter.myclass_obj_count) # expected output: 3
Instead of getting the expected output of 3, I got an output of 0. Why is this happening?

First, be aware of the += operator; it's implementation is quite subtle:
a += b
becomes
a = a.__iadd__(b)
This perhaps strange definition allows python to support it even for immutable types (like strings).
Note what happens when used for a class variable that is referred to by the alias self
class ObjectCounter: # I want this to count the number of objects in each class
myclass_obj_count = 0
def add_myclass(self):
self.myclass_obj_count += 1
# effectively becomes:
# self.myclass_obj_count = self.myclass_obj_count.__iadd__(1)
This will introduce an instance variable of the same name, shadowing the class variable.
You don't even need the subclass to test this:
>>> x = ObjectCounter()
>>> x.add_myclass()
>>> x.add_myclass()
>>> x.add_myclass()
>>> x.myclass_obj_count
3
>>> ObjectCounter.myclass_obj_count
0
Referring to the class variable directly instead of using self fixes this
def add_myclass(self):
ObjectCounter.myclass_obj_count += 1
I'm hesitant to give definite answers of what happens under the hood when class variables, super() and assignments are used, other than it just doesn't work. Perhaps because it would be quite ambiguous of whether or not we are defining class variables or new instance variables.
super() won't let you assign to either;
class ObjectCounter:
myclass_obj_count = 0
def __init__(self):
self.x = 'test'
class MyClass(ObjectCounter):
def __init__(self):
super().__init__()
print(super().myclass_obj_count) # reading works just fine
print(type(super())) # this isn't actually exactly the same as "ObjectCounter"
super().myclass_obj_count = 123 # no good
super().x = 'foo' # also no good.
All in all, for any assignment to class variables you can use the class name itself.

Modifying a class attribute from instance, different behaviour depending on type?

If I create a class in Python and I give it a class attribute (this is taken directly from the docs, here), as
class Dog:
tricks = []
def __init__(self, name):
self.name = name
def add_trick(self, trick):
self.tricks.append(trick)
I see that, as the docs suggest, when doing
d1 = Dog('d1')
d2 = Dog('d2')
d1.add_trick('trick')
print d2.tricks
I get that the trick is added to d2 as well:
['trick']
This is because tricks is a class attribute rather than an instance attribute so gets shared across all instances (correct me if this is not orthodox!).
Now, suppose I do this instead
class Dog:
a = 1
def __init__(self, name):
self.name = name
def improve_a(self):
self.a += 1
and I run
d1 = Dog('d1')
d2 = Dog('d2')
d1.improve_a()
print d1.a, d2.a
this gives me 2 and 1 respectively, namely the count for the second instance has not changed. Why is this, why the behaviour difference?

The int class does not define the += operator (__iadd__ method). That wouldn't make sense because it is immutable.
That's why += defaults to + and then =. reference
self.a += 1 becomes self.a = self.a + 1
Now the first time you call improve_a the following happens:
read class attribute a and put it on the stack
add 1 to the item on the stack
create a new instance attribute a and assign it the value on the stack
That means the class attribute is not changed at all and you add a new instance attribute.
On every subsequent call of improve on the same object the instance attribute is incremented, because attribute lookup starts on the instance dict and will only go to the class dict if that attribute does not exist.
If you do the same with a mutable class which overloads the __iadd__ method you can get different behaviour:
class HasList:
some_list = []
def add_something(self, value):
some_list += [value]
fst = HasList()
sec = HasList()
fst.add_something(1)
fst.add_something(2)
sec.add_something(3)
print(HasList.some_list, fst.some_list, sec.some_list)
You will see that all instances and the class itself still hold the same list. The print shows the same list [1, 2, 3] each time. You can also check that all three lists are identical: fst.some_list is sec.some_list and fst.some_list is HasList.some_list # -> True.
That is because list.__iadd__ just calls list.extend and returns itself inside the method (at least if it was written in python).

Python - Can't change value of item inside a class

I am new to Python, and don't understand why I can't do this.
When I try to change the values in Packet.ItemData from the parent object, it doesn't work. See the "Doesn't work" comments in the code.
import json
from copy import deepcopy
class Event():
__slots__= 'itemName'
def __init__(self, itemName):
self.itemName = itemName
def encode(self):
obj = {}
obj['itemName'] = str(self.itemName)
return json.dumps(obj)
def decode(self, json_Str):
obj = json.loads(json_Str)
self.itemName = obj['itemName']
class Packet():
__slots__= 'pID', 'itemData'
def __init__(self, pID, itemData):
self.pID = pID
self.itemData = itemData
def encode(self):
obj = {}
obj['pID'] = int(self.pID)
obj['itemData'] = str(self.itemData.encode())
return json.dumps(obj)
def decode(self, json_Str):
obj = json.loads(json_Str)
self.pID = obj['pID']
self.itemData = Event(0,'')
defaultEvent = Event('Dflt')
defaultPacket = Packet(1, defaultEvent)
event2 = Event('NoName')
print 'event : ', event2.encode()
packet3 = deepcopy(defaultPacket)
packet3.ItemData = event2; #direct assign doesn't work
packet3.ItemData = deepcopy(event2); #deep copy doesn't work
packet3.ItemData.itemName = 'Hello' #event this doesn't work
print 'packet : ', packet3.encode()
I wanted to wrap up the data so I get exactly what I am expecting when encoding and decoding from JSON.

What FJ pointed out is indeed correct...the name of the attribute is itemData not ItemData.
Being new to Python you may have expected that typo to throw an error...something like 'Packet has no attribute ItemData', but this does not happen. Python classes by default store attributes in a dictionary, so assigning to one that did not exist previously is just like adding a new mapping to the dictionary.
Curiously, your code in a roundabout way highlights an exception to this...and that has to do with the descriptor __slots__. __slots__ changes the class to no longer store attributes in a dynamic dictionary, but in a static structure instead. See Usage of __slots__? for more info.
Normally, when __slots__ is defined you can not assign to an attribute that wasn't specified in __slots__. So you may ask....since you defined __slots__ for your classes why didn't you get an AttributeError? The answer is __slots__ will only work for 'new-style classes' See What is the difference between old style and new style classes in Python? to learn about the distinction between old and new-style classes.
Had you defined Packet to inherit from the base class object like class Packet(object): instead of just class Packet() it would have made it a new-style class and you would have indeed gotten an AttributeError when you tried to assign to ItemData

The attribute name is itemData, not ItemData. If you change all of the packet3.ItemData references to packet3.itemData this should work fine.

Restore Python class to original state

I have a class where I add some attributes dynamically and at some point I want to restore the class to it's pristine condition without the added attributes.
The situation:
class Foo(object):
pass
Foo.x = 1
# <insert python magic here>
o = Foo() # o should not have any of the previously added attributes
print o.x # Should raise exception
My initial thought was to create a copy of the original class:
class _Foo(object):
pass
Foo = _Foo
Foo.x = 1
Foo = _Foo # Clear added attributes
o = Foo()
print o.x # Should raise exception
But since Foo is just a reference to _Foo any attributes get added to the original _Foo as well. I also tried
Foo = copy.deepcopy(_Foo)
in case that would help but apparently it does not.
clarification:
The user should not need to care about how the class is implemented. It should, therefore, have the same features of a "normally defined" class, i.e. introspection, built-in help, subclassing, etc. This pretty much rules out anything based on __getattr__

I agree with Glenn that this is a horribly broken idea. Anyways, here how you'd do it with a decorator. Thanks to Glenn's post as well for reminding me that you can delete items from a class's dictionary, just not directly. Here's the code.
def resetable(cls):
cls._resetable_cache_ = cls.__dict__.copy()
return cls
def reset(cls):
cache = cls._resetable_cache_ # raises AttributeError on class without decorator
for key in [key for key in cls.__dict__ if key not in cache]:
delattr(cls, key)
for key, value in cache.items(): # reset the items to original values
try:
setattr(cls, key, value)
except AttributeError:
pass
I'm torn on whether to reset the values by catching attempts to update non-assignable attributes with a try as I've shown or building a list of such attributes. I'll leave it up to you.
And here's a use:
#resetable # use resetable on a class that you want to do this with
class Foo(object):
pass
Foo.x = 1
print Foo.x
reset(Foo)
o = Foo()
print o.x # raises AttributeError as expected

You can use inspect and maintain an original list of members and than delete all members that are not in the original list
import inspect
orig_members = []
for name, ref in inspect.getmembers(o):
orig_members.append(name)
...
Now, when you need to restore back to original
for name, ref in inspect.getmembers(o):
if name in orig_members:
pass
else:
#delete ref here

You have to record the original state and restore it explicitly. If the value existed before you changed it, restore that value; otherwise delete the value you set.
class Foo(object):
pass
try:
original_value = getattr(Foo, 'x')
originally_existed = True
except AttributeError:
originally_existed = False
Foo.x = 1
if originally_existed:
Foo.x = original_value
else:
del Foo.x
o = Foo() # o should not have any of the previously added attributes
print o.x # Should raise exception
You probably don't want to be doing this. There are valid cases for monkey patching, but you generally don't want to try to monkey unpatch. For example, if two independent bits of code monkey patch the same class, one of them trying to reverse the action without being aware of the other is likely to break things. For an example of a case where this is actually useful, see https://stackoverflow.com/questions/3829742#3829849.

The simplest way I found was this:
def foo_maker():
class Foo(object):
pass
return Foo
Foo = foo_maker()
Foo.x = 1
Foo = foo_maker() # Foo is now clean again
o = Foo() # Does not have any of the previously added attributes
print o.x # Raises exception
edit: As pointed out in comments, does not actually reset class but has the same effect in practice.

In your second example you're making a reference to the class rather than an instance.
Foo = _Foo # Reference
If you instead made an instance copy, what you want to do is exactly the way it will work. You can modify the instance all you want and 'revert' it by creating a new instance.
Foo = _Foo()
#!/usr/bin/python
class FooClass(object):
pass
FooInstance = FooClass() # Create an instance
FooInstance.x = 100 # Modify the instance
print dir(FooClass) # Verify FooClass doesn't have an 'x' attribute
FooInstance = FooClass() # Creates a new instance
print FooInstance.x # Exception

I don't know if you can accept an additional module file for class, if you can:
my_class.py
class Foo(object):
pass
You main script:
import my_class
Foo = my_class.Foo
Foo.x = 1
p = Foo()
print p.x # Printing '1'
# Some code....
reload(my_class) # reload to reset
Foo = my_class.Foo
o = Foo()
print p.x # Printing '1'
print o.__class__ == p.__class__ # Printing 'False'
print o.x # Raising exception
I am not sure if there is any side-effect. It seems to do what OP wants, though this is really unusal.

I don't fully understand why you need this, but I'll have a go. Ordinary inheritance probably won't do because you want to 'reset' to the old state. How about a proxy pattern?
class FooProxy(object):
def __init__(self, f):
self.f = foo
self.magic = {}
def set_magic(self, k, v):
self.magic[k] = v
def get_magic(self, k):
return self.magic.get(k)
def __getattr__(self, k):
return getattr(self.f, k)
def __setattr__(self, k, v):
setattr(self.f, k, v)
f = Foo()
p = FooProxy(f)
p.set_magic('m_bla', 123)
use f for ordinary, 'original' access, use p for proxied access, it should behave mostly like Foo. Re-proxy f with new configuration if you need to

I don't understand what you are trying to do, but keep in mind that you don't have to add attributes to the class in order to make it look like you added attributes to the class.
You can give the class a __getattr__ method that will be invoked for any missing attribute. Where it gets the value from is up to you:
class MyTrickyClass(object):
self.magic_prefix = "m_"
self.other_attribute_source = SomeOtherObject()
def __getattr__(self, name):
if name.startswith(self.magic_prefix):
stripped = name[len(self.magic_prefix):]
return getattr(self.other_attribute_source, stripped)
raise AttributeError
m = MyTrickyClass()
assert hasattr(m, "m_other")
MyTrickyClass.magic_prefix = "f_"
assert hasattr(m, "f_other")

If all the stuff you added starts with a given distinctive prefix, you could search the object's __dict__ for members with that prefix, and delete them, when it's time to restore.

To create a deep copy of a class you can use the new.classobj function
class Foo:
pass
import new, copy
FooSaved = new.classobj(Foo.__name__, Foo.__bases__, copy.deepcopy(Foo.__dict__))
# ...play with original class Foo...
# revert changes
Foo = FooSaved
UPD: module new is deprecated. Instead you should use types.ClassType with the same args

Getting an instance name inside class __init__() [duplicate]

This question already has answers here:
Getting the name of a variable as a string
(32 answers)
Closed 3 years ago.
While building a new class object in python, I want to be able to create a default value based on the instance name of the class without passing in an extra argument. How can I accomplish this? Here's the basic pseudo-code I'm trying for:
class SomeObject():
defined_name = u""
def __init__(self, def_name=None):
if def_name == None:
def_name = u"%s" % (<INSTANCE NAME>)
self.defined_name = def_name
ThisObject = SomeObject()
print ThisObject.defined_name # Should print "ThisObject"

Well, there is almost a way to do it:
#!/usr/bin/env python
import traceback
class SomeObject():
def __init__(self, def_name=None):
if def_name == None:
(filename,line_number,function_name,text)=traceback.extract_stack()[-2]
def_name = text[:text.find('=')].strip()
self.defined_name = def_name
ThisObject = SomeObject()
print ThisObject.defined_name
# ThisObject
The traceback module allows you to peek at the code used to call SomeObject().
With a little string wrangling, text[:text.find('=')].strip() you can
guess what the def_name should be.
However, this hack is brittle. For example, this doesn't work so well:
ThisObject,ThatObject = SomeObject(),SomeObject()
print ThisObject.defined_name
# ThisObject,ThatObject
print ThatObject.defined_name
# ThisObject,ThatObject
So if you were to use this hack, you have to bear in mind that you must call SomeObject()
using simple python statement:
ThisObject = SomeObject()
By the way, as a further example of using traceback, if you define
def pv(var):
# stack is a list of 4-tuples: (filename, line number, function name, text)
# see http://docs.python.org/library/traceback.html#module-traceback
#
(filename,line_number,function_name,text)=traceback.extract_stack()[-2]
# ('x_traceback.py', 18, 'f', 'print_var(y)')
print('%s: %s'%(text[text.find('(')+1:-1],var))
then you can call
x=3.14
pv(x)
# x: 3.14
to print both the variable name and its value.

Instances don't have names. By the time the global name ThisObject gets bound to the instance created by evaluating the SomeObject constructor, the constructor has finished running.
If you want an object to have a name, just pass the name along in the constructor.
def __init__(self, name):
self.name = name

You can create a method inside your class that check all variables in the current frame and use hash() to look for the self variable.
The solution proposed here will return all the variables pointing to the instance object.
In the class below, isinstance() is used to avoid problems when applying hash(), since some objects like a numpy.array or a list, for example, are unhashable.
import inspect
class A(object):
def get_my_name(self):
ans = []
frame = inspect.currentframe().f_back
tmp = dict(frame.f_globals.items() + frame.f_locals.items())
for k, var in tmp.items():
if isinstance(var, self.__class__):
if hash(self) == hash(var):
ans.append(k)
return ans
The following test has been done:
def test():
a = A()
b = a
c = b
print c.get_my_name()
The result is:
test()
#['a', 'c', 'b']

This cannot work, just imagine this: a = b = TheMagicObjet(). Names have no effect on Values, they just point to them.

One horrible, horrible way to accomplish this is to reverse the responsibilities:
class SomeObject():
def __init__(self, def_name):
self.defined_name = def_name
globals()[def_name] = self
SomeObject("ThisObject")
print ThisObject.defined_name
If you wanted to support something other than global scope, you'd have to do something even more awful.

In Python, all data is stored in objects. Additionally, a name can be bound with an object, after which that name can be used to look up that object.
It makes no difference to the object what names, if any, it might be bound to. It might be bound to dozens of different names, or none. Also, Python does not have any "back links" that point from an object to a name.
Consider this example:
foo = 1
bar = foo
baz = foo
Now, suppose you have the integer object with value 1, and you want to work backwards and find its name. What would you print? Three different names have that object bound to them, and all are equally valid.
print(bar is foo) # prints True
print(baz is foo) # prints True
In Python, a name is a way to access an object, so there is no way to work with names directly. You could search through various name spaces until you find a name that is bound with the object of interest, but I don't recommend this.
How do I get the string representation of a variable in python?
There is a famous presentation called "Code Like a Pythonista" that summarizes this situation as "Other languages have 'variables'" and "Python has 'names'"
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#other-languages-have-variables

If you want an unique instance name for a class, try __repr__() or id(self)
class Some:
def __init__(self):
print(self.__repr__()) # = hex(id(self))
print(id(self))
It will print the memory address of the instance, which is unique.

Inspired by the answers of unutbu and Saullo Castro, I have created a more sophisticated class that can even be subclassed. It solves what was asked for in the question.
"create a default value based on the instance name of the class
without passing in an extra argument."
Here's what it does, when an instance of this class or a subclass is created:
Go up in the frame stack until the first frame which does not belong to a method of the current instance.
Inspect this frame to get the attributes self.creation_(name/file/module/function/line/text).
Perform an an additional check whether an object with name self.creation_name was actually defined in the frame's locals() namespace to make 100% sure the found creation_name is correct or raise an error otherwise.
The Code:
import traceback, threading, time
class InstanceCreationError(Exception):
pass
class RememberInstanceCreationInfo:
def __init__(self):
for frame, line in traceback.walk_stack(None):
varnames = frame.f_code.co_varnames
if varnames is ():
break
if frame.f_locals[varnames[0]] not in (self, self.__class__):
break
# if the frame is inside a method of this instance,
# the first argument usually contains either the instance or
# its class
# we want to find the first frame, where this is not the case
else:
raise InstanceCreationError("No suitable outer frame found.")
self._outer_frame = frame
self.creation_module = frame.f_globals["__name__"]
self.creation_file, self.creation_line, self.creation_function, \
self.creation_text = \
traceback.extract_stack(frame, 1)[0]
self.creation_name = self.creation_text.split("=")[0].strip()
super().__init__()
threading.Thread(target=self._check_existence_after_creation).start()
def _check_existence_after_creation(self):
while self._outer_frame.f_lineno == self.creation_line:
time.sleep(0.01)
# this is executed as soon as the line number changes
# now we can be sure the instance was actually created
error = InstanceCreationError(
"\nCreation name not found in creation frame.\ncreation_file: "
"%s \ncreation_line: %s \ncreation_text: %s\ncreation_name ("
"might be wrong): %s" % (
self.creation_file, self.creation_line, self.creation_text,
self.creation_name))
nameparts = self.creation_name.split(".")
try:
var = self._outer_frame.f_locals[nameparts[0]]
except KeyError:
raise error
finally:
del self._outer_frame
# make sure we have no permament inter frame reference
# which could hinder garbage collection
try:
for name in nameparts[1:]: var = getattr(var, name)
except AttributeError:
raise error
if var is not self: raise error
def __repr__(self):
return super().__repr__()[
:-1] + " with creation_name '%s'>" % self.creation_name
A simple example:
class MySubclass(RememberInstanceCreationInfo):
def __init__(self):
super().__init__()
def print_creation_info(self):
print(self.creation_name, self.creation_module, self.creation_function,
self.creation_line, self.creation_text, sep=", ")
instance = MySubclass()
instance.print_creation_info()
#out: instance, __main__, <module>, 68, instance = MySubclass()
If the creation name cannot be determined properly an error is raised:
variable, another_instance = 2, MySubclass()
# InstanceCreationError:
# Creation name not found in creation frame.
# creation_file: /.../myfile.py
# creation_line: 71
# creation_text: variable, another_instance = 2, MySubclass()
# creation_name (might be wrong): variable, another_instance

I think that names matters if they are the pointers to any object..
no matters if:
foo = 1
bar = foo
I know that foo points to 1 and bar points to the same value 1 into the same memory space.
but supose that I want to create a class with a function that adds a object to it.
Class Bag(object):
def __init__(self):
some code here...
def addItem(self,item):
self.__dict__[somewaytogetItemName] = item
So, when I instantiate the class bag like below:
newObj1 = Bag()
newObj2 = Bag()
newObj1.addItem(newObj2)I can do this to get an attribute of newObj1:
newObj1.newObj2

The best way is really to pass the name to the constructor as in the chosen answer. However, if you REALLY want to avoid asking the user to pass the name to the constructor, you can do the following hack:
If you are creating the instance with 'ThisObject = SomeObject()' from the command line, you can get the object name from the command string in command history:
import readline
import re
class SomeObject():
def __init__(self):
cmd = readline.get_history_item(readline.get_current_history_length())
self.name = re.split('=| ',cmd)[0]
If you are creating the instance using 'exec' command, you can handle this with:
if cmd[0:4] == 'exec': self.name = re.split('\'|=| ',cmd)[1] # if command performed using 'exec'
else: self.name = re.split('=| ',cmd)[0]