We Keep Coding

Python allows adding a list and a dictionary if using += [duplicate]

The += operator in python seems to be operating unexpectedly on lists. Can anyone tell me what is going on here?
class foo:
bar = []
def __init__(self,x):
self.bar += [x]
class foo2:
bar = []
def __init__(self,x):
self.bar = self.bar + [x]
f = foo(1)
g = foo(2)
print f.bar
print g.bar
f.bar += [3]
print f.bar
print g.bar
f.bar = f.bar + [4]
print f.bar
print g.bar
f = foo2(1)
g = foo2(2)
print f.bar
print g.bar
OUTPUT
[1, 2]
[1, 2]
[1, 2, 3]
[1, 2, 3]
[1, 2, 3, 4]
[1, 2, 3]
[1]
[2]
foo += bar seems to affect every instance of the class, whereas foo = foo + bar seems to behave in the way I would expect things to behave.
The += operator is called a "compound assignment operator".

The general answer is that += tries to call the __iadd__ special method, and if that isn't available it tries to use __add__ instead. So the issue is with the difference between these special methods.
The __iadd__ special method is for an in-place addition, that is it mutates the object that it acts on. The __add__ special method returns a new object and is also used for the standard + operator.
So when the += operator is used on an object which has an __iadd__ defined the object is modified in place. Otherwise it will instead try to use the plain __add__ and return a new object.
That is why for mutable types like lists += changes the object's value, whereas for immutable types like tuples, strings and integers a new object is returned instead (a += b becomes equivalent to a = a + b).
For types that support both __iadd__ and __add__ you therefore have to be careful which one you use. a += b will call __iadd__ and mutate a, whereas a = a + b will create a new object and assign it to a. They are not the same operation!
>>> a1 = a2 = [1, 2]
>>> b1 = b2 = [1, 2]
>>> a1 += [3] # Uses __iadd__, modifies a1 in-place
>>> b1 = b1 + [3] # Uses __add__, creates new list, assigns it to b1
>>> a2
[1, 2, 3] # a1 and a2 are still the same list
>>> b2
[1, 2] # whereas only b1 was changed
For immutable types (where you don't have an __iadd__) a += b and a = a + b are equivalent. This is what lets you use += on immutable types, which might seem a strange design decision until you consider that otherwise you couldn't use += on immutable types like numbers!

For the general case, see Scott Griffith's answer. When dealing with lists like you are, though, the += operator is a shorthand for someListObject.extend(iterableObject). See the documentation of extend().
The extend function will append all elements of the parameter to the list.
When doing foo += something you're modifying the list foo in place, thus you don't change the reference that the name foo points to, but you're changing the list object directly. With foo = foo + something, you're actually creating a new list.
This example code will explain it:
>>> l = []
>>> id(l)
13043192
>>> l += [3]
>>> id(l)
13043192
>>> l = l + [3]
>>> id(l)
13059216
Note how the reference changes when you reassign the new list to l.
As bar is a class variable instead of an instance variable, modifying in place will affect all instances of that class. But when redefining self.bar, the instance will have a separate instance variable self.bar without affecting the other class instances.

The problem here is, bar is defined as a class attribute, not an instance variable.
In foo, the class attribute is modified in the init method, that's why all instances are affected.
In foo2, an instance variable is defined using the (empty) class attribute, and every instance gets its own bar.
The "correct" implementation would be:
class foo:
def __init__(self, x):
self.bar = [x]
Of course, class attributes are completely legal. In fact, you can access and modify them without creating an instance of the class like this:
class foo:
bar = []
foo.bar = [x]

There are two things involved here:
1. class attributes and instance attributes
2. difference between the operators + and += for lists
+ operator calls the __add__ method on a list. It takes all the elements from its operands and makes a new list containing those elements maintaining their order.
+= operator calls __iadd__ method on the list. It takes an iterable and appends all the elements of the iterable to the list in place. It does not create a new list object.
In class foo the statement self.bar += [x] is not an assignment statement but actually translates to
self.bar.__iadd__([x]) # modifies the class attribute
which modifies the list in place and acts like the list method extend.
In class foo2, on the contrary, the assignment statement in the init method
self.bar = self.bar + [x]
can be deconstructed as:
The instance has no attribute bar (there is a class attribute of the same name, though) so it accesses the class attribute bar and creates a new list by appending x to it. The statement translates to:
self.bar = self.bar.__add__([x]) # bar on the lhs is the class attribute
Then it creates an instance attribute bar and assigns the newly created list to it. Note that bar on the rhs of the assignment is different from the bar on the lhs.
For instances of class foo, bar is a class attribute and not instance attribute. Hence any change to the class attribute bar will be reflected for all instances.
On the contrary, each instance of the class foo2 has its own instance attribute bar which is different from the class attribute of the same name bar.
f = foo2(4)
print f.bar # accessing the instance attribute. prints [4]
print f.__class__.bar # accessing the class attribute. prints []
Hope this clears things.

Although much time has passed and many correct things were said, there is no answer which bundles both effects.
You have 2 effects:
a "special", maybe unnoticed behaviour of lists with += (as stated by Scott Griffiths)
the fact that class attributes as well as instance attributes are involved (as stated by Can Berk Büder)
In class foo, the __init__ method modifies the class attribute. It is because self.bar += [x] translates to self.bar = self.bar.__iadd__([x]). __iadd__() is for inplace modification, so it modifies the list and returns a reference to it.
Note that the instance dict is modified although this would normally not be necessary as the class dict already contains the same assignment. So this detail goes almost unnoticed - except if you do a foo.bar = [] afterwards. Here the instances's bar stays the same thanks to the said fact.
In class foo2, however, the class's bar is used, but not touched. Instead, a [x] is added to it, forming a new object, as self.bar.__add__([x]) is called here, which doesn't modify the object. The result is put into the instance dict then, giving the instance the new list as a dict, while the class's attribute stays modified.
The distinction between ... = ... + ... and ... += ... affects as well the assignments afterwards:
f = foo(1) # adds 1 to the class's bar and assigns f.bar to this as well.
g = foo(2) # adds 2 to the class's bar and assigns g.bar to this as well.
# Here, foo.bar, f.bar and g.bar refer to the same object.
print f.bar # [1, 2]
print g.bar # [1, 2]
f.bar += [3] # adds 3 to this object
print f.bar # As these still refer to the same object,
print g.bar # the output is the same.
f.bar = f.bar + [4] # Construct a new list with the values of the old ones, 4 appended.
print f.bar # Print the new one
print g.bar # Print the old one.
f = foo2(1) # Here a new list is created on every call.
g = foo2(2)
print f.bar # So these all obly have one element.
print g.bar
You can verify the identity of the objects with print id(foo), id(f), id(g) (don't forget the additional ()s if you are on Python3).
BTW: The += operator is called "augmented assignment" and generally is intended to do inplace modifications as far as possible.

The other answers would seem to pretty much have it covered, though it seems worth quoting and referring to the Augmented Assignments PEP 203:
They [the augmented assignment operators] implement the same operator
as their normal binary form, except that the operation is done
`in-place' when the left-hand side object supports it, and that the
left-hand side is only evaluated once.
...
The idea behind augmented
assignment in Python is that it isn't just an easier way to write the
common practice of storing the result of a binary operation in its
left-hand operand, but also a way for the left-hand operand in
question to know that it should operate `on itself', rather than
creating a modified copy of itself.

>>> elements=[[1],[2],[3]]
>>> subset=[]
>>> subset+=elements[0:1]
>>> subset
[[1]]
>>> elements
[[1], [2], [3]]
>>> subset[0][0]='change'
>>> elements
[['change'], [2], [3]]
>>> a=[1,2,3,4]
>>> b=a
>>> a+=[5]
>>> a,b
([1, 2, 3, 4, 5], [1, 2, 3, 4, 5])
>>> a=[1,2,3,4]
>>> b=a
>>> a=a+[5]
>>> a,b
([1, 2, 3, 4, 5], [1, 2, 3, 4])

>>> a = 89
>>> id(a)
4434330504
>>> a = 89 + 1
>>> print(a)
90
>>> id(a)
4430689552 # this is different from before!
>>> test = [1, 2, 3]
>>> id(test)
48638344L
>>> test2 = test
>>> id(test)
48638344L
>>> test2 += [4]
>>> id(test)
48638344L
>>> print(test, test2) # [1, 2, 3, 4] [1, 2, 3, 4]```
([1, 2, 3, 4], [1, 2, 3, 4])
>>> id(test2)
48638344L # ID is different here
We see that when we attempt to modify an immutable object (integer in this case), Python simply gives us a different object instead. On the other hand, we are able to make changes to an mutable object (a list) and have it remain the same object throughout.
ref : https://medium.com/#tyastropheus/tricky-python-i-memory-management-for-mutable-immutable-objects-21507d1e5b95
Also refer below url to understand the shallowcopy and deepcopy
https://www.geeksforgeeks.org/copy-python-deep-copy-shallow-copy/

listname.extend() works great for this purpose :)

python the difference between class(dict) and class(str)

I am wondering what kind of difference exists between class(dict) and class(str)
Here is my code
class MyDict3(str):
def __init__(self):
self.a = None
class MyDict(dict):
def __init__(self):
self.a = None
These classes are what I made for clarification
and then I type below
>>> mydict['a'] = 1
>>> mydict
{'a': 1}
>>> mydict3['a'] = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'MyDict3' object does not support item assignment
Why does my mydict3['a'] make an error?
The difference that I made is only MyDict(dict) and MyDict(str)
As far as I know, the object that I specified(dict, str) is just nothing but constructer like c++,java
Please give me a clear answer on that.

Why does my mydict3['a'] make an error? The difference that I made is only MyDict(dict) and MyDict(str) As far as I know, the object that I specified(dict, str) is just nothing but constructer like c++,java
I believe that you're doing a confusion here, thinking that a class attribute and an item are the same thing, like the following javascript code:
> foo = {'a': 42};
{ a: 42 }
> foo.a
42
> foo['a']
42
> foo.a === foo['a']
true
But in python foo.a and foo['a'] are two different mechanisms. When you call foo.a you're actually accessing the a attribute of a class, which is defined through the class definition:
class Foo:
def __init__(self):
self.a = 42 # declaring and defining the a attribute
so then you can access it using:
>>> foo = Foo()
>>> print(foo.a)
42
But to have foo['a'] working, you have to use the indexing mechanism, which is usually used for dicts or lists:
>>> foo = {'a': 42}
>>> foo['a']
42
That mechanism is being implemented by the __getitem__ method of your class, so you can overload it if you want:
class Foo:
def __getitem__(self, val):
if val == 'a':
return 42
raise KeyError('Unknown key') # when the key is unknown, you raise a key error exception
>>> foo = Foo()
>>> foo['a']
42
>>> foo['b']
KeyError: 'Unknown key'
So, the dict class is a class that implements __getitem__ (and __setitem__ and many others), in order to provide you a proper mapping mechanism called a dictionary. There keys can be any immutable objects, and values anything. For a list, it shall be only integers (which are the positions in the list).
That being said, let's answer your question:
Why does my mydict3['a'] make an error?
obviously it's because you defined mydict3 as being an implementation of a string, which has a special implementation for the __getitem__ method: it's giving you a character at the parameter position like if the list was a list of character (like in C).
So when you're trying to index mydict3 with 'a', python just tells you that what you're asking makes no sense!
So in the end, when you say:
The difference that I made is only MyDict(dict) and MyDict(str)
it's actually a very big difference! A dict and an str do not have the same interface, and thus what you want to do cannot work!
P.S.: Actually, nothing is black or white. The implementation of a class actually is a dict, and you can access all members of a class' instance through the __dict__ member of an instance:
class Foo():
def __init__(self):
self.a = 42
>>> foo = Foo()
>>> foo.__dict__['a']
42
but you shall never directly access the __dict__ instance directly, and use helper functions setattr and getattr:
>>> setattr(foo, 'b', 42)
>>> getattr(foo, 'b')
42
>>> getattr(foo, 'a')
42
This is some advanced python tricks, and they should be use with care. If there's really no other way to do it, then maybe you should use that.
Also, there exists a special class that transform dict items as class members, it's the namedtuple:
>>> from collections import namedtuple
>>> d = {'a': 42, 'b': 69}
>>> SpecialDict = namedtuple('SpecialDict', d.keys())
>>> foo = SpecialDict(**d)
>>> d.a
42
>>> d.b
69
HTH

Why doesn't del do the same thing?

Why does the following code change both variables:
>>> a = []
>>> b = a
>>> a.append(9)
>>> a
[9]
>>> b
[9]
>>>
But the del statement does not achieve the same effect?
>>> a = []
>>> b = a
>>> del(a)
>>> a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>> b
[]
>>>

When you do:
a = b
What you're doing is assigning the label b to the same object that the label a is refering to.
When you do:
a.append(9)
You're adding 9 to the list object pointed to by both a and b. It's the same object, so they show the same result.
When you do:
del a
You're deleting the reference to the object, not the object itself. If it's the only reference, then the object will be garbage collected. But in your case, there's another reference - b - so the object continues to exist.

Instead of "variables", think in terms of names and objects.
>>> a = []
This makes an empty list object and binds the name a to it.
>>> b = a
This simply says that b is now a new name for the object named by a. We have
>>> a is b
True
del a means that we're forgetting the name a: it is no longer bound to an object.
>>> del a
>>> a
Traceback (most recent call last):
File "<ipython-input-8-60b725f10c9c>", line 1, in <module>
a
NameError: name 'a' is not defined
But that you're no longer calling that list object a, only b, doesn't affect the object itself in any way. Objects don't care, or even know about, what names you've given them. [One semi-exception is that if an object no longer has any references, it may -- or may not, no promises -- be garbage-collected.]

The append method works against the actual object, while del works against the reference i.e. variable name.

(I already answered the question in the other question of yours, so I'm going to use it here as well, with slight modifications:)
del does not delete objects; in fact in Python, it is not even possible to tell the interpreter/VM to remove an object from memory because Python is a garbage collected language (like Java, C#, Ruby, Haskell etc).
Instead, what del does when called on a variable (as opposed to a dictionary key or list item) like this:
del a
is that it only removes the local (or global) variable not what it points to (every variable in Python holds a pointer/reference to its contents not the content itself). In fact, since locals and globals are stored as a dictionary under the hood (see locals() and globals()), del a is equivalent to:
del locals()['a']
(or del globals()['a'] when applied to a global.)
so if you have:
a = []
b = a
you're making a list, storing a reference to it in a and then copying that reference into b without copying/touching the list object itself. Therefore these two calls affect one and the same object:
>>> a.append(1)
>>> b.append(2)
>>> a
[1, 2]
>>> b
[1, 2]
>>> a is b # would be False for 2 identical but different list objects
True
>>> id(a) == id(b)
True
(id returns the memory address of an object)
whereas deleting b is in no way related to touching what b points to:
>>> a = []
>>> b = a
>>> del b # a is still untouched and points to a list
>>> b
NameError: name 'b' is not defined
>>> a
[]

Python, how to deal with A(a) when type(a) is yet A

I need to create a class that mimics this behavior (in mathematics, we say list, dict, are "idempotent"):
>>> list(list([3,4]))
[3, 4]
>>> dict({'a':1,'b':2})
{'a':1,'b':2}
So, if A is my class, I want to write
>>> a = A(1)
>>> b = A(a)
>>> b == a
True
I imagine my class A has to look like this :
class A(object):
def __init__(self,x):
if isinstance(x, A) :
self = x
else :
self.x = x
self.y = 'hello'
I try it
>>> A(1).x
1
>>> A(A(1)).x
Traceback (most recent call last):
File "<input>", line 1, in <module>
AttributeError: 'A' object has no attribute 'x'
It does not work !
I don't want to copy x attributes in self, i just want self to BE x or "point" x
Some idea ?
Thanks

What you are looking for is the __new__() method, which takes is run before the class is constructed, as opposed to __init__(), which takes place after. With __new__() you can hook in and replace the object being created.
def __new__(cls, x):
if isinstance(x, A):
return x
else:
return object.__new__(cls, x)
You can't do this in __init__() as the object has already been created. Changing self simply changes the value of the local variable, it doesn't affect the object.
It's also worth noting that type-checking is almost always the wrong thing to do in Python. Instead, check to see if the class has the information/attributes you need. This way, someone can create a class that acts like yours and works with your code.
As a final word of warning, this is pretty confusing behaviour - people won't expect your class to act like this and it's generally not a great idea. Your example of list() and dict() isn't accurate to what you are doing here, as list(some_list) does not give some_list, it gives a new list which is a copy of some_list - the same is true for dict():
>>> x = [1, 2, 3]
>>> list(x) is x
False
When you call a constructor, it's natural to expect a new object, rather than a reference to the existing one. I would recommend making A(some_a) copy some_a, and restructure your calling code not to rely on A(some_a) is some_a).

Understanding instance and class variable python

Let's suppose I have 2 classes in different scenario.
Scenario 1
class MyClass():
temp = 5
Scenario 2
class MyClass():
temp = 5
def myfunc(self):
print self.temp
Now when will variable temp will be treated as a class variable and instance variable. I am confused because in both the scenarios I am able to access the value of variable temp using both.
Object.Temp (behaving as instance variable)
ClassName.Temp (behaving as class variable)
I believe similar questions have been asked before but it will be a great help if someone can explain this in context of my question.

Class variables are shared between all instances of a class. With immutable types (like int, str, ...) you won't note much of a difference. But consider this:
class MyClass():
temp = []
def myfunc(self, val):
self.temp.append(val)
print self.temp
instance1 = MyClass()
instance1.myfunc(1) # [1]
instance2 = MyClass()
instance2.myfunc(2) # [1, 2]
In this case both instances share the same list, that is if the instance doesn't have a temp member itself, then that of the class is used.
So if you further do:
MyClass.temp.append(3)
print instance1.temp # [1, 2, 3]
instance1.temp = []
print instance1.temp # [] uses the instances temp
print instance2.temp # [1, 2, 3]
del instance1.temp
print instance1.temp # [1, 2, 3] uses the class' temp again

Basically, MyClass.temp is always a class variable. Getting obj.temp returns the class variable, until you try to set obj.temp, which creates a member variable that masks the class variable. I hope this helps:
>>> class MyClass(object):
... temp = 5
...
>>> a = MyClass()
>>> b = MyClass()
>>> a.temp
5
>>> b.temp
5
>>> b.temp = 6
>>> a.temp
5
>>> MyClass.temp = 7
>>> a.temp
7
>>> b.temp
6
>>> a.__dict__
{}
>>> b.__dict__
{'temp': 6}
>>> MyClass.__dict__
{..., 'temp': 7}
Edit: As mata says, calling methods (such as append()) on obj.temp does not count as "setting" it.

temp is a class variable. When you access the variable it is searched through the layers of inheritance so since it is not found in the instance itself it checks the class(next layer up) and finds it there.