Related
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
I understand that in Python regular c++ style variable assignment is replaced by references to stuff ie
a=[1,2,3]
b=a
a.append(4)
print(b) #gives [1,2,3,4]
print(a) #gives [1,2,3,4]
but I'm still confused why an analogous situation with basic types eg. integers works differently?
a=1
b=a
a+=1
print(b) # gives 1
print(a) # gives 2
But wait, it gets even more confusing when we consider loops!
li=[1,2,3]
for x in li:
x+=1
print(li) #gives [1,2,3]
Which is what I expected, but what happens if we do:
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #gives [1,2,3]
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there. The only thing I can come up short of using
for x in range(len(li)):
Do stuff to li[x]
is packaging the integers in one element list. But there must be a better way.
Well, you need to think of mutable and immutable type.
For a list, it's mutable.
For a integer, it's immutable, which means you will refer to a new object if you change it. When a+=1 is executed, a will be assigned a new object, but b is still refer to the same one.
a=[1,2,3]
b=a
a.append(4)
print(b) #[1,2,3,4]
print(a) #[1,2,3,4]
Here you are modifying the list. The list content changes, but the list identity remains.
a=1
b=a
a+=1
This, however, is a reassignment. You assign a different object to a.
Note that if you did a += [4] in the 1st example, you would have seen the same result. This comes from the fact that a += something is the same as a = a.__iadd__(something), with a fallback to a = a.__add__(something) if __iadd__() doesn't exist.
The difference is that __iadd__() tries to do its job "inplace", by modifying the object it works on and returning it. So a refers to the same as before. This only works with mutable objects such as lists.
On immutable objects such as ints __add__() is called. It returns a different object, which leads to a pointing to another object than before. There is no other choice, as ints are immutable.
a,b,c=1,2,3
li=[a,b,c]
for x in li:
x+=1
print(li) #[1,2,3]
Here x += 1 means the same as x = x + 1. It changes where x refers to, but not the list contents.
Maybe my question should be how to loop over a list of integers and change them without >map() as i need a if statement in there.
for i, x in enumerate(li):
li[i] = x + 1
assigns to every list position the old value + 1.
The important thing here are the variable names. They really are just keys to a dictionary. They are resolved at runtime, depending on the current scope.
Let's have a look what names you access in your code. The locals function helps us: It shows the names in the local scope (and their value). Here's your code, with some debugging output:
a = [1, 2, 3] # a is bound
print(locals())
for x in a: # a is read, and for each iteration x is bound
x = x + 3 # x is read, the value increased and then bound to x again
print(locals())
print(locals())
print(x)
(Note I expanded x += 3 to x = x + 3 to increase visibility for the name accesses - read and write.)
First, you bind the list [1, 2, 3]to the name a. Then, you iterate over the list. During each iteration, the value is bound to the name x in the current scope. Your assignment then assigns another value to x.
Here's the output
{'a': [1, 2, 3]}
{'a': [1, 2, 3], 'x': 4}
{'a': [1, 2, 3], 'x': 5}
{'a': [1, 2, 3], 'x': 6}
{'a': [1, 2, 3], 'x': 6}
6
At no point you're accessing a, the list, and thus will never modify it.
To fix your problem, I'd use the enumerate function to get the index along with the value and then access the list using the name a to change it.
for idx, x in enumerate(a):
a[idx] = x + 3
print(a)
Output:
[4, 5, 6]
Note you might want to wrap those examples in a function, to avoid the cluttered global namespace.
For more about scopes, read the chapter in the Python tutorial. To further investigate that, use the globals function to see the names of the global namespace. (Not to be confused with the global keyword, note the missing 's'.)
Have fun!
For a C++-head it easiest tho think that every Python object is a pointer. When you write a = [1, 2, 3] you essentially write List * a = new List(1, 2, 3). When you write a = b, you essentially write List * b = a.
But when you take out actual items from the lists, these items happen to be numbers. Numbers are immutable; holding a pointer to an immutable object is about as good as holding this object by value.
So your for x in a: x += 1 is essentially
for (int x, it = a.iterator(); it->hasMore(); x=it.next()) {
x+=1; // the generated sum is silently discarded
}
which obviously has no effect.
If list elements were mutable objects you could mutate them exactly the way you wrote. See:
a = [[1], [2], [3]] # list of lists
for x in a: # x iterates over each sub-list
x.append(10)
print a # prints [[1, 10], [2, 10], [3, 10]]
But unless you have a compelling reason (e.g. a list of millions of objects under heavy memory load) you are better off making a copy of the list, applying a transformation and optionally a filter. This is easily done with a list comprehension:
a = [1, 2, 3, 0]
b = [n + 1 for n in a] # [2, 3, 4, 1]
c = [n * 10 for n in a if n < 3] # [10, 20, 0]
Either that, or you can write an explicit loop that creates another list:
source = [1, 2, 3]
target = []
for n in source:
n1 = <many lines of code involving n>
target.append(n1)
Your question has multiple parts, so it's going to be hard for one answer to cover all of them. glglgl has done a great job on most of it, but your final question is still unexplained:
Maybe my question should be how to loop over a list of integers and change them without map() as i need a if statement in there
"I need an if statement in there" doesn't mean you can't use map.
First, if you want the if to select which values you want to keep, map has a good friend named filter that does exactly that. For example, to keep only the odd numbers, but add one to each of them, you could do this:
>>> a = [1, 2, 3, 4, 5]
>>> b = []
>>> for x in a:
... if x%2:
... b.append(x+1)
Or just this:
>>> b = map(lambda x: x+1, filter(lambda x: x%2, a))
If, on the other hand, you want the if to control the expression itself—e.g., to add 1 to the odd numbers but leave the even ones alone, you can use an if expression the same way you'd use an if statement:
>>> for x in a:
... if x%2:
... b.append(x+1)
... else:
... b.append(x)
>>> b = map(lambda x: x+1 if x%2 else x, a)
Second, comprehensions are basically equivalent to map and filter, but with expressions instead of functions. If your expression would just be "call this function", then use map or filter. If your function would just be a lambda to "evaluate this expression", then use a comprehension. The above two examples get more readable this way:
>>> b = [x+1 for x in a if x%2]
>>> b = [x+1 if x%2 else x for x in a]
You can do something like this: li = [x+1 for x in li]
I would like to ask what the following does in Python.
It was taken from http://danieljlewis.org/files/2010/06/Jenks.pdf
I have entered comments telling what I think is happening there.
# Seems to be a function that returns a float vector
# dataList seems to be a vector of flat.
# numClass seems to an int
def getJenksBreaks( dataList, numClass ):
# dataList seems to be a vector of float. "Sort" seems to sort it ascendingly
dataList.sort()
# create a 1-dimensional vector
mat1 = []
# "in range" seems to be something like "for i = 0 to len(dataList)+1)
for i in range(0,len(dataList)+1):
# create a 1-dimensional-vector?
temp = []
for j in range(0,numClass+1):
# append a zero to the vector?
temp.append(0)
# append the vector to a vector??
mat1.append(temp)
(...)
I am a little confused because in the pdf there are no explicit variable declarations. However I think and hope I could guess the variables.
Yes, the method append() adds elements to the end of the list. I think your interpretation of the code is correct.
But note the following:
x =[1,2,3,4]
x.append(5)
print(x)
[1, 2, 3, 4, 5]
while
x.append([6,7])
print(x)
[1, 2, 3, 4, 5, [6, 7]]
If you want something like
[1, 2, 3, 4, 5, 6, 7]
you may use extend()
x.extend([6,7])
print(x)
[1, 2, 3, 4, 5, 6, 7]
Python doesn't have explicit variable declarations. It's dynamically typed, variables are whatever type they get assigned to.
Your assessment of the code is pretty much correct.
One detail: The range function goes up to, but does not include, the last element. So the +1 in the second argument to range causes the last iterated value to be len(dataList) and numClass, respectively. This looks suspicious, because the range is zero-indexed, which means it will perform a total of len(dataList) + 1 iterations (which seems suspicious).
Presumably dataList.sort() modifies the original value of dataList, which is the traditional behavior of the .sort() method.
It is indeed appending the new vector to the initial one, if you look at the full source code there are several blocks that continue to concatenate more vectors to mat1.
append is a list function used to append a value at the end of the list
mat1 and temp together are creating a 2D array (eg = [[], [], []]) or matrix of (m x n)
where m = len(dataList)+1 and n = numClass
the resultant matrix is a zero martix as all its value is 0.
In Python, variables are implicitely declared. When you type this:
i = 1
i is set to a value of 1, which happens to be an integer. So we will talk of i as being an integer, although i is only a reference to an integer value. The consequence of that is that you don't need type declarations as in C++ or Java.
Your understanding is mostly correct, as for the comments. [] refers to a list. You can think of it as a linked-list (although its actual implementation is closer to std::vectors for instance).
As Python variables are only references to objects in general, lists are effectively lists of references, and can potentially hold any kind of values. This is valid Python:
# A vector of numbers
vect = [1.0, 2.0, 3.0, 4.0]
But this is perfectly valid code as well:
# The list of my objects:
list = [1, [2,"a"], True, 'foo', object()]
This list contains an integer, another list, a boolean... In Python, you usually rely on duck typing for your variable types, so this is not a problem.
Finally, one of the methods of list is sort, which sorts it in-place, as you correctly guessed, and the range function generates a range of numbers.
The syntax for x in L: ... iterates over the content of L (assuming it is iterable) and sets the variable x to each of the successive values in that context. For example:
>>> for x in ['a', 'b', 'c']:
... print x
a
b
c
Since range generates a range of numbers, this is effectively the idiomatic way to generate a for i = 0; i < N; i += 1 type of loop:
>>> for i in range(4): # range(4) == [0,1,2,3]
... print i
0
1
2
3
I'm having a problem with Python 2.7 that is driving me insane.
I'm passing an array to some functions and altough that variable is suposed to be local, in the end the value of the variable inside main is changed.
I'm a bit new to Python, but this goes against any common sense I got.
Any ideas of what I'm doing wrong?
def mutate(chromo):
# chooses random genes and mutates them randomly to 0 or 1
for gene in chromo:
for codon in gene:
for base in range(2):
codon[randint(0, len(codon)-1)] = randint(0, 1)
return chromo
def mate(chromo1, chromo2):
return mutate([choice(pair) for pair in zip(chromo1, chromo2)])
if __name__ == '__main__':
# top 3 is a multidimensional array with 3 levels (in here I put just 2 for simplicity)
top3 = [[1, 0], [0, 0], [1, 1]]
offspring = []
for item in top3:
offspring.append(mate(top3[0], item))
# after this, top3 is diferent from before the for cycle
UPDATE
Because Python passes by reference, I must make a real copy fo the arrays before using them, so the mate functions must be changed to:
import copy
def mate(chromo1, chromo2):
return mutate([choice(pair) for pair in zip(copy.deepcopy(chromo1), copy.deepcopy(chromo2))])
The problem you are having is stemming from the fact that arrays and dictionaries in python are passed by reference. This means that instead of a fresh copy being created by the def and used locally you are getting a pointer to your array in memory...
x = [1,2,3,4]
def mystery(someArray):
someArray.append(4)
print someArray
mystery(x)
[1, 2, 3, 4, 4]
print x
[1, 2, 3, 4, 4]
You manipulate chromo, which you pass by reference. Therefore the changes are destructive... the return is therefore kind of moot as well (codon is in gene and gene is in chromo). You'll need to make a (deep) copy of your chromos, I think.
try changing
offspring.append(mate(top3[0], item)) to
offspring.append(mate(top3[0][:], item[:]))
or use the list() function