Is i = i + n truly the same as i += n? [duplicate] - python

This question already has answers here:
When is "i += x" different from "i = i + x" in Python?
(3 answers)
Closed 4 years ago.
One block of code works but the other does not. Which would make sense except the second block is the same as the first only with an operation written in shorthand. They are practically the same operation.
l = ['table']
i = []
Version 1
for n in l:
i += n
print(i)
Output: ['t', 'a', 'b', 'l', 'e']
Version 2
for n in l:
i = i + n
print(i)
Output:
TypeError: can only concatenate list (not "str") to list
What is causing this strange error?

They don't have to be the same.
Using the + operator calls the method __add__ while using the += operator calls __iadd__. It is completely up to the object in question what happens when one of these methods is called.
If you use x += y but x does not provide an __iadd__ method (or the method returns NotImplemented), __add__ is used as a fallback, meaning that x = x + y happens.
In the case of lists, using l += iterable actually extends the list l with the elements of iterable. In your case, every character from the string (which is an iterable) is appended during the extend operation.
Demo 1: using __iadd__
>>> l = []
>>> l += 'table'
>>> l
['t', 'a', 'b', 'l', 'e']
Demo 2: using extend does the same
>>> l = []
>>> l.extend('table')
>>> l
['t', 'a', 'b', 'l', 'e']
Demo 3: adding a list and a string raises a TypeError.
>>> l = []
>>> l = l + 'table'
[...]
TypeError: can only concatenate list (not "str") to list
Not using += gives you the TypeError here because only __iadd__ implements the extending behavior.
Demo 4: common pitfall: += does not build a new list. We can confirm this by checking for equal object identities with the is operator.
>>> l = []
>>> l_ref = l # another name for l, no data is copied here
>>> l += [1, 2, 3] # uses __iadd__, mutates l in-place
>>> l is l_ref # confirm that l and l_ref are names for the same object
True
>>> l
[1, 2, 3]
>>> l_ref # mutations are seen across all names
[1, 2, 3]
However, the l = l + iterable syntax does build a new list.
>>> l = []
>>> l_ref = l # another name for l, no data is copied here
>>> l = l + [1, 2, 3] # uses __add__, builds new list and reassigns name l
>>> l is l_ref # confirm that l and l_ref are names for different objects
False
>>> l
[1, 2, 3]
>>> l_ref
[]
In some cases, this can produce subtle bugs, because += mutates the original list, while
l = l + iterable builds a new list and reassigns the name l.
BONUS
Ned Batchelder's challenge to find this in the docs

No.
7.2.1. Augmented assignment statements:
An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the
actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.

If in the second case, you wrap a list around n to avoid errors:
for n in l:
i = i + [n]
print(i)
you get
['table']
So they are different operations.

Related

is iterable object a copy of original object?

nums = [1,2,3,4,5]
it = iter(nums)
print(next(it))
print(next(it))
for i in nums:
print(i)
here the result is:
1
2
1
2
3
4
5
So my question is that when we apply iter method on a object then does it create a copy of object on which it runs next method?
iter(object) returns an iterator object which is an iterator version of the object given to it given that it implements __iter__. iter(object) doesn't create a copy of the object.
>>> l=[[1,2],[4,5]]
>>> it=iter(l)
>>>next(it).append(3) #appending to the output of next() mutates the list l
>>> l
[[1,2,3],[4,5]]
>>> next(it).append(6)
>>> l
[[1,2,3],[4,5,6]]
>>> it=iter(l)
>>> l.pop() #Mutating the list l mutated iterator it.
[4,5,6]
>>>list(it)
[[1,2,3]]
Here is one way to figure it out:
lst = ['Hi', 'I am a copy!']
itr = iter(lst)
print(next(itr))
lst[1] = 'I am _not_ a copy!'
print(next(itr))
(iter(lst) does not create a copy of lst)
No, they don't. Some Python types, e.g. all its collections, just support being iterated over multiple times. Multiple iterator objects can hold references to the very same list, they all just maintain their own position within the list.
Notice some effects:
lst = [1,2,3,4,5]
it = iter(lst)
lst.pop() # modify the original list
list(it) # the iterator is affected
# [1,2,3,4]
Even more obvious is the case of exhaustable iterators and calling iter on them:
it1 = iter(range(10))
it2 = iter(it1)
next(it)
# 0
next(it2)
# 1
next(it)
# 2
next(it2)
# 3
Clearly the iterators share state.
The = operator assigns values from right side operands to left side operands" i.e. c = a + b assigns value of a + b into c Operators
You're not altering any variables present in the right side of an assignment line, a copy of the value is having a function applied to it and then that result is being assigned the new variable name it.

Understanding *x ,= lst

I'm going through some old code trying to understand what it does, and I came across this odd statement:
*x ,= p
p is a list in this context. I've been trying to figure out what this statement does. As far as I can tell, it just sets x to the value of p. For example:
p = [1,2]
*x ,= p
print(x)
Just gives
[1, 2]
So is this any different than x = p? Any idea what this syntax is doing?
*x ,= p is basically an obfuscated version of x = list(p) using extended iterable unpacking. The comma after x is required to make the assignment target a tuple (it could also be a list though).
*x, = p is different from x = p because the former creates a copy of p (i.e. a new list) while the latter creates a reference to the original list. To illustrate:
>>> p = [1, 2]
>>> *x, = p
>>> x == p
True
>>> x is p
False
>>> x = p
>>> x == p
True
>>> x is p
True
It's a feature that was introduced in Python 3.0 (PEP 3132). In Python 2, you could do something like this:
>>> p = [1, 2, 3]
>>> q, r, s = p
>>> q
1
>>> r
2
>>> s
3
Python 3 extended this so that one variable could hold multiple values:
>>> p = [1, 2, 3]
>>> q, *r = p
>>> q
1
>>> r
[2, 3]
This, therefore, is what is being used here. Instead of two variables to hold three values, however, it is just one variable that takes each value in the list. This is different from x = p because x = p just means that x is another name for p. In this case, however, it is a new list that just happens to have the same values in it. (You may be interested in "Least Astonishment" and the Mutable Default Argument)
Two other common ways of producing this effect are:
>>> x = list(p)
and
>>> x = p[:]
Since Python 3.3, the list object actually has a method intended for copying:
x = p.copy()
The slice is actually a very similar concept. As nneonneo pointed out, however, that works only with objects such as lists and tuples that support slices. The method you mention, however, works with any iterable: dictionaries, sets, generators, etc.
You should always throw these to dis and see what it throws back at you; you'll see how *x, = p is actually different from x = p:
dis('*x, = p')
1 0 LOAD_NAME 0 (p)
2 UNPACK_EX 0
4 STORE_NAME 1 (x)
While, the simple assignment statement:
dis('x = p')
1 0 LOAD_NAME 0 (p)
2 STORE_NAME 1 (x)
(Stripping off unrelated None returns)
As you can see UNPACK_EX is the different op-code between these; it's documented as:
Implements assignment with a starred target: Unpacks an iterable in TOS (top of stack) into individual values, where the total number of values can be smaller than the number of items in the iterable: one of the new values will be a list of all leftover items.
Which is why, as Eugene noted, you get a new object that's referred to by the name x and not a reference to an already existing object (as is the case with x = p).
*x, does seem very odd (the extra comma there and all) but it is required here. The left hand side must either be a tuple or a list and, due to the quirkiness of creating a single element tuple in Python, you need to use a trailing ,:
i = 1, # one element tuple
If you like confusing people, you can always use the list version of this:
[*x] = p
which does exactly the same thing but doesn't have that extra comma hanging around there.
You can clearly understand it from below example
L = [1, 2, 3, 4]
while L:
temp, *L = L
print(temp, L)
what it does is, the front variable will get the first item every time and the remaining list will be given to L.
The output will look shown below.
1 [2, 3, 4]
2 [3, 4]
3 [4]
4 []
Also look at below example
x, *y, z = "python"
print(x,y,z)
In this both x,z will get each one letter from the string meaning first letter is assigned to x and the last letter will be assigned to z and the remaining string will be assigned to variable y.
p ['y', 't', 'h', 'o'] n
One more example,
a, b, *c = [0,1,2,3]
print(a,b,c)
0 1 [2,3]
Boundary case: If there is nothing remaining for star variable then it will get an empty list.
Example:
a,b=[1]
print(a,b)
1 []

Copy values from one list to another without altering the reference in python

In python objects such as lists are passed by reference. Assignment with the = operator assigns by reference. So this function:
def modify_list(A):
A = [1,2,3,4]
Takes a reference to list and labels it A, but then sets the local variable A to a new reference; the list passed by the calling scope is not modified.
test = []
modify_list(test)
print(test)
prints []
However I could do this:
def modify_list(A):
A += [1,2,3,4]
test = []
modify_list(test)
print(test)
Prints [1,2,3,4]
How can I assign a list passed by reference to contain the values of another list? What I am looking for is something functionally equivelant to the following, but simpler:
def modify_list(A):
list_values = [1,2,3,4]
for i in range(min(len(A), len(list_values))):
A[i] = list_values[i]
for i in range(len(list_values), len(A)):
del A[i]
for i in range(len(A), len(list_values)):
A += [list_values[i]]
And yes, I know that this is not a good way to do <whatever I want to do>, I am just asking out of curiosity not necessity.
You can do a slice assignment:
>>> def mod_list(A, new_A):
... A[:]=new_A
...
>>> liA=[1,2,3]
>>> new=[3,4,5,6,7]
>>> mod_list(liA, new)
>>> liA
[3, 4, 5, 6, 7]
The simplest solution is to use:
def modify_list(A):
A[::] = [1, 2, 3, 4]
To overwrite the contents of a list with another list (or an arbitrary iterable), you can use the slice-assignment syntax:
A = B = [1,2,3]
A[:] = [4,5,6,7]
print(A) # [4,5,6,7]
print(A is B) # True
Slice assignment is implemented on most of the mutable built-in types. The above assignment is essentially the same the following:
A.__setitem__(slice(None, None, None), [4,5,6,7])
So the same magic function (__setitem__) is called when a regular item assignment happens, only that the item index is now a slice object, which represents the item range to be overwritten. Based on this example you can even support slice assignment in your own types.

Value Incrementation Confusion

Hey am new to python development and i am fully filled with a lots of doubts since am a newbie.Suppose
s = 'something'
for something in s:
something = something + 1
print something
I know here something act as an index and it would print out the whole elements in s.
And in
s = 'something'
for something in s:
s[something] = s[something] + 1
print something
I didnt understand the correct meaning of the second part of the code..Is it possible in python??..
'
Sorry for low grade question and any help would be appreciated ..
When you loop through a string like this:
for c in 'something':
print(c)
c does not act as an index, it acts as character of the string, so the output would be:
s
o
m
e
t
h
i
n
g
If you want to loop through the indices you can do:
s = 'something'
for i in range(len(s)):
print(i)
And the output would be:
0
1
2
3
4
5
6
7
8
You can access a character from the string by indexing like this:
s = 'something'
for i in range(len(s)):
print(s[i])
And the output of that would be:
s
o
m
e
t
h
i
n
g
If you want to loop through a string so that you get the characters as well as the indices, you can use the enumerate() function:
s = 'something'
for i, c in enumerate(s):
print(i, c)
The output:
0 s
1 o
2 m
3 e
4 t
5 h
6 i
7 n
8 g
Note that strings are immutable, so you can't change them:
>>> s = 'something'
>>> s[0] = 'a'
TypeError: 'str' object does not support item assignment
When you do string concatenation, you are not actually changing the string, you are creating a new one.
EDIT 1
Strings have methods that can be called on them to do certain tasks, such as the .split() method:
>>> s = 'something'
>>> s.split('e')
['som', 'thing']
They also have some special methods like __getitem__. The following two are equivalent:
>>> s = 'something'
>>> s[0]
's'
>>> s.__getitem__(0)
's'
Other sequences like lists are mutable, so they also have a __setitem__ method:
>>> s = ['s', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g']
>>> s[0] = 't'
>>> s
['t', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g']
>>> s.__setitem__(0, 's')
>>> s
['s', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g']
EDIT 2
This is what happens when you try to do this s[something] = s[something] + 1:
>>> s = 'something'
>>> s[0] = s[0] + 1
TypeError: Can't convert 'int' object to str implicitly
The reason this happens is because s[0] is 's' so you are trying to add a number to a string, which doesn't make any sense. Then if you try and do s[something] = s[something] + 'a' you will get a TypeError because strings are immutable:
>>> s = 'something'
>>> s[0] = s[0] + 'a'
TypeError: 'str' object does not support item assignment
And this will definitely not work:
>>> s = 'something'
>>> s['a']
TypeError: string indices must be integers
s[something] = s[something] + 1 shouldn't work; string values are immutable.
Syntax like s += "foo" actually creates a new string value from s + "foo", then assigns it to s, releasing the original value of s to be garbage collected.
A key thing to remember about all variables in Python is that they're just references to values. There's no guarantee the values aren't pooled somewhere and have a copy-on-write semantic. Another example is that a like like x = 5 doesn't set x to 5, it creates (or otherwise obtains) the value 5 and sets x to refer to it.
For the most part this distinction really doesn't matter. In general, the Right Thing(TM) happens.
The code:
s = 'something'
for something in s:
# ...
treats s like a list of characters and sets something to each one in sequence through the loop. (This is unlike JavaScript.) If you want the indices and not just the characters, use:
s = 'something'
for i, something in enumerate(s):
# ...
so s[something] = s[something] + 1 is not possible in any situations ..right ?
It works fine for lists (e.g. [1, 2, 3]) and dictionaries (e.g. {"a": 1, "b": 2}). Just not for strings.
If you simply want to get a string where every character is replaced with the next one, first split the string with a list comprehension:
l = [c for c in s]
Replace each character with the next one:
l2 = [chr(ord(c) + 1) for c in l]
and glue them back together into a new string:
s2 = ''.join(l2)
Putting it all together:
s = 'something'
s2 = ''.join([chr(ord(c) + 1) for c in s])
The square brackets after a variable name invoke __getitem__ or __setitem__ on the variable, depending on the context. So for example, x[i] = x[i] + 1 is equivalent to x.__setitem__(i, x.__getitem__(i) + 1). You can read up about this in the docs here:
https://docs.python.org/2/reference/datamodel.html
There are several built-in types that implement one or both of these, for example strings, tuples, lists, and dictionaries. For the sequence types (strings, tuples, lists) the "item" being accessed or set is an index, so for example print 'hello'[0] would print h because you are getting the character at the first index in the list.
In this case, it looks like the second piece of code would actually cause an error because strings are not mutable. This means that string objects can't be modified, so they won't have __setitem__ implemented and s[something] = s[something] + 1 would fail. This could work with a mutable type like list or dict though, for example:
s = [1, 1, 1]
s[0] = s[0] + 1
# s is now [2, 1, 1]

Why doesn't list have safe "get" method like dictionary?

Why doesn't list have a safe "get" method like dictionary?
>>> d = {'a':'b'}
>>> d['a']
'b'
>>> d['c']
KeyError: 'c'
>>> d.get('c', 'fail')
'fail'
>>> l = [1]
>>> l[10]
IndexError: list index out of range
Ultimately it probably doesn't have a safe .get method because a dict is an associative collection (values are associated with names) where it is inefficient to check if a key is present (and return its value) without throwing an exception, while it is super trivial to avoid exceptions accessing list elements (as the len method is very fast). The .get method allows you to query the value associated with a name, not directly access the 37th item in the dictionary (which would be more like what you're asking of your list).
Of course, you can easily implement this yourself:
def safe_list_get (l, idx, default):
try:
return l[idx]
except IndexError:
return default
You could even monkeypatch it onto the __builtins__.list constructor in __main__, but that would be a less pervasive change since most code doesn't use it. If you just wanted to use this with lists created by your own code you could simply subclass list and add the get method.
This works if you want the first element, like my_list.get(0)
>>> my_list = [1,2,3]
>>> next(iter(my_list), 'fail')
1
>>> my_list = []
>>> next(iter(my_list), 'fail')
'fail'
I know it's not exactly what you asked for but it might help others.
Probably because it just didn't make much sense for list semantics. However, you can easily create your own by subclassing.
class safelist(list):
def get(self, index, default=None):
try:
return self.__getitem__(index)
except IndexError:
return default
def _test():
l = safelist(range(10))
print l.get(20, "oops")
if __name__ == "__main__":
_test()
Instead of using .get, using like this should be ok for lists. Just a usage difference.
>>> l = [1]
>>> l[10] if 10 < len(l) else 'fail'
'fail'
Credits to jose.angel.jimenez and Gus Bus.
For the "oneliner" fans…
If you want the first element of a list or if you want a default value if the list is empty try:
liste = ['a', 'b', 'c']
value = (liste[0:1] or ('default',))[0]
print(value)
returns a
and
liste = []
value = (liste[0:1] or ('default',))[0]
print(value)
returns default
Examples for other elements…
liste = ['a', 'b', 'c']
print(liste[0:1]) # returns ['a']
print(liste[1:2]) # returns ['b']
print(liste[2:3]) # returns ['c']
print(liste[3:4]) # returns []
With default fallback…
liste = ['a', 'b', 'c']
print((liste[0:1] or ('default',))[0]) # returns a
print((liste[1:2] or ('default',))[0]) # returns b
print((liste[2:3] or ('default',))[0]) # returns c
print((liste[3:4] or ('default',))[0]) # returns default
Possibly shorter:
liste = ['a', 'b', 'c']
value, = liste[:1] or ('default',)
print(value) # returns a
It looks like you need the comma before the equal sign, the equal sign and the latter parenthesis.
More general:
liste = ['a', 'b', 'c']
f = lambda l, x, d: l[x:x+1] and l[x] or d
print(f(liste, 0, 'default')) # returns a
print(f(liste, 1, 'default')) # returns b
print(f(liste, 2, 'default')) # returns c
print(f(liste, 3, 'default')) # returns default
Tested with Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13)
Try this:
>>> i = 3
>>> a = [1, 2, 3, 4]
>>> next(iter(a[i:]), 'fail')
4
>>> next(iter(a[i + 1:]), 'fail')
'fail'
A reasonable thing you can do is to convert the list into a dict and then access it with the get method:
>>> my_list = ['a', 'b', 'c', 'd', 'e']
>>> my_dict = dict(enumerate(my_list))
>>> print my_dict
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'}
>>> my_dict.get(2)
'c'
>>> my_dict.get(10, 'N/A')
So I did some more research into this and it turns out there isn't anything specific for this. I got excited when I found list.index(value), it returns the index of a specified item, but there isn't anything for getting the value at a specific index. So if you don't want to use the safe_list_get solution which I think is pretty good. Here are some 1 liner if statements that can get the job done for you depending on the scenario:
>>> x = [1, 2, 3]
>>> el = x[4] if len(x) > 4 else 'No'
>>> el
'No'
You can also use None instead of 'No', which makes more sense.:
>>> x = [1, 2, 3]
>>> i = 2
>>> el_i = x[i] if len(x) == i+1 else None
Also if you want to just get the first or last item in the list, this works
end_el = x[-1] if x else None
You can also make these into functions but I still liked the IndexError exception solution. I experimented with a dummied down version of the safe_list_get solution and made it a bit simpler (no default):
def list_get(l, i):
try:
return l[i]
except IndexError:
return None
Haven't benchmarked to see what is fastest.
Dictionaries are for look ups. It makes sense to ask if an entry exists or not. Lists are usually iterated. It isn't common to ask if L[10] exists but rather if the length of L is 11.
If you
want a one liner,
prefer not having try / except in your happy code path where you needn't, and
want the default value to be optional,
you can use this:
list_get = lambda l, x, d=None: d if not l[x:x+1] else l[x]
Usage looks like:
>>> list_get(['foo'], 4) == None
True
>>> list_get(['hootenanny'], 4, 'ho down!')
'ho down!'
>>> list_get([''], 0)
''
For small index values you can implement
my_list.get(index, default)
as
(my_list + [default] * (index + 1))[index]
If you know in advance what index is then this can be simplified, for example if you knew it was 1 then you could do
(my_list + [default, default])[index]
Because lists are forward packed the only fail case we need to worry about is running off the end of the list. This approach pads the end of the list with enough defaults to guarantee that index is covered.
This isn't an extremely general-purpose solution, but I had a case where I expected a list of length 3 to 5 (with a guarding if), and I was breaking out the values to named variables. A simple and concise way I found for this involved:
foo = (argv + [None, None])[3]
bar = (argv + [None, None])[4]
Now foo and bar are either the 4th and 5th values in the list, or None if there weren't that many values.
Your usecase is basically only relevant for when doing arrays and matrixes of a fixed length, so that you know how long they are before hand. In that case you typically also create them before hand filling them up with None or 0, so that in fact any index you will use already exists.
You could say this: I need .get() on dictionaries quite often. After ten years as a full time programmer I don't think I have ever needed it on a list. :)

Categories