nums = [1,2,3,4,5]
it = iter(nums)
print(next(it))
print(next(it))
for i in nums:
print(i)
here the result is:
1
2
1
2
3
4
5
So my question is that when we apply iter method on a object then does it create a copy of object on which it runs next method?
iter(object) returns an iterator object which is an iterator version of the object given to it given that it implements __iter__. iter(object) doesn't create a copy of the object.
>>> l=[[1,2],[4,5]]
>>> it=iter(l)
>>>next(it).append(3) #appending to the output of next() mutates the list l
>>> l
[[1,2,3],[4,5]]
>>> next(it).append(6)
>>> l
[[1,2,3],[4,5,6]]
>>> it=iter(l)
>>> l.pop() #Mutating the list l mutated iterator it.
[4,5,6]
>>>list(it)
[[1,2,3]]
Here is one way to figure it out:
lst = ['Hi', 'I am a copy!']
itr = iter(lst)
print(next(itr))
lst[1] = 'I am _not_ a copy!'
print(next(itr))
(iter(lst) does not create a copy of lst)
No, they don't. Some Python types, e.g. all its collections, just support being iterated over multiple times. Multiple iterator objects can hold references to the very same list, they all just maintain their own position within the list.
Notice some effects:
lst = [1,2,3,4,5]
it = iter(lst)
lst.pop() # modify the original list
list(it) # the iterator is affected
# [1,2,3,4]
Even more obvious is the case of exhaustable iterators and calling iter on them:
it1 = iter(range(10))
it2 = iter(it1)
next(it)
# 0
next(it2)
# 1
next(it)
# 2
next(it2)
# 3
Clearly the iterators share state.
The = operator assigns values from right side operands to left side operands" i.e. c = a + b assigns value of a + b into c Operators
You're not altering any variables present in the right side of an assignment line, a copy of the value is having a function applied to it and then that result is being assigned the new variable name it.
Related
first line of code:
for i in list:
print(i)
second line of code:
print(i for i in list)
what would I use each of them for?
You can see for yourself what the difference is.
The first one iterates over range and then prints integers.
>>> for i in range(4):
... print(i)
...
0
1
2
3
The second one is a generator expression.
>>> print(i for i in range(4))
<generator object <genexpr> at 0x10b6c20f0>
How iteration works in the generator. Python generators are a simple way of creating iterators.
Simply speaking, a generator is a function that returns an object (iterator) which we can iterate over (one value at a time).
>>> g=(i for i in range(4))
>>> print(g)
<generator object <genexpr> at 0x100f015d0>
>>> print(next(g))
0
>>>
>>> print(next(g))
1
>>> print(next(g))
2
>>> print(next(g))
3
>>> print(next(g))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> g=(i for i in range(4))
>>> for i in g:
... print(i)
...
0
1
2
3
>>> for i in g:
... print(i)
...
>>>
>>>
In python3, you can use tuple unpacking to print the generator. If that's what you were going for.
>>> print(*(i for i in range(4)))
0 1 2 3
The first code snippet will iterate over your list and print the value of i for each pass through the loop. In most cases you will want to use something like this to print the values in a list:
my_list = list(range(5))
for i in my_list:
print(i)
0
1
2
3
4
The second snippet will evaluate the expression in the print statement and print the result. Since the expression in print statement, i for i in my_list evaluates to a generator expression, that string representation of that generator expression will be outputted. I cannot think of any real world cases where that is the result you would want.
my_list = list(range(5))
print(i for i in my_list)
<generator object <genexpr> at 0x0E9EB2F0>
The first way is just a loop going through a list and printing the elements one by one:
l = [1, 2, 3]
for i in l:
print(i)
output:
1
2
3
The second way, list comprehension, creates an iterable you can store (list, dictionary, etc.)
l = [i for i in l]
print( l ) #[1, 2, 3]
print( l[0] ) #1
print( l[1:] ) #[2, 3]
output:
[1, 2, 3]
1
[2, 3]
The second is used for doing 1 thing to all the elements e.g. turn all elements from string to int:
l = ['1', '2', '3']
l = [int(i) for i in l] #now the list is [1, 2, 3]
loops are better for doing a lot of things:
for i in range(4):
#Code
#more code
#lots of more code
pass
In response to the (since edited) answers that suggested otherwise:
the second one is NOT a list comprehension.
the two code snippets do NOT do the same thing.
>>> x = [1,2,3,4,5]
>>> print(i for i in x)
<generator object <genexpr> at 0x000002322FA1BA50>
The second one is printing a generator object because (i for i in x) is a generator. The first snippet simply prints the elements in the list one at a time.
BTW: don't use list as a variable name. It's the name of a built-in type in Python, so when you use it as a variable name, you overwrite the constructor for that type. Basically, you're erasing the built-in list() function.
The second one is a generator expression. Both will give same result if you convert the generator expression to list comprehension. In short, list comprehensions are used to increase both memory and execution efficiency. However, they are generally applicable to small blocks of codes, generally one to two lines.
For more information, see this link on official python website - https://docs.python.org/3/tutorial/datastructures.html?highlight=list%20comprehensions
This question already has answers here:
When is "i += x" different from "i = i + x" in Python?
(3 answers)
Closed 4 years ago.
One block of code works but the other does not. Which would make sense except the second block is the same as the first only with an operation written in shorthand. They are practically the same operation.
l = ['table']
i = []
Version 1
for n in l:
i += n
print(i)
Output: ['t', 'a', 'b', 'l', 'e']
Version 2
for n in l:
i = i + n
print(i)
Output:
TypeError: can only concatenate list (not "str") to list
What is causing this strange error?
They don't have to be the same.
Using the + operator calls the method __add__ while using the += operator calls __iadd__. It is completely up to the object in question what happens when one of these methods is called.
If you use x += y but x does not provide an __iadd__ method (or the method returns NotImplemented), __add__ is used as a fallback, meaning that x = x + y happens.
In the case of lists, using l += iterable actually extends the list l with the elements of iterable. In your case, every character from the string (which is an iterable) is appended during the extend operation.
Demo 1: using __iadd__
>>> l = []
>>> l += 'table'
>>> l
['t', 'a', 'b', 'l', 'e']
Demo 2: using extend does the same
>>> l = []
>>> l.extend('table')
>>> l
['t', 'a', 'b', 'l', 'e']
Demo 3: adding a list and a string raises a TypeError.
>>> l = []
>>> l = l + 'table'
[...]
TypeError: can only concatenate list (not "str") to list
Not using += gives you the TypeError here because only __iadd__ implements the extending behavior.
Demo 4: common pitfall: += does not build a new list. We can confirm this by checking for equal object identities with the is operator.
>>> l = []
>>> l_ref = l # another name for l, no data is copied here
>>> l += [1, 2, 3] # uses __iadd__, mutates l in-place
>>> l is l_ref # confirm that l and l_ref are names for the same object
True
>>> l
[1, 2, 3]
>>> l_ref # mutations are seen across all names
[1, 2, 3]
However, the l = l + iterable syntax does build a new list.
>>> l = []
>>> l_ref = l # another name for l, no data is copied here
>>> l = l + [1, 2, 3] # uses __add__, builds new list and reassigns name l
>>> l is l_ref # confirm that l and l_ref are names for different objects
False
>>> l
[1, 2, 3]
>>> l_ref
[]
In some cases, this can produce subtle bugs, because += mutates the original list, while
l = l + iterable builds a new list and reassigns the name l.
BONUS
Ned Batchelder's challenge to find this in the docs
No.
7.2.1. Augmented assignment statements:
An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the
actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.
If in the second case, you wrap a list around n to avoid errors:
for n in l:
i = i + [n]
print(i)
you get
['table']
So they are different operations.
In python objects such as lists are passed by reference. Assignment with the = operator assigns by reference. So this function:
def modify_list(A):
A = [1,2,3,4]
Takes a reference to list and labels it A, but then sets the local variable A to a new reference; the list passed by the calling scope is not modified.
test = []
modify_list(test)
print(test)
prints []
However I could do this:
def modify_list(A):
A += [1,2,3,4]
test = []
modify_list(test)
print(test)
Prints [1,2,3,4]
How can I assign a list passed by reference to contain the values of another list? What I am looking for is something functionally equivelant to the following, but simpler:
def modify_list(A):
list_values = [1,2,3,4]
for i in range(min(len(A), len(list_values))):
A[i] = list_values[i]
for i in range(len(list_values), len(A)):
del A[i]
for i in range(len(A), len(list_values)):
A += [list_values[i]]
And yes, I know that this is not a good way to do <whatever I want to do>, I am just asking out of curiosity not necessity.
You can do a slice assignment:
>>> def mod_list(A, new_A):
... A[:]=new_A
...
>>> liA=[1,2,3]
>>> new=[3,4,5,6,7]
>>> mod_list(liA, new)
>>> liA
[3, 4, 5, 6, 7]
The simplest solution is to use:
def modify_list(A):
A[::] = [1, 2, 3, 4]
To overwrite the contents of a list with another list (or an arbitrary iterable), you can use the slice-assignment syntax:
A = B = [1,2,3]
A[:] = [4,5,6,7]
print(A) # [4,5,6,7]
print(A is B) # True
Slice assignment is implemented on most of the mutable built-in types. The above assignment is essentially the same the following:
A.__setitem__(slice(None, None, None), [4,5,6,7])
So the same magic function (__setitem__) is called when a regular item assignment happens, only that the item index is now a slice object, which represents the item range to be overwritten. Based on this example you can even support slice assignment in your own types.
Code:
num_of_iterations_for_outer_loop = 2
num_of_iterations_for_inner_loop = 3
data = []
for j in range(num_of_iterations_for_outer_loop):
lst2 = []
for k in range(num_of_iterations_for_inner_loop):
lst1 = {}
lst1['1'] = "first_value" # these are not default, in my original code they are coming from some other values after calculation
lst1['2'] = "second_value"
lst1['3'] = "third_value"
lst2.append(lst1)
value = {'values':lst2}
data.append(value)
In the outer loop I have to clear lst2 list again and again for reusing it.
In the inner loop I have to clear lst1 dictionary again and again for reusing it.
I know 2 methods of clearing a list:
del lst2[:]
lst2 = []
and 1 method for clearing a dictionary:
lst1 = {}
But I don't know differences between these methods and which one I should use.
Is there any other method to clear the list and dictionary? And is that better method, and why?
Is there any better method to write the whole code?
Here is a brief idea of what each method does:
lst2 = [] does not actually clear the contents of the list. It just creates a new empty list and assign it to the name lst2. Then it is the task of GC to delete the actual contents based on the remaining references.
>>> a = [1,2,3]
>>> b = a
>>> a = []
>>> a
[]
>>> b
[1, 2, 3]
del lst2[:] clears the contents of the list in-place.
>>> a = [1,2,3]
>>> b = a
>>> del a[:]
>>> a
[]
>>> b
[]
Note that the other list b also gets cleared because the contents are deleted in-place.
lst1 = {} is same as point 1. It creates an empty dictionary and assigns the reference to the name lst1.
Another method to clear the list in-place is:
lst2[:] = []
The differences between your two methods of "clearing" the list is that the first operates in-place, deleting the contents of the original list object, while the second simply creates a new list object and assigns it to the same name. You must use the second version here, otherwise all of the lists in your dictionary will be copies of the same emptied list at the end:
>>> l = [1, 2, 3]
>>> d = {'a': l}
>>> del l[:]
>>> d
{'a': []}
Similarly, you can del all the key-value pairs from a dictionary, but you would have the same problem. As you are putting the original container objects into some other container, you should create a new container by assignment (e.g. lst1 = {}) in each loop.
Is there any better method to write the whole code ?
I don't know what else you're doing with the data, but the dictionary value may be unnecessary, and a three-tuple of the innermost data could be sufficient:
data = [[(1, 2, 3), (4, 5, 6)], [(7, 8, 9), ...], ...]
Assignments are pretty much good and efficient methods to clear list or dict.
my_dict = {}
and my_list = []
Nothing fancy needed.
What exactly are views in Python3.1? They seem to behave in a similar manner as that of iterators and they can be materialized into lists too. How are iterators and views different?
From what I can tell, a view is still attached to the object it was created from. Modifications to the original object affect the view.
from the docs (for dictionary views):
>>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, 'spam': 500}
>>> keys = dishes.keys()
>>> values = dishes.values()
>>> # iteration
>>> n = 0
>>> for val in values:
... n += val
>>> print(n)
504
>>> # keys and values are iterated over in the same order
>>> list(keys)
['eggs', 'bacon', 'sausage', 'spam']
>>> list(values)
[2, 1, 1, 500]
>>> # view objects are dynamic and reflect dict changes
>>> del dishes['eggs']
>>> del dishes['sausage']
>>> list(keys)
['spam', 'bacon']
>>> # set operations
>>> keys & {'eggs', 'bacon', 'salad'}
{'bacon'}
I would recommend that you read this. It seems to do the best job of explaining.
As far as I can tell, views seem to be associated more with dicts and can be forced into lists. You can also make an iterator out of them, through which you could then iterate (in a for loop or by calling next)
Update: updated link from wayback machine
How are iterators and views different?
I'll rephrase the question as "what's the difference between an iterable objects and an iterator"?
An iterable is an object that can be iterated over (e.g. used in a for loop).
An iterator is an object that can be called with the next() function, that is it implements the .next() method in Python2 and .__next__() in python3. An iterator is often used to wrap an iterable and return each item of interest. All iterators are iterable, but the reverse is not necessarily true (all iterables are not iterators).
Views are iterable objects, not iterators.
Let's look at some code to see the distinction (Python 3):
The "What's new in Python 3" document is very specific about which functions return iterators. map(), filter(), and zip() definitely return an iterator, whereas dict.items(), dict.values(), dict.keys() are said to return a view object. As for range(), although the description of what it returns exactly lacks precision, we know it's not an iterator.
Using map() to double all numbers in a list
m = map(lambda x: x*2, [0,1,2])
hasattr(m, '__next__')
# True
next(m)
# 0
next(m)
# 2
next(m)
# 4
next(m)
# StopIteration ...
Using filter() to extract all odd numbers
f = filter(lambda x: x%2==1, [0,1,2,3,4,5,6])
hasattr(f, '__next__')
# True
next(f)
# 1
next(f)
# 3
next(f)
# 5
next(f)
# StopIteration ...
Trying to use range() in the same manner to produce a sequence of number
r = range(3)
hasattr(r, '__next__')
# False
next(r)
# TypeError: 'range' object is not an iterator
But it's an iterable, so we should be able to wrap it with an iterator
it = iter(r)
next(it)
# 0
next(it)
# 1
next(it)
# 2
next(it)
# StopIteration ...
dict.items() as well as dict.keys() and dict.values() also do not return iterators in Python 3
d = {'a': 0, 'b': 1, 'c': 2}
items = d.items()
hasattr(items, '__next__')
# False
it = iter(items)
next(it)
# ('b', 1)
next(it)
# ('c', 2)
next(it)
# ('a', 0)
An iterator can only be used in a single for loop, whereas an iterable can be used repeatedly in subsequent for loops. Each time an iterable is used in this context it implicitely returns a new iterator (from its __iter__() method). The following custom class demonstrates this by outputting the memory id of both the list object and the returning iterator object:
class mylist(list):
def __iter__(self, *a, **kw):
print('id of iterable is still:', id(self))
rv = super().__iter__(*a, **kw)
print('id of iterator is now:', id(rv))
return rv
l = mylist('abc')
A for loop can use the iterable object and will implicitly get an iterator
for c in l:
print(c)
# id of iterable is still: 139696242511768
# id of iterator is now: 139696242308880
# a
# b
# c
A subsequent for loop can use the same iterable object, but will get another iterator
for c in l:
print(c)
# id of iterable is still: 139696242511768
# id of iterator is now: 139696242445616
# a
# b
# c
We can also obtain an iterator explicitly
it = iter(l)
# id of iterable is still: 139696242511768
# id of iterator is now: 139696242463688
but it can then only be used once
for c in it:
print(c)
# a
# b
# c
for c in it:
print(c)
for c in it:
print(c)