What does for row_number, row in enumerate(cursor): do in Python?
What does enumerate mean in this context?
The enumerate() function adds a counter to an iterable.
So for each element in cursor, a tuple is produced with (counter, element); the for loop binds that to row_number and row, respectively.
Demo:
>>> elements = ('foo', 'bar', 'baz')
>>> for elem in elements:
... print elem
...
foo
bar
baz
>>> for count, elem in enumerate(elements):
... print count, elem
...
0 foo
1 bar
2 baz
By default, enumerate() starts counting at 0 but if you give it a second integer argument, it'll start from that number instead:
>>> for count, elem in enumerate(elements, 42):
... print count, elem
...
42 foo
43 bar
44 baz
If you were to re-implement enumerate() in Python, here are two ways of achieving that; one using itertools.count() to do the counting, the other manually counting in a generator function:
from itertools import count
def enumerate(it, start=0):
# return an iterator that adds a counter to each element of it
return zip(count(start), it)
and
def enumerate(it, start=0):
count = start
for elem in it:
yield (count, elem)
count += 1
The actual implementation in C is closer to the latter, with optimisations to reuse a single tuple object for the common for i, ... unpacking case and using a standard C integer value for the counter until the counter becomes too large to avoid using a Python integer object (which is unbounded).
It's a builtin function that returns an object that can be iterated over. See the documentation.
In short, it loops over the elements of an iterable (like a list), as well as an index number, combined in a tuple:
for item in enumerate(["a", "b", "c"]):
print item
prints
(0, "a")
(1, "b")
(2, "c")
It's helpful if you want to loop over a sequence (or other iterable thing), and also want to have an index counter available. If you want the counter to start from some other value (usually 1), you can give that as second argument to enumerate.
I am reading a book (Effective Python) by Brett Slatkin and he shows another way to iterate over a list and also know the index of the current item in the list but he suggests that it is better not to use it and to use enumerate instead.
I know you asked what enumerate means, but when I understood the following, I also understood how enumerate makes iterating over a list while knowing the index of the current item easier (and more readable).
list_of_letters = ['a', 'b', 'c']
for i in range(len(list_of_letters)):
letter = list_of_letters[i]
print (i, letter)
The output is:
0 a
1 b
2 c
I also used to do something, even sillier before I read about the enumerate function.
i = 0
for n in list_of_letters:
print (i, n)
i += 1
It produces the same output.
But with enumerate I just have to write:
list_of_letters = ['a', 'b', 'c']
for i, letter in enumerate(list_of_letters):
print (i, letter)
As other users have mentioned, enumerate is a generator that adds an incremental index next to each item of an iterable.
So if you have a list say l = ["test_1", "test_2", "test_3"], the list(enumerate(l)) will give you something like this: [(0, 'test_1'), (1, 'test_2'), (2, 'test_3')].
Now, when this is useful? A possible use case is when you want to iterate over items, and you want to skip a specific item that you only know its index in the list but not its value (because its value is not known at the time).
for index, value in enumerate(joint_values):
if index == 3:
continue
# Do something with the other `value`
So your code reads better because you could also do a regular for loop with range but then to access the items you need to index them (i.e., joint_values[i]).
Although another user mentioned an implementation of enumerate using zip, I think a more pure (but slightly more complex) way without using itertools is the following:
def enumerate(l, start=0):
return zip(range(start, len(l) + start), l)
Example:
l = ["test_1", "test_2", "test_3"]
enumerate(l)
enumerate(l, 10)
Output:
[(0, 'test_1'), (1, 'test_2'), (2, 'test_3')]
[(10, 'test_1'), (11, 'test_2'), (12, 'test_3')]
As mentioned in the comments, this approach with range will not work with arbitrary iterables as the original enumerate function does.
The enumerate function works as follows:
doc = """I like movie. But I don't like the cast. The story is very nice"""
doc1 = doc.split('.')
for i in enumerate(doc1):
print(i)
The output is
(0, 'I like movie')
(1, " But I don't like the cast")
(2, ' The story is very nice')
I am assuming that you know how to iterate over elements in some list:
for el in my_list:
# do something
Now sometimes not only you need to iterate over the elements, but also you need the index for each iteration. One way to do it is:
i = 0
for el in my_list:
# do somethings, and use value of "i" somehow
i += 1
However, a nicer way is to user the function "enumerate". What enumerate does is that it receives a list, and it returns a list-like object (an iterable that you can iterate over) but each element of this new list itself contains 2 elements: the index and the value from that original input list:
So if you have
arr = ['a', 'b', 'c']
Then the command
enumerate(arr)
returns something like:
[(0,'a'), (1,'b'), (2,'c')]
Now If you iterate over a list (or an iterable) where each element itself has 2 sub-elements, you can capture both of those sub-elements in the for loop like below:
for index, value in enumerate(arr):
print(index,value)
which would print out the sub-elements of the output of enumerate.
And in general you can basically "unpack" multiple items from list into multiple variables like below:
idx,value = (2,'c')
print(idx)
print(value)
which would print
2
c
This is the kind of assignment happening in each iteration of that loop with enumerate(arr) as iterable.
the enumerate function calculates an elements index and the elements value at the same time. i believe the following code will help explain what is going on.
for i,item in enumerate(initial_config):
print(f'index{i} value{item}')
Related
I am trying to get a deeper understanding to how for loops for different data types in Python. The simplest way of using a for loop an iterating over an array is as
for i in range(len(array)):
do_something(array[i])
I also know that I can
for i in array:
do_something(i)
What I would like to know is what this does
for i, j in range(len(array)):
# What is i and j here?
or
for i, j in array:
# What is i and j in this case?
And what happens if I try using this same idea with dictionaries or tuples?
The simplest and best way is the second one, not the first one!
for i in array:
do_something(i)
Never do this, it's needlessly complicating the code:
for i in range(len(array)):
do_something(array[i])
If you need the index in the array for some reason (usually you don't), then do this instead:
for i, element in enumerate(array):
print("working with index", i)
do_something(element)
This is just an error, you will get TypeError: 'int' object is not iterable when trying to unpack one integer into two names:
for i, j in range(len(array)):
# What is i and j here?
This one might work, assumes the array is "two-dimensional":
for i, j in array:
# What is i and j in this case?
An example of a two-dimensional array would be a list of pairs:
>>> for i, j in [(0, 1), ('a', 'b')]:
... print('i:', i, 'j:', j)
...
i: 0 j: 1
i: a j: b
Note: ['these', 'structures'] are called lists in Python, not arrays.
Your third loop will not work as it will throw a TypeError for an int not being iterable. This is because you are trying to "unpack" the int that is the array's index into i, and j which is not possible. An example of unpacking is like so:
tup = (1,2)
a,b = tup
where you assign a to be the first value in the tuple and b to be the second. This is also useful when you may have a function return a tuple of values and you want to unpack them immediately when calling the function. Like,
train_X, train_Y, validate_X, validate_Y = make_data(data)
More common loop cases that I believe you are referring to is how to iterate over an arrays items and it's index.
for i, e in enumerate(array):
...
and
for k,v in d.items():
...
when iterating over the items in a dictionary. Furthermore, if you have two lists, l1 and l2 you can iterate over both of the contents like so
for e1, e2 in zip(l1,l2):
...
Note that this will truncate the longer list in the case of unequal lengths while iterating. Or say that you have a lists of lists where the outer lists are of length m and the inner of length n and you would rather iterate over the elements in the inner lits grouped together by index. This is effectively iterating over the transpose of the matrix, you can use zip to perform this operation as well.
for inner_joined in zip(*matrix): # will run m times
# len(inner_joined) == m
...
Python's for loop is an iterator-based loop (that's why bruno desthuilliers says that it "works for all iterables (lists, tuples, sets, dicts, iterators, generators etc)". A string is also another common type of iterable).
Let's say you have a list of tuples. Using that nomenclature you shared, one can iterate through both the keys and values simultaneously. For instance:
tuple_list = [(1, "Countries, Cities and Villages"),(2,"Animals"),(3, "Objects")]
for k, v in tuple_list:
print(k, v)
will give you the output:
1 Countries, Cities and Villages
2 Animals
3 Objects
If you use a dictionary, you'll also gonna be able to do this. The difference here is the need for .items()
dictionary = {1: "Countries, Cities and Villages", 2: "Animals", 3: "Objects"}
for k, v in dictionary.items():
print(k, v)
The difference between dictionary and dictionary.items() is the following
dictionary: {1: 'Countries, Cities and Villages', 2: 'Animals', 3: 'Objects'}
dictionary.items(): dict_items([(1, 'Countries, Cities and Villages'), (2, 'Animals'), (3, 'Objects')])
Using dictionary.items() we'll get a view object containig the key-value pairs of the dictionary, as tuples in a list. In other words, with dictionary.items() you'll also get a list of tuples. If you don't use it, you'll get
TypeError: cannot unpack non-iterable int object
If you want to get the same output using a simple list, you'll have to use something like enumerate()
list = ["Countries, Cities and Villages","Animals", "Objects"]
for k, v in enumerate(list, 1): # 1 means that I want to start from 1 instead of 0
print(k, v)
If you don't, you'll get
ValueError: too many values to unpack (expected 2)
So, this naturally raises the question... do I need always a list of tuples? No. Using enumerate() we'll get an enumerate object.
Actually, "the simplest way of using a for loop an iterating over an array" (the Python type is named "list" BTW) is the second one, ie
for item in somelist:
do_something_with(item)
which FWIW works for all iterables (lists, tuples, sets, dicts, iterators, generators etc).
The range-based C-style version is considered highly unpythonic, and will only work with lists or list-like iterables.
What I would like to know is what this does
for i, j in range(len(array)):
# What is i and j here?
Well, you could just test it by yourself... But the result is obvious: it will raise a TypeError because unpacking only works on iterables and ints are not iterable.
or
for i, j in array:
# What is i and j in this case?
Depends on what is array and what it yields when iterating over it. If it's a list of 2-tuples or an iterator yielding 2-tuples, i and j will be the elements of the current iteration item, ie:
array = [(letter, ord(letter)) for letter in "abcdef"]
for letter, letter_ord in array:
print("{} : {}".format(letter, letter_ord))
Else, it will most probably raise a TypeError too.
Note that if you want to have both the item and index, the solution is the builtin enumerate(sequence), which yields an (index, item) tuple for each item:
array = list("abcdef")
for index, letter in enumerate(array):
print("{} : {}".format(index, letter)
Understood your question, explaining it using a different example.
1. Multiple Assignments using -> Dictionary
dict1 = {1: "Bitcoin", 2: "Ethereum"}
for key, value in dict1.items():
print(f"Key {key} has value {value}")
print(dict1.items())
Output:
Key 1 has value Bitcoin
Key 2 has value Ethereum
dict_items([(1, 'Bitcoin'), (2, 'Ethereum')])
Explaining dict1.items():
dict1_items() creates values dict_items([(1, 'Bitcoin'), (2, 'Ethereum')])
It comes in pairs (key, value) for each iteration.
2. Multiple Assignments using -> enumerate() Function
coins = ["Bitcoin", "Ethereum", "Cardano"]
prices = [48000, 2585, 2]
for i, coin in enumerate(coins):
price = prices[i]
print(f"${price} for 1 {coin}")
Output:
$48000 for 1 Bitcoin
$2585 for 1 Ethereum
$2 for 1 Cardano
Explaining enumerate(coins):
enumerate(coins) creates values ((0, 'Bitcoin'), (1, 'Ethereum'), (2, 'Cardano'))
It comes in pairs (index, value) for each (one) iteration
3. Multiple Assignments using -> zip() Function
coins = ["Bitcoin", "Ethereum", "Cardano"]
prices = [48000, 2585, 2]
for coin, price in zip(coins, prices):
print(f"${price} for 1 {coin}")
Output:
$48000 for 1 Bitcoin
$2585 for 1 Ethereum
$2 for 1 Cardano
Explaining
zip(coins, prices) create values (('Bitcoin', 48000), ('Ethereum', 2585), ('Cardano', 2))
It comes in pairs (value-list1, value-list2) for each (one) iteration.
I just wanted to add that, even in Python, you can get a for in effect using a classic C style loop by just using a local variable
l = len(mylist) #I often need to use this more than once anyways
for n in range(l-1):
i = mylist[n]
print("iterator:",n, " item:",i)
Can anyone explain what is going on here as I am flummoxed
I have a module-wide list variable with elements that have fields - mylist with 'n' entries, each of field1, field2..fieldx
I want to access them in a procedure, so have (with some trace/debug statements)
print mylist [1].dataFieldCheckType
for lIndex, lField in enumerate(mylist, start = 1):
print lField.dataFieldCheckType
The first print statement gives the value -4 (which is correct), the second gives a different value, 0, over a simple one-statement step.
To my mind, lField is being created as a new element with default values but I do not know, nor understand, why. Why is the second print statement giving a different value from the first?
What am I doing wrong? Or, probably more pertinently, what am I not understanding?
I have asked this in another forum but no-one has come up with a plausible explanation.
In enumerate(), start does not specify the starting index into the iterable. It specifies the starting value of the count. enumerate() iterates over the whole iterable, from the first (index 0) to the last element, regardless of the start parameter.
The first print statement in your loop prints mylist[0].dataFieldCheckType, just as it ought to. You're just hoping it would be mylist[1].dataFieldCheckType.
If you want to take all elements of the list starting at the second (index 1), just slice it:
mylist[1:]
And if you really do need the index, too, combine the slice with enumerate():
enumerate(mylist[1:], start=1)
enumerate yields (index + start, value) tuples for every value of an iterable. The optional start parameter is used as an offset value to compute the first element of the generated tuples:
>>> a = ['hi', 'stack', 'overflow']
>>> for x in enumerate(a, -4):
... x
...
(-4, 'hi') # 0 + (-4)
(-3, 'stack') # 1 + (-4)
(-2, 'overflow') # 2 + (-4)
If you want to skip elements of an iterable, but don't need that particular slice in memory (all you want to do is iteration), use itertools.islice:
>>> from itertools import islice
>>> for x in islice(a, 2, None):
... x
...
'overflow'
Of course, you could combine the two for great justice.
>>> for x in islice(enumerate(a), 0, 2):
... x
...
(0, 'hi')
(1, 'stack')
I am supposed to do the following:
Define a function my_enumerate(items) that behaves in a similar way to the built-in enumerate function. It should return a list of pairs (i, item) where item is the ith item, with 0 origin, of the list items (see the examples below). Check the test cases for how the function should work. Your function must not call Python's in-built enumerate function.
Examples:
Input:
ans = my_enumerate([10, 20, 30])
print(ans)
Output:
[(0, 10), (1, 20), (2, 30)]
What does enumerate do? Try expressing it in English, and it may help you understand how to write the necessary code. If it doesn't then the practice of learning English language descriptions into code will be useful.
One way of describing enumerate is to say it iterates over each item in the list, and for each item in the input list it produces a pair of the item's index in the input list and the item.
So we know we need to iterate over the list:
for item in input_list:
pass
And we need to keep track of the index of the current item.:
index = 0
for item in input_list:
index += 1
Hmm, there's a better way of doing that:
for index in range(len(input_list)):
pass
Now to produce the pairs:
for index in range(len(input_list)):
pair = index, input_list[index]
You also need somewhere to store these pairs:
def my_enumerate(input_list):
output_list = []
for index in range(len(input_list)):
pair = index, input_list[index]
output_list.append(pair)
return output_list
Are there other ways to write code that produces the same output? Yes. Is this the best way to write this function? Not by a long shot. What this exercise should help you with is turning your thoughts into code, as you gain more experience doing that then you can combine multiple steps at a time, and start using more complicated programming concepts.
Use itertools.count and zip:
from itertools import count
def my_enumerate(values):
return list(zip(count(), values))
Ss there a simple way to iterate over an iterable object that allows the specification of an end point, say -1, as well as the start point in enumerate. e.g.
for i, row in enumerate(myiterable, start=2): # will start indexing at 2
So if my object has a length of 10, what is the simplest way to to get it to start iterating at index 2 and stop iterating at index 9?
Alternatively, is there something from itertools that is more suitable for this. I am specifically interested in high performance methods.
In addition, when the start option was introduced in 2.6, is there any reason why a stop option was not?
Cheers
for i, row in enumerate(myiterable[2:], start=2):
if i>= limit: break
...
or
for i,row in itertools.takewhile(lambda (i,val):i < limit,enumerate(myiterable[2:],2)):
to rephrase the other suggestion (note that it will only work if your iterable is a sliceable object)
start,stop = 11,20
my_items = range(100)
for i,row in enumerate(my_items[start:stop],start):
....
I think you've misunderstood the 'start' keyword, it doesn't skip to the nth item in the iterable, it starts counting at n, for example:
for i, c in enumerate(['a', 'b', 'c'], start=5):
print i, c
gives:
5 a
6 b
7 c
For simple iterables like lists and tuples the simplest and fastest method is going to be something like:
obj = range(100)
start = 11
stop = 22
for i, item in enumerate(obj[start:stop], start=start):
pass
If I'm not going to be using the whole length of the iterable item, I find it easier to use something like:
for i in range(2, len(myiterable)):
instead of enumerate. And then index based on i within the loop. It requires less typing than other approaches that use enumerate.
Creating the slice of an iterable object could be expensive. To avoid this, use itertools.islice:
import itertools
for item in itertools.islice('abc', 0, 2):
print(item)
# will print:
1
2
This question already has answers here:
'Return' keyword returns only one element from a loop?
(3 answers)
Closed 8 years ago.
I am trying to print all the elements of a list using a user made function.
y = [1,2,3]
def ash(list1):
for x in range(len(list)):
return list[x],x
what i want to do is return all the values in the list with their indices,but all i get is a single element.I can get the elements to print but not return.
You are returning on the very first iteration. Rather, you need to create a list in your function, and add all the tuples in that list, and finally return that list.
And for getting both index and element, you can use enumerate():
def ash(list1):
new_list = []
for x, elem in enumerate(list1):
new_list.append((elem,x))
return new_list
Or, even better you can simply use List comprehension:
return [(elem, x) for x, elem in enumerate(list1)]
The previous two methods creates the list in memory. If you have a very large list to deal with, then you should probably use generators, using yield keyword:
def ash(list1):
for x, elem in enumerate(list1):
yield elem, x
Some issues with your code:
Don't iterate using range unless necessary. Iterate the list directly, or here, use enumerate
Don't use list as a variable - you'll shadow the built-in of the same name. It's confusing to the reader.
You're returning out of the loop. This is why you only get the first iteration. If you want to return successive values, use yield, which turns your function into a generator:
def ash(l):
for x in range(len(l)):
yield l[x],x
This is really a reimplementation of enumerate:
list(enumerate('abc')) #=> [(0, 'a'), (1, 'b'), (2, 'c')]
If you really want to swap the order of the pairs, you can do:
[b,a for a,b in enumerate('abc')]
Alternative implementation: l='abc';zip(l,xrange(len(l)))
enumerate(list) is what you're looking for. (see doc). Also, calling return will give you only the first value of the list when calling the function, what you want here is probably the yield statement:
def ash(list):
for i,item in enumerate(list):
yield item,i
if __name__ == '__main__':
y = [1,2,3]
ys_with_indices = list(ash(y))
print ys_with_indices
Note that this will return a generator object, which you have to convert to a list by calling list() on it. Alternatively, just use a normal list that you append the individual values to:
def ash(list):
items_with_indices = []
for i,item in enumerate(list):
items_with_indices.append((item,i))
return items_with_indices
if __name__ == '__main__':
y = [1,2,3]
ys_with_indices = ash(y)
print ys_with_indices
def ash(lst):
return [(lst[x],x) for x in range(0,len(lst))]
You will get a list of tuples where the first value of the tuple is an element of the original list and the second the index of the element in the list.
For y = [1,2,3] the result is [(1, 0), (2, 1), (3, 2)]