Edit: I want to clarify that I am asking here about situations when the index is explicitly required. I know that for item in items is better when it is not required. I have a long term habit of thinking of list items as my_list[i] or arr[i]. I find seeing the name of the list and the index next to each other very clear and helpful. So my question is really whether there is any advantage in changing the way I think about index and value to fit the way enumerate works.
As the title says. I have read many answers saying for i in range(len(arr)) is bad, but not heard a good reason as to why this is the case. One argument is that it is "un-Pythonic", but it looks pretty Pythonic superficially, compared to the c-style versions.
Since enumerate has an additional argument which is sometimes not needed, I'm confused as to the advantage, as increased simplicity doesn't seem guaranteed. I would be like to hear more about why I should avoid for i in range(len(arr)), or whether it is in fact just down to personal preference.
Iterating over a list directly is simpler.
for x in some_list:
print(x)
versus
for i in range(len(some_list)):
print(some_list[i])
If you do need the index, you can use enumerate (even if you don't care about the value at that index, though such uses cases are rare):
indices_of_positive = [i for i, x in enumerate(some_list) if x > 0]
versus
indices_of_positive = [i for i in range(len(some_list)) if some_list[i] > 0]
Note that enumerate doesn't require the iterable to support the length protocol.
Related
I'm a sucker for reducing code to its bare minimum and love keeping it short and slim, but occasionally I get into the dilemma of whether I'm doing more harm than good. Below is an example of a situation I frequently encounter and where I start pondering if I am minifying at the expense of speed.
str = "my name is john"
##Alternative 1
for el in str.split(" "):
print(el)
##Alternative 2
splittedStr = str.split(" ")
for el in splittedStr:
print(el)
What is faster? I'd assume it's the second one because we don't split the string after every iteration (not even sure we do that)?
str.split(" ") does the exact same thing in both cases. It creates an anonymous list of the split strings. In the second case you have the minor overhead of assigning it to a variable and then fetching the value of the variable. Its wasted time if you don't need to keep the object for other reasons. But this is a trivial amount of time compared to other object referencing taking place in the same loop. Alternative 2 also leaves the data in memory which is another small performance issue.
The real reason Alternative 1 is better than 2, IMHO, is that it doesn't leave the hint that splittedStr is going to be needed later.
Look my friend, if you want to actually reduce the amount of time in the code in general,loop on a tuple instead of list but assigning the result in a variable then using the variable is not the best approach is you just reserved a memory location just to store the value but sometimes you can do that just for the sake of having a clean code like if you have more than one operation in one line like
min(str.split(mylist)[3:10])
In this case, it is better to have a variable called min_value for example just to make things cleaner.
returning back to the performance issue, you could actually notice the difference in performance if you loop through a list or a tuple like
This is looping through a tuple
for i in (1,2,3):
print(i)
& This is looping through a list
for i in [1,2,3]:
print(i)
you will find that using tuple will be faster !
I have a function:
def fun(l):
for i in l:
if len(i)==10:
l.append('+91 {} {}'.format(i[:5],i[5:]))
l.remove(i)
if len(i)==11:
j=list(''.join(i))
j.remove(i[0])
l.append('+91 {} {}'.format(''.join(j[:5]),''.join(j[5:])))
l.remove(i)
if len(i)==12:
j=list(''.join(i))
j.remove(i[0])
j.remove(i[1])
l.append('+91 {} {}'.format(''.join(j[:5]),''.join(j[5:])))
l.remove(i)
if len(i)==13:
j=list(''.join(i))
j.remove(i[0])
j.remove(i[1])
j.remove(i[2])
l.append('+91 {} {}'.format(''.join(j[:5]),''.join(j[5:])))
l.remove(i)
return l
say l=['9195969878','07895462130','919875641230']
I am getting the output as
['+91 91959 69878','7895462130','+91 98756 41230']
But i have suppose to get the output as:
['+91 91959 69878','+91 78954 62130,'+91 98756 41230']
Actually this function is escaping all that is positioned even no in 'l' list. Kindly suggest
The first problem is that you're mutating the list while iterating over it. In this particular case, this caused the loop to skip some items, as you deleted items that were earlier. In other Python versions it might trigger an error. But you're returning your result, so I don't see why you're mutating the list at all.
Secondly your code does some roundabout things, in particular ''.join(i) which is absolutely redundant (it literally rebuilds the same string), and series of remove() calls which almost certainly don't do what you expect. If you remove the first item from [1,2,3], the list becomes [2,3], and if you follow that by removing the second item (index 1) you end up with [2]. This is the same sort of issue your for loop has with the other remove.
I would also restructure the code a bit to avoid code duplication. I get something like:
def fun(l):
return ['+91 {} {}'.format(i[-10:-5],i[-5:])
for i in l]
This never alters l, makes one single pass, and joins all the different length behaviours by observing that we're using parts at a fixed distance from the end. There is one caveat: other lengths aren't handled separately. I don't know if those occur, or how you actually want them handled (the old code would leave them as is). We can easily enough specify other behaviour:
def fun(l):
return ['+91 {} {}'.format(i[-10:-5],i[-5:]) if 10<=len(i)<=13
else i
for i in l]
This still doesn't reproduce the behaviour that reformatted numbers were appended at the end, but I'm not sure you really wanted that. It made little sense for the loop to process its own output in the first place.
You are modifying the list l as you go - I would suggest to create a new list and add things to this list. Is there a reason you want to mutate in place?
If you are intent on mutating in place, why not just do something like this?
l[index] = '+91 {} {}'.format(i[:5],i[5:])
Also, here is the first google result for "python phone number library": https://github.com/daviddrysdale/python-phonenumbers as it may be of use to you. (Never used it, am not the maintainer.)
Many of us know that, enumerate is being using in a situation you use the for loop and need to know the index. However, it has its downsides. According to my tests with the timeit module, just using enumerate makes the code 2x slower. Adding this a tuple assignment makes it slower up to 3x. These numbers may come as fast enough for any programmer, but people dealing with algorithms know that every bit of code you can optimize, is a huge advantage. Now to my question,
An example of this usage would be, the need of finding indexes of multiple elements in a list. Say that there is two elements we need to find. The first two solutions that occur to me is like so:
x, y = 0, 0
for ind, val in enumerate(lst):
if x and y:
break
if val == "a":
x = ind
elif val == "b":
y = ind
The solution above iterates the list, assign the values, than break if the two is found.
x = lst.index("a")
y = lst.index("b")
This is an other solution, which I didn't want to use because it appeared really naive. It iterates over the same list twice, to find two elements. The first solution, does this in a single iteration. So by complexity terms, even though we make extra assignments in the first solution, it should be faster than the second one in larger lists. But my assumption failed.
Here is the code I tested the performance: https://codeshare.io/XfvGA
The second solution was 2x to 10x faster than the first one, changing with the position of these two elements. There are several possibilities which this would occur.
There is an optimization in index() method that I am unaware of.
Lower level assignments being made in index() method. Possible use of C++ code.
The conditions and extra assignments in the first solution, makes it slower than expected.
Even these reasons fall short of explaining the speed of iterating the list twice over iterating it once. Though languages have much difference in time while running code, iteration process itself is independant from the programming language, if you need to check a million elements, you still have to check a million elements (Could be exampled by map() being not much faster than using a loop to change values).
So though I need you to examine the cases I presented, in order to clarify what is being asked here, question can be put together like this. We know that Python's for loop is actually a while running in background (possibly in C ?). So this means, the index is being stored as it is incremented somewhere in the memory. If there was a way to access it, this would eliminate the cost of calling and unpacking enumerate. My question is:
Is there such a way exists ?, If not, could be made (why, or why not) ?
The sources I used for more information on the subject:
Python speed
Python objects time complexity
Performance tips for Python
I dont think that the enumerate is the problem, to prove this you can do:
x, y = 0, 0
for val in a:
if x and y:
break
if val == "a":
x = val
elif val == "b":
y = val
This doesnt do the same thing you wanted in the first place (you dont get the index) but if you messure it with timeit, you will find that the diffrence is not so significant, meaning that the enumerate is not the source of the problem ( in my case it was 0.185 to 0.155 when running your example, so it is faster but the second solution got 0.055 at my computer )
The reason that lst.index is faster is that it is implemented in C .
You can see it's source code here:
https://svn.python.org/projects/python/trunk/Objects/listobject.c
the index function is called listindex in this file and is defined like
static PyObject *
listindex(PyListObject *self, PyObject *args)
( i couldnt find a way to add a link directly to the function )
You are trying to be un-Pythonic, which isn't going to end terribly well for you. If you really need to have that iterator count information available, there is a well-known and optimized way to do that: enumerate(). If you need to find an item in a list, there is a well-known and optimized way to do that: lst.index(). As DorElias showed above/below, enumerate is not the problem, it's that you're attempting to reinvent the wheel with the rest of your for loop. enumerate is going to be the best-supported (clearest, fastest, etc.) way to maintain an iteration count in every situation where an iteration count is actually the thing you need.
Suppose I have a list called icecream_flavours, and two lists called new_flavours and unavailable. I want to remove the elements in flavours that appear in 'unavailable', and add those in new_flavours to the original one. I wrote the following program:
for i in unavailable:
icecream_flavours.remove(i)
for j in new_flavours:
icecream_flavours.append(j)
the append one is fine, but it keeps showing 'ValueError: list.remove(x): x not in list' for the first part of the program. What's the problem?
thanks
There are two possibilities here.
First, maybe there should never be anything in unavailable that wasn't in icecream_flavours, but, because of some bug elsewhere in your program, that isn't true. In that case, you're going to need to debug where things first go wrong, whether by running under the debugger or by adding print calls all over the code. At any rate, since the problem is most likely in code that you haven't shown us here, we can't help if that's the problem.
Alternatively, maybe it's completely reasonable for something to appear in unavailable even though it's not in icecream_flavours, and in that case you just want to ignore it.
That's easy to do, you just need to write the code that does it. As the docs for list.remove explain, it:
raises ValueError when x is not found in s.
So, if you want to ignore cases when i is not found in icecream_flavours, just use a try/except:
for i in unavailable:
try:
icecream_flavours.remove(i)
except ValueError:
# We already didn't have that one... which is fine
pass
That being said, there are better ways to organize your code.
First, using the right data structure always makes things easier. Assuming you don't want duplicate flavors, and the order of flavors doesn't matter, what you really want here is sets, not lists. And if you had sets, this would be trivial:
icecream_flavours -= unavailable
icecream_flavours |= new_flavours
Even if you can't do that, it's usually simpler to create a new list than to mutate one in-place:
icecream_flavours = [flavour for flavour in icecream_flavours
if flavour not in set(unavailable)]
(Notice that I converted unavailable to a set, so we don't have to brute-force search for each flavor in a list.)
Either one of these changes makes the code shorter, and makes it more efficient. But, more importantly, they both make the code easier to reason about, and eliminate the possibility of bugs like the one you're trying to fix.
To add all the new_flavours that are not unavailable, you can use a list comprehension, then use the += operator to add it to the existing flavors.
icecream_flavours += [i for i in new_flavours if i not in unavailable]
If there are already flavors in the original list you want to remove, you can remove them in the same way
icecream_flavours = [i for i in icecream_flavours if i not in unavailable]
If you first want to remove all the unavailable flavours from icecream_flavours and then add the new flavours, you can use this list comprehension:
icecream_flavours = [i for i in icecream_flavours if i not in unavailable] + new_flavours
Your error is caused because unavailable contains flavours that are not in icecream_flavours.
Unless order is important, you could use set instead of list as they have operations for differences and unions and you don't need to worry about duplicates
If you must use lists, a list comprehension is a better way to filter the list
icecream_flavours = [x for x in icecream_flavours if x not in unavaiable]
You can extend the list of flavours like this
icecream_flavours += new_flavours
assuming there are no duplicates.
I'm a Python newbie. I have a series of objects that need to be inserted at specific indices of a list, but they come out of order, so I can't just append them. How can I grow the list whenever necessary to avoid IndexErrors?
def set(index, item):
if len(nodes) <= index:
# Grow list to index+1
nodes[index] = item
I know you can create a list with an initial capacity via nodes = (index+1) * [None] but what's the usual way to grow it in place? The following doesn't seem efficient:
for _ in xrange(len(nodes), index+1):
nodes.append(None)
In addition, I suppose there's probably a class in the Standard Library that I should be using instead of built-in lists?
This is the best way to of doing it.
>>> lst.extend([None]*additional_size)
oops seems like I misunderstood your question at first. If you are asking how to expand the length of a list so you can insert something at an index larger than the current length of the list, then lst.extend([None]*(new_size - len(lst)) would probably be the way to go, as others have suggested. Of course, if you know in advance what the maximum index you will be needing is, it would make sense to create the list in advance and fill it with Nones.
For reference, I leave the original text: to insert something in the middle of the existing list, the usual way is not to worry about growing the list yourself. List objects come with an insert method that will let you insert an object at any point in the list. So instead of your set function, just use
lst.insert(item, index)
or you could do
lst[index:index] = item
which does the same thing. Python will take care of resizing the list for you.
There is not necessarily any class in the standard library that you should be using instead of list, especially if you need this sort of random-access insertion. However, there are some classes in the collections module which you should be aware of, since they can be useful for other situations (e.g. if you're always appending to one end of the list, and you don't know in advance how many items you need, deque would be appropriate).
Perhaps something like:
lst += [None] * additional_size
(you shouldn't call your list variable list, since it is also the name of the list constructor).