I'm not sure if this has been asked before as it seems to be a simple question, I tried to search it but did not find a very good answer.
Say I have a list of numbers a. I know I can get out every fifth number by a[::5].
What should be the simplest way to get the rest of the list (items in a but not in a[::5])?
Edit:
this is not the same question as Finding elements not in a list as I already know that the rest of the list consists of a[1::5]+a[2::5]+a[3::5]+a[4::5], I'm just asking is there a simpler way / syntax sugar for that?
Edit:
Thank you all for answering.
Is there a solution without scanning over the whole list?
[x for i, x in enumerate(a) if i % 5 != 0]
not in set(a[::5]) will not work correctly when list contains equal elements. not in a[::5] is also not optimal from performance point of view.
Edit There is no syntactic sugar for your problem in Python. Take a look at numpy: it has extensive slicing possibilities.
Edit To avoid list scanning, you could create wrapper class with __getitem__ method.
I would start by something like:
[v for v in a if v not in a[::5]]
Store the unwanted value in a set before could lead to better perf, because not in method is in O(1) for sets:
unwanted = set(a[::5])
[v for v in a if v not in unwanted]
If the amount of data is huge, bloom filter could be a good substitute.
Does this help you?
items = [1,2,3,4]
Z = [3,4,5,6]
a = list(set(items)-set(Z))
# a =[1, 2]
Such as:
fifths = a[::5]
not_fifths = list(set(a)-set(fifths))
temp = [x for x in a if x not in a[::5]]
You can try this:
[a[i-1] for i in range(1, len(a) + 1) if i % 5]
Basically, it checks for every position to see if it is a multiple of 5 or not. You don't have to create a set object this way.
This would do it, without any index look up overheads.
result = [n for i,n in enumerate(a) if i % 5 != 0]
Related
I'm currently developing a program in python and I just noticed that something was wrong with the foreach loop in the language, or maybe the list structure. I'll just give a generic example of my problem to simplify, since I get the same erroneous behavior on both my program and my generic example:
x = [1,2,2,2,2]
for i in x:
x.remove(i)
print x
Well, the problem here is simple, I though that this code was supposed to remove all elements from a list. Well, the problem is that after it's execution, I always get 2 remaining elements in the list.
What am I doing wrong? Thanks for all the help in advance.
Edit: I don't want to empty a list, this is just an example...
This is a well-documented behaviour in Python, that you aren't supposed to modify the list being iterated through. Try this instead:
for i in x[:]:
x.remove(i)
The [:] returns a "slice" of x, which happens to contain all its elements, and is thus effectively a copy of x.
When you delete an element, and the for-loop incs to the next index, you then skip an element.
Do it backwards. Or please state your real problem.
I think, broadly speaking, that when you write:
for x in lst:
# loop body goes here
under the hood, python is doing something like this:
i = 0
while i < len(lst):
x = lst[i]
# loop body goes here
i += 1
If you insert lst.remove(x) for the loop body, perhaps then you'll be able to see why you get the result you do?
Essentially, python uses a moving pointer to traverse the list. The pointer starts by pointing at the first element. Then you remove the first element, thus making the second element the new first element. Then the pointer move to the new second – previously third – element. And so on. (it might be clearer if you use [1,2,3,4,5] instead of [1,2,2,2,2] as your sample list)
Why don't you just use:
x = []
It's probably because you're changing the same array that you're iterating over.
Try Chris-Jester Young's answer if you want to clear the array your way.
I know this is an old post with an accepted answer but for those that may still come along...
A few previous answers have indicated it's a bad idea to change an iterable during iteration. But as a way to highlight what is happening...
>>> x=[1,2,3,4,5]
>>> for i in x:
... print i, x.index(i)
... x.remove(i)
... print x
...
1 0
[2, 3, 4, 5]
3 1
[2, 4, 5]
5 2
[2, 4]
Hopefully the visual helps clarify.
I agree with John Fouhy regarding the break condition. Traversing a copy of the list works for the remove() method, as Chris Jester-Young suggested. But if one needs to pop() specific items, then iterating in reverse works, as Erik mentioned, in which case the operation can be done in place. For example:
def r_enumerate(iterable):
"""enumerator for reverse iteration of an iterable"""
enum = enumerate(reversed(iterable))
last = len(iterable)-1
return ((last - i, x) for i,x in enum)
x = [1,2,3,4,5]
y = []
for i,v in r_enumerate(x):
if v != 3:
y.append(x.pop(i))
print 'i=%d, v=%d, x=%s, y=%s' %(i,v,x,y)
or with xrange:
x = [1,2,3,4,5]
y = []
for i in xrange(len(x)-1,-1,-1):
if x[i] != 3:
y.append(x.pop(i))
print 'i=%d, x=%s, y=%s' %(i,x,y)
If you need to filter stuff out of a list it may be a better idea to use list comprehension:
newlist = [x for x in oldlist if x%2]
for instance would filter all even numbers out of an integer list
The list stored in the memory of a computer. This deals with the pointer to a memory artifact. When you remove an element, in a by-element loop, you are then moving the pointer to the next available element in the memory address
You are modifying the memory and iterating thru the same.
The pointer to the element moves through the list to the next spot available.
So in the case of the Size being 5...enter code here
[**0**,1,2,3,4]
remove 0 ---> [1,**2**,3,4] pointer moves to second index.
remove 2 ---> [1,3,**4**] pointer moves to 3rd index.
remove 4 ---> [1,3]
I was just explaining this to my students when they used pop(1). Another very interesting side-effect error.
x=[1,**2**,3,4,5]
for i in x:
x.pop(1)
print(x,i)
[1, **3**, 4, 5] 1 at index 0 it removed the index 1 (2)
[1, **4**, 5] 3 at index 1 it removed the index 1 (3)
[1, 5] 5 at index 2 it removed the index 1 (4)
heh.
They were like why isnt this working... I mean... it did... exactly what you told it to do. Not a mind reader. :)
Given a list of integers, I want to check a second list and remove from the first only those which can not be made from the sum of two numbers from the second. So given a = [3,19,20] and b = [1,2,17], I'd want [3,19].
Seems like a a cinch with two nested loops - except that I've gotten stuck with break and continue commands.
Here's what I have:
def myFunction(list_a, list_b):
for i in list_a:
for a in list_b:
for b in list_b:
if a + b == i:
break
else:
continue
break
else:
continue
list_a.remove(i)
return list_a
I know what I need to do, just the syntax seems unnecessarily confusing. Can someone show me an easier way? TIA!
You can do like this,
In [13]: from itertools import combinations
In [15]: [item for item in a if item in [sum(i) for i in combinations(b,2)]]
Out[15]: [3, 19]
combinations will give all possible combinations in b and get the list of sum. And just check the value is present in a
Edit
If you don't want to use the itertools wrote a function for it. Like this,
def comb(s):
for i, v1 in enumerate(s):
for j in range(i+1, len(s)):
yield [v1, s[j]]
result = [item for item in a if item in [sum(i) for i in comb(b)]]
Comments on code:
It's very dangerous to delete elements from a list while iterating over it. Perhaps you could append items you want to keep to a new list, and return that.
Your current algorithm is O(nm^2), where n is the size of list_a, and m is the size of list_b. This is pretty inefficient, but a good start to the problem.
Thee's also a lot of unnecessary continue and break statements, which can lead to complicated code that is hard to debug.
You also put everything into one function. If you split up each task into different functions, such as dedicating one function to finding pairs, and one for checking each item in list_a against list_b. This is a way of splitting problems into smaller problems, and using them to solve the bigger problem.
Overall I think your function is doing too much, and the logic could be condensed into much simpler code by breaking down the problem.
Another approach:
Since I found this task interesting, I decided to try it myself. My outlined approach is illustrated below.
1. You can first check if a list has a pair of a given sum in O(n) time using hashing:
def check_pairs(lst, sums):
lookup = set()
for x in lst:
current = sums - x
if current in lookup:
return True
lookup.add(x)
return False
2. Then you could use this function to check if any any pair in list_b is equal to the sum of numbers iterated in list_a:
def remove_first_sum(list_a, list_b):
new_list_a = []
for x in list_a:
check = check_pairs(list_b, x)
if check:
new_list_a.append(x)
return new_list_a
Which keeps numbers in list_a that contribute to a sum of two numbers in list_b.
3. The above can also be written with a list comprehension:
def remove_first_sum(list_a, list_b):
return [x for x in list_a if check_pairs(list_b, x)]
Both of which works as follows:
>>> remove_first_sum([3,19,20], [1,2,17])
[3, 19]
>>> remove_first_sum([3,19,20,18], [1,2,17])
[3, 19, 18]
>>> remove_first_sum([1,2,5,6],[2,3,4])
[5, 6]
Note: Overall the algorithm above is O(n) time complexity, which doesn't require anything too complicated. However, this also leads to O(n) extra auxiliary space, because a set is kept to record what items have been seen.
You can do it by first creating all possible sum combinations, then filtering out elements which don't belong to that combination list
Define the input lists
>>> a = [3,19,20]
>>> b = [1,2,17]
Next we will define all possible combinations of sum of two elements
>>> y = [i+j for k,j in enumerate(b) for i in b[k+1:]]
Next we will apply a function to every element of list a and check if it is present in above calculated list. map function can be use with an if/else clause. map will yield None in case of else clause is successful. To cater for this we can filter the list to remove None values
>>> list(filter(None, map(lambda x: x if x in y else None,a)))
The above operation will output:
>>> [3,19]
You can also write a one-line by combining all these lines into one, but I don't recommend this.
you can try something like that:
a = [3,19,20]
b= [1,2,17,5]
n_m_s=[]
data=[n_m_s.append(i+j) for i in b for j in b if i+j in a]
print(set(n_m_s))
print("after remove")
final_data=[]
for j,i in enumerate(a):
if i not in n_m_s:
final_data.append(i)
print(final_data)
output:
{19, 3}
after remove
[20]
What is the best way to add values to a List in terms of processing time, memory usage and just generally what is the best programming option.
list = []
for i in anotherArray:
list.append(i)
or
list = range(len(anotherArray))
for i in list:
list[i] = anotherArray[i]
Considering that anotherArray is for example an array of Tuples. (This is just a simple example)
It really depends on your use case. There is no generic answer here as it depends on what you are trying to do.
In your example, it looks like you are just trying to create a copy of the array, in which case the best way to do this would be to use copy:
from copy import copy
list = copy(anotherArray)
If you are trying to transform the array into another array you should use list comprehension.
list = [i[0] for i in anotherArray] # get the first item from tuples in anotherArray
If you are trying to use both indexes and objects, you should use enumerate:
for i, j in enumerate(list)
which is much better than your second example.
You can also use generators, lambas, maps, filters, etc. The reason all of these possibilities exist is because they are all "better" for different reasons. The writters of python are pretty big on "one right way", so trust me, if there was one generic way which was always better, that is the only way that would exist in python.
Edit: Ran some results of performance for tuple swap and here are the results:
comprehension: 2.682028295999771
enumerate: 5.359116118001111
for in append: 4.177091988000029
for in indexes: 4.612594166001145
As you can tell, comprehension is usually the best bet. Using enumerate is expensive.
Here is the code for the above test:
from timeit import timeit
some_array = [(i, 'a', True) for i in range(0,100000)]
def use_comprehension():
return [(b, a, i) for i, a, b in some_array]
def use_enumerate():
lst = []
for j, k in enumerate(some_array):
i, a, b = k
lst.append((b, a, i))
return lst
def use_for_in_with_append():
lst = []
for i in some_array:
i, a, b = i
lst.append((b, a, i))
return lst
def use_for_in_with_indexes():
lst = [None] * len(some_array)
for j in range(len(some_array)):
i, a, b = some_array[j]
lst[j] = (b, a, i)
return lst
print('comprehension:', timeit(use_comprehension, number=200))
print('enumerate:', timeit(use_enumerate, number=200))
print('for in append:', timeit(use_for_in_with_append, number=200))
print('for in indexes:', timeit(use_for_in_with_indexes, number=200))
Edit2:
It was pointed out to me the the OP just wanted to know the difference between "indexing" and "appending". Really, those are used for two different use cases as well. Indexing is for replacing objects, whereas appending is for adding. However, in a case where the list starts empty, appending will always be better because the indexing has the overhead of creating the list initially. You can see from the results above that indexing is slightly slower, mostly because you have to create the first list.
Best way is list comprehension :
my_list=[i for i in anotherArray]
But based on your problem you can use a generator expression (is more efficient than list comprehension when you just want to loop over your items and you don't need to use some list methods like indexing or len or ... )
my_list=(i for i in anotherArray)
I would actually say the best is a combination of index loops and value loops with enumeration:
for i, j in enumerate(list): # i is the index, j is the value, can't go wrong
I have a list which has the following structure:
a = [[1,'a'], [2,'b'], [3,'c']]
I would like to create a range of the first element in every sub-list without making a second for-loop. I was thinking about something like this:
for i in a[][0]:
print i
However, the above last code does not work (SyntaxError: invalid syntax). Any idea if it's possible to do this in Python?
EDIT:
The output I would like to get with the above loop is:
1
2
3
and not
1
a
for sublist in a:
print sublist[0]
To build a list of first items, use list comprehension:
first_items = [sublist[0] for sublist in a]
for i,_ in a:
print i
should do the trick
This is probably overkill for such a simple example, but for variety:
for i in map(operator.itemgetter(0), a):
print i
In Python 2 map builds the whole list of first elements. You could use itertools.imap to avoid that. Of course for 3 elements it doesn't matter anyway.
I mention this because it's more flexible than for i, _ in a (there don't need to be exactly two elements in each sublist) and it gives you the i you want instead of doing for i in a and using i[0] (perhaps multiple times in a less simple example). But of course you could just as easily get the i you want with:
for l in a:
i = l[0]
print i
... not everything needs to be done in the loop header, it's just nice that it can be :-)
>>> a = [[1,'a'], [2,'b'], [3,'c']]
>>> for i in a:
... print i[0]
...
1
2
3
I think this method is kind of close to what you were trying.
>>> a = [[1,'a'], [2,'b'], [3,'c']]
>>> for [x,y] in a:
... print(x)
...
1
2
3
However,if your lists are of unequal size, then #warwaruk's answer is better.
I have the following list:
a = [100, 34, 2, 100]
I want the index of the largest value:
$ a.index(max(a))
Returns: 0
What is the Pythonic way of getting the last index (with the largest value) in this case 3.
(Since the largest value is repeated multiple times.)
I think this might work for you.
len(a) - a[::-1].index(max(a)) - 1
a[::-1] is the Pythonic way to reverse the list, and then the index function will find the "first" time that max(a) occurs in the reversed list, which will be the last time it occurs in the original list.
Another way to do it could be:
def last_max_index2(s):
m_index = m = None
for i, elem in enumerate(s):
if elem >= m:
m, m_index = elem, i
return m_index
last_max_index2 has the advantage that it is calculating the max as it goes, and hence only needs one pass over the array. But it's more of an algorithm you would see written in C++ or Java, and less in Python. Often this is correct: the shorter ways that rely on built-ins are better.
However, I think this is a much more readable and intuitive approach than any solution using reduce or a single-liner with enumerate and keys with lambdas. Those solutions will only be readily understandable by very familiar Python programmers.
And a solution like that in the comments is very obfuscated:
last_max_index3 = lambda s: max((x, i) for i, x in enumerate(s))[1]
I know most folks familiar with Python would disagree, but I think this code is so misdirectional to a Python beginner that it's actually harmful to do it this way, regardless of one-liner-ness, use of enumerate, and small number of characters typed.
Seems as simple as:
max(enumerate(a), key=lambda x: (x[1], x[0]))[0]
max() takes a key parameter. More info in the docs provided.
Not sure that it is very pythonic:
l = [100, 34, 2, 100]
>>> reduce(lambda m, p: p if p[1]>=m[1] else m, enumerate(l))
(3, 100)
FWIW, here's another option. I dare say more pythonic than the accepted answer
max(reversed(xrange(len(a))), key=a.__getitem__)