What is the best way to add values to a List in terms of processing time, memory usage and just generally what is the best programming option.
list = []
for i in anotherArray:
list.append(i)
or
list = range(len(anotherArray))
for i in list:
list[i] = anotherArray[i]
Considering that anotherArray is for example an array of Tuples. (This is just a simple example)
It really depends on your use case. There is no generic answer here as it depends on what you are trying to do.
In your example, it looks like you are just trying to create a copy of the array, in which case the best way to do this would be to use copy:
from copy import copy
list = copy(anotherArray)
If you are trying to transform the array into another array you should use list comprehension.
list = [i[0] for i in anotherArray] # get the first item from tuples in anotherArray
If you are trying to use both indexes and objects, you should use enumerate:
for i, j in enumerate(list)
which is much better than your second example.
You can also use generators, lambas, maps, filters, etc. The reason all of these possibilities exist is because they are all "better" for different reasons. The writters of python are pretty big on "one right way", so trust me, if there was one generic way which was always better, that is the only way that would exist in python.
Edit: Ran some results of performance for tuple swap and here are the results:
comprehension: 2.682028295999771
enumerate: 5.359116118001111
for in append: 4.177091988000029
for in indexes: 4.612594166001145
As you can tell, comprehension is usually the best bet. Using enumerate is expensive.
Here is the code for the above test:
from timeit import timeit
some_array = [(i, 'a', True) for i in range(0,100000)]
def use_comprehension():
return [(b, a, i) for i, a, b in some_array]
def use_enumerate():
lst = []
for j, k in enumerate(some_array):
i, a, b = k
lst.append((b, a, i))
return lst
def use_for_in_with_append():
lst = []
for i in some_array:
i, a, b = i
lst.append((b, a, i))
return lst
def use_for_in_with_indexes():
lst = [None] * len(some_array)
for j in range(len(some_array)):
i, a, b = some_array[j]
lst[j] = (b, a, i)
return lst
print('comprehension:', timeit(use_comprehension, number=200))
print('enumerate:', timeit(use_enumerate, number=200))
print('for in append:', timeit(use_for_in_with_append, number=200))
print('for in indexes:', timeit(use_for_in_with_indexes, number=200))
Edit2:
It was pointed out to me the the OP just wanted to know the difference between "indexing" and "appending". Really, those are used for two different use cases as well. Indexing is for replacing objects, whereas appending is for adding. However, in a case where the list starts empty, appending will always be better because the indexing has the overhead of creating the list initially. You can see from the results above that indexing is slightly slower, mostly because you have to create the first list.
Best way is list comprehension :
my_list=[i for i in anotherArray]
But based on your problem you can use a generator expression (is more efficient than list comprehension when you just want to loop over your items and you don't need to use some list methods like indexing or len or ... )
my_list=(i for i in anotherArray)
I would actually say the best is a combination of index loops and value loops with enumeration:
for i, j in enumerate(list): # i is the index, j is the value, can't go wrong
Related
I want delete some elements of one list equal to a value:
I can do it :
List =[1,2,3.....]
List = [x for x in List if x != 2]
How can i save the indexs of the deleted elements ?
I want to use this index to delete elements of another list.
Simplest solution is to make a list of indices to keep, then use that to strip the elements from both of your lists. itertools provides a handy compress utility to apply the indices to keep quickly:
from itertools import compress
tokeep = [x != 2 for x in List]
List = list(compress(List, tokeep))
otherlist = list(compress(otherlist, tokeep))
Alternatively (and frankly more clearly) you can just use one loop to strip both inputs; listcomps are fun, but sometimes they're not the way to go.
newlist = []
newotherlist = []
for x, y in zip(List, otherlist):
if x != 2:
newlist.append(x)
newotherlist.append(y)
which gets the same effect in a single pass. Even if it does feel less overtly clever, it's very clear, which is a good thing; brevity for the sake of brevity that creates complexity is not a win.
And now, to contradict that last paragraph, the amusingly overtly clever and brief solution to one-line this:
List, otherlist = map(list, zip(*[(x, y) for x, y in zip(List, otherlist) if x != 2]))
For the love of sanity, please don't actually use this, I just had to write it for funsies.
You can also leverage enumerate
for index, val in enumerate(List):
if val == value:
del List[index]
break
print(index)
Based on documentation
list_first = ['d', 'a']
list_second = ['x', 'z']
def remove_from_lists(element):
index_deleted = list_first.index(element)
list_first.remove(element)
list_second.pop(index_deleted)
remove_from_lists('d')
Given a list of integers, I want to check a second list and remove from the first only those which can not be made from the sum of two numbers from the second. So given a = [3,19,20] and b = [1,2,17], I'd want [3,19].
Seems like a a cinch with two nested loops - except that I've gotten stuck with break and continue commands.
Here's what I have:
def myFunction(list_a, list_b):
for i in list_a:
for a in list_b:
for b in list_b:
if a + b == i:
break
else:
continue
break
else:
continue
list_a.remove(i)
return list_a
I know what I need to do, just the syntax seems unnecessarily confusing. Can someone show me an easier way? TIA!
You can do like this,
In [13]: from itertools import combinations
In [15]: [item for item in a if item in [sum(i) for i in combinations(b,2)]]
Out[15]: [3, 19]
combinations will give all possible combinations in b and get the list of sum. And just check the value is present in a
Edit
If you don't want to use the itertools wrote a function for it. Like this,
def comb(s):
for i, v1 in enumerate(s):
for j in range(i+1, len(s)):
yield [v1, s[j]]
result = [item for item in a if item in [sum(i) for i in comb(b)]]
Comments on code:
It's very dangerous to delete elements from a list while iterating over it. Perhaps you could append items you want to keep to a new list, and return that.
Your current algorithm is O(nm^2), where n is the size of list_a, and m is the size of list_b. This is pretty inefficient, but a good start to the problem.
Thee's also a lot of unnecessary continue and break statements, which can lead to complicated code that is hard to debug.
You also put everything into one function. If you split up each task into different functions, such as dedicating one function to finding pairs, and one for checking each item in list_a against list_b. This is a way of splitting problems into smaller problems, and using them to solve the bigger problem.
Overall I think your function is doing too much, and the logic could be condensed into much simpler code by breaking down the problem.
Another approach:
Since I found this task interesting, I decided to try it myself. My outlined approach is illustrated below.
1. You can first check if a list has a pair of a given sum in O(n) time using hashing:
def check_pairs(lst, sums):
lookup = set()
for x in lst:
current = sums - x
if current in lookup:
return True
lookup.add(x)
return False
2. Then you could use this function to check if any any pair in list_b is equal to the sum of numbers iterated in list_a:
def remove_first_sum(list_a, list_b):
new_list_a = []
for x in list_a:
check = check_pairs(list_b, x)
if check:
new_list_a.append(x)
return new_list_a
Which keeps numbers in list_a that contribute to a sum of two numbers in list_b.
3. The above can also be written with a list comprehension:
def remove_first_sum(list_a, list_b):
return [x for x in list_a if check_pairs(list_b, x)]
Both of which works as follows:
>>> remove_first_sum([3,19,20], [1,2,17])
[3, 19]
>>> remove_first_sum([3,19,20,18], [1,2,17])
[3, 19, 18]
>>> remove_first_sum([1,2,5,6],[2,3,4])
[5, 6]
Note: Overall the algorithm above is O(n) time complexity, which doesn't require anything too complicated. However, this also leads to O(n) extra auxiliary space, because a set is kept to record what items have been seen.
You can do it by first creating all possible sum combinations, then filtering out elements which don't belong to that combination list
Define the input lists
>>> a = [3,19,20]
>>> b = [1,2,17]
Next we will define all possible combinations of sum of two elements
>>> y = [i+j for k,j in enumerate(b) for i in b[k+1:]]
Next we will apply a function to every element of list a and check if it is present in above calculated list. map function can be use with an if/else clause. map will yield None in case of else clause is successful. To cater for this we can filter the list to remove None values
>>> list(filter(None, map(lambda x: x if x in y else None,a)))
The above operation will output:
>>> [3,19]
You can also write a one-line by combining all these lines into one, but I don't recommend this.
you can try something like that:
a = [3,19,20]
b= [1,2,17,5]
n_m_s=[]
data=[n_m_s.append(i+j) for i in b for j in b if i+j in a]
print(set(n_m_s))
print("after remove")
final_data=[]
for j,i in enumerate(a):
if i not in n_m_s:
final_data.append(i)
print(final_data)
output:
{19, 3}
after remove
[20]
I've got a list in which some items shall be moved into a separate list (by a comparator function). Those elements are pure dicts. The question is how should I iterate over such list.
When iterating the simplest way, for element in mylist, then I don't know the index of the element. There's no .iteritems() methods for lists, which could be useful here. So I've tried to use for index in range(len(mylist)):, which [1] seems over-complicated as for python and [2] does not satisfy me, since range(len()) is calculated once in the beginning and if I remove an element from the list during iteration, I'll get IndexError: list index out of range.
Finally, my question is - how should I iterate over a python list, to be able to remove elements from the list (using a comparator function and put them in another list)?
You can use enumerate function and make a temporary copy of the list:
for i, value in enumerate(old_list[:]):
# i == index
# value == dictionary
# you can safely remove from old_list because we are iterating over copy
Creating a new list really isn't much of a problem compared to removing items from the old one. Similarly, iterating twice is a very minor performance hit, probably swamped by other factors. Unless you have a very good reason to do otherwise, backed by profiling your code, I'd recommend iterating twice and building two new lists:
from itertools import ifilter, ifilterfalse
l1 = list(ifilter(condition, l))
l2 = list(ifilterfalse(condition, l))
You can slice-assign the contents of one of the new lists into the original if you want:
l[:] = l1
If you're absolutely sure you want a 1-pass solution, and you're absolutely sure you want to modify the original list in place instead of creating a copy, the following avoids quadratic performance hits from popping from the middle of a list:
j = 0
l2 = []
for i in range(len(l)):
if condition(l[i]):
l[j] = l[i]
j += 1
else:
l2.append(l[i])
del l[j:]
We move each element of the list directly to its final position without wasting time shifting elements that don't really need to be shifted. We could use for item in l if we wanted, and it'd probably be a bit faster, but when the algorithm involves modifying the thing we're iterating over, I prefer the explicit index.
I prefer not to touch the original list and do as #Martol1ni, but one way to do it in place and not be affected by the removal of elements would be to iterate backwards:
for i in reversed(range(len()):
# do the filtering...
That will affect only the indices of elements that you have tested/removed already
Try the filter command, and you can override the original list with it too if you don't need it.
def cmp(i): #Comparator function returning a boolean for a given item
...
# mylist is the initial list
mylist = filter(cmp, mylist)
mylist is now a generator of suitable items. You can use list(mylist) if you need to use it more than once.
Haven't tried this yet but.. i'll give it a quick shot:
new_list = [old.pop(i) for i, x in reversed(list(enumerate(old))) if comparator(x)]
You can do this, might be one line too much though.
new_list1 = [x for x in old_list if your_comparator(x)]
new_list2 = [x for x in old_list if x not in new_list1]
I want to remove the list items found in list B from the list A. This is the function I wrote:
def remove(A,B):
to_remove=[];
for i in range(len(A)):
for j in range(len(B)):
if (B[j]==A[i]):
to_remove.append(i);
for j in range(len(to_remove)):
A.pop(to_remove[j]);
Is this the normal way to do it ? Although, this works completely fine (if typos, I don't know), I think there might be more pythonic way to do it. Please suggest.
Convert B to a set first and then create a new array from A using a list comprehension:
s = set(B)
A = [item for item in A if item not in s]
Item lookup in a set is an O(1) operation.
If you don't want to change the id() of A, then:
A[:] = [item for item in A if item not in s]
First, note that your function doesn't work right. Try this:
A = [1, 2, 3]
B = [1, 2, 3]
remove(A, B)
You'll get an IndexError, because the correct indices to delete change each time you do a .pop().
You'll doubtless get answers recommending using sets, and that's indeed much better if the array elements are hashable and comparable, but in general you may need something like this:
def remove(A, B):
A[:] = [avalue for avalue in A if avalue not in B]
That works for any kinds of array elements (provided only they can be compared for equality), and preserves the original ordering. But it takes worst-case time proportional to len(A) * len(B).
List comprehenstion to the rescue:
[item for item in A if item not in B]
This however creates a new list. You can return the list from the function.
Or, if you are ok with loosing any duplicates in list A, or there are no duplicates, you can use set difference:
return list(set(A) - set(B))
One caveat is, this won't preserve the order of elements in A. So, if you want elements in order, this is not what you want. Use the 1st approach instead.
What about list comprehension?
def remove(removeList, fromList):
return [x for x in fromList if x not in removeList]
Also, to make life easier and removing faster you can make a set from list removeList, leaving only unique elements:
def remove(removeList, fromList):
removeSet = set(removeList)
return [x for x in fromList if x not in removeSet]
>>> print remove([1,2,3], [1,2,3,4,5,6,7])
[4, 5, 6, 7]
And, of course, you can use built-in filter function, though someone will say that it's non-pythonic, and you should use list generators instead. Either way, here is an example:
def remove(removeList, fromList):
removeSet = set(removeList)
return filter(lambda x : x not in removeSet, fromList)
are the both approaches equivalent from a point of view of performance and coding-style?
def foo(n):
return n*2
# case 1 with ".append(function())"
mylist = [1,2,3,4,5,6,7,8,9,10]
result = list()
for l in mylist:
result.append(foo(l))
# case 2 ".append(result)"
mylist = [1,2,3,4,5,6,7,8,9,10]
result = list()
for l in mylist:
r = foo(l)
result.append(r)
Your approaches are more or less equivalent; option 2 does a little more work; it has to do an extra store and name lookup for the temporary variable. In practice, the difference is minimal and you should be focusing on code readability more than premature optimization.
For your specific example you'd be much better off using a list comprehension:
result = [foo(l) for l in mylist]
The list comprehension constructs the list in C code and does not have to look up the .append() method each loop iteration.
I think they are the same, but best option would be map(foo, mylist) - even from performance point of view as append can cause list resize which takes extra time
You might have some performance gain in just removing the tempoarary assignment, but remember, premature optimization is evil, and micro-optimization is not worth the effort
Off course there are better ways to do, for example
List Comprehension
mylist = [1,2,3,4,5,6,7,8,9,10]
result = [n*2 for n in mylist]
result = [foo(i) for i in mylist]
is what you want.