I have a list of lists and my goal is to create the list of iterators corresponding to the slices of these lists.
My original attempt is as follows (this is minimal "working" example without most of the unrelated logic):
my_lists = [1,2,3], [6,7,8]
iterators = []
for my_list in my_lists:
it = (my_list[i] for i in range(1,3))
iterators.append(it)
print(next(iterators[0]))
Unfortunately this does not work as intended as the scope of the iterator is shared and the my_list variable is thus identical in all the iterators. Meaning that the message of the print is 6 and not 1 as desired.
I am quite struggling to come up with some clean solution. All my attempt to get one fails. They either create unnecessary lists, like:
it = iter(my_list[1:3])
and
it = iter([my_list[i] for i in range(1,3)])
are unnecessarily clutterring:
def apply_range(l, start, stop):
return (l[i] for i in range(start, stop))
...
it = apply_range(my_list, 1, 3)
or outright hack-ish:
it = (lambda l: (l[i] for i in range(1,3)))(my_list)
or iterate from the starts of the lists:
from itertools import islice
...
it = islice(my_list, 1, 3)
Please note that the code i need is a bit more complicated than in the first snippet (i.e. start and stop of the range is computed -- i.e. not constant -- and I need more than just first element of first iterator -- in particular I combine them in one iterator with inner logic to pick correct iterator to select next element from).
The issue with your original code is that the generator expressions need to use the my_list variable from the outside scope. Because that variable changes as the loop continues to run, all the iterators end up yielding values from the last list in the input.
You can fix this by encapsulating the creation of the generator in a function. The function will have its own namespace where the specific list to be iterated on will be bound to a local variable (that won't change based on outside code). The function can either be a generator function, or it can be a normal function that returns a generator expression (the two approaches are almost identical).
def make_iter(lst, start, stop):
# this loop could be replaced by `return (lst[i] for i in range(start, stop))`
for i in range(start, stop):
yield lst[i]
my_lists = [1,2,3], [6,7,8]
start = 1
stop = 3
iterators = [make_iter(lst, start, stop) for lst in my_lists]
This is a simple neat code that I think satisfies you desire to make some iterators;
my_lists = [1,2,3], [6,7,8]
iterators = []
for my_list in my_lists:
iterators.append(iter(my_list))
allows you to access each "iterator" and the values by using "next";
next(iterators[1])
>>> 6
next(iterators[1])
>>> 7
Related
Are there any ways range print an array without using list()?
def every_three_nums(start):
lst = []
if start <= 100:
lst += range(start, 101, 3)
# this returns an array but if I print(range(start, 101, 3)), it doesn't
return lst
else:
return lst
This returns an array but if I print(range(start, 101, 3)), it doesn't. Why?
How does this code lst += range(start, 101, 3) turn range into an array of numbers?
If I print(range(start, 101, 3)), it doesn't [return an array]. Why?
Among the changes that Python 3 made from Python 2 was an emphasis on generators rather than lists or tuples. A list takes up memory for every item in the list, while a generator only creates one item at a time, whenever needed. A generator thus almost always uses less memory, a precious resource in a program. Also, since a generator needs to create only one item (at most) immediately after it is created and the other items may never be needed, it can also save a great deal of time compared to a list which needs to create all its items whether they are needed or not.
Thus many things that made a list in Python 2 were changed to create a generator instead. range is one of those items. So range is an object, but the items are created only when asked for in one way or another. So the command print(range(start, 101, 3)) creates the range object then prints the object. Python prints a note that basically says that the range is an object, a special range object, and gives a few more details.
As #ShadowRanger points out in a comment (and I meant to include but forgot), range is more than a generator. It can work as a sequence (you can pull one item out from the middle, not just the beginning) and has other special characteristics. That is why your command to print a range does not just show that it is an object, it shows that it is a range object which differs from other objects. I was speaking in general terms when I wrote of "generators"--there are other objects that are similar.
Are there any ways the range function prints an array without using list()?
If you want to print a range as a list, ask for the items. There are multiple ways:
print(*range(start, 101, 3))
That uses Pythons "splat" operator * to get the items of the range in a form suitable as parameters to a function--in this case, the print function. This method comes the closest to never using a "list" to show the contents of the range. The printout is not an "array", just the items separated by spaces.
print([*range(start, 101, 3)])
This uses splat to create a list, then prints the list.
print(list(range(start, 101, 3))
That uses apparent type-casting to convert the range to a list immediately so the list can be printed. You said you do not want this method, but I show it for completeness.
print(tuple(range(start, 101, 3))
This uses a tuple rather than a list.
print([v for v in range(start, 101, 3)])
That uses a list comprehension to create the list. By adding an if clause in the comprehension you could get just some few desired items from the list. You could use a set or other kind of comprehension if you don't want a list.
lst = []
lst += range(start, 101, 3)
print(list)
For details on this one, see the end of this answer.
lst = []
for v in range(start, 101, 3):
lst.append(v)
print(lst)
You could use a loop to get a list to print. There are many variations on this loop that would work.
How does this code lst += range(start, 101, 3) turn range into an array of numbers?
It doesn't. Using the += command on a list uses whatever is on the right side of the operator as an iterable, not a list. Your code picks the items one-by-one out of the range and appends them one-by-one to the list. The range is never converted to a list, it is just used to extend a list. You can see that by trying to execute lst = lst + range(start, 101, 3)--the range is not converted to a list so an error results.. See this link for a closely related question, though it talks about tuples rather than ranges.
In python 3, range return a immutable sequence type, the advantage is that the range object...
...will always take the same (small) amount of memory, no matter the
size of the range it represents (as it only stores the start, stop and
step values, calculating individual items and subranges as needed)
Range used to return a list in python 2.
The key is what:
lst += range(start, 101, 3)
does when lst is a list which is:
lst.extend(range(start, 101, 3))
extend adds all the items from its argument to the list as if it were:
for v in range(start, 101, 3):
lst.append(v)
In both of these it doesn't matter if range returns an iterable or a list as both are iterable and iterate is what extend does to get the items from its argument.
I'm trying to write a pairwise multiplier function which takes two arguments, both being lists. pairwise_multiply should return a new list with each of the elements in the two input lists multiplied together in a pairwise fashion. e.g.
result = pairwise_multiply([1, 2], [3, 4])
print(result)
> [3, 8]
This is my current function but I keep getting syntax errors:
def pairwise_multiply([l1], [l2]):
i = 0
while 1 <= len(l1):
lst = int(l1[i] * l2[i])
i = i + 1
return lst
In your code here -
def pairwise_multiply([l1], [l2]):
You don't need square brackets to pass lists as arguments. Replace it with -
def pairwise_multiply(l1, l2):
Another implementation, more pythonic would be to use list comprehension with zip -
[i*j for i, j in zip(l1, l2)]
What zip does is (from official documentation)-
Make an iterator that aggregates elements from each of the iterables.
Returns an iterator of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The iterator stops when the shortest input iterable is exhausted. With a single iterable argument, it returns an iterator of 1-tuples
There are some syntax and logic errors in this snippet.
def pairwise_multiply([l1], [l2]) As #FHTMitchell pointed out, you should cannot not use [...] when naming the arguments. This should be def pairwise_multiply(l1, l2)
while 1 <= len(l1) you mean i, not 1, right? Otherwise you will have an infinite loop. Also, since Python uses zero-based indexing, <= should become <.
You overwrite lst in every iteration. Your function will only return (if at all, see previous point) the result of the last multiplication.
Taking these into account, your code can be transformed to
def pairwise_multiply(l1, l2):
lst = []
i = 0
while i < len(l1):
lst.append(int(l1[i] * l2[i]))
i = i + 1
return lst
But it has many points of failure (for example, what if l1 and l2 are not the same length?), too long and not pythonic.
We can use zip and list comprehension like #ThatBird suggested in their answer.
Given a list of integers, I want to check a second list and remove from the first only those which can not be made from the sum of two numbers from the second. So given a = [3,19,20] and b = [1,2,17], I'd want [3,19].
Seems like a a cinch with two nested loops - except that I've gotten stuck with break and continue commands.
Here's what I have:
def myFunction(list_a, list_b):
for i in list_a:
for a in list_b:
for b in list_b:
if a + b == i:
break
else:
continue
break
else:
continue
list_a.remove(i)
return list_a
I know what I need to do, just the syntax seems unnecessarily confusing. Can someone show me an easier way? TIA!
You can do like this,
In [13]: from itertools import combinations
In [15]: [item for item in a if item in [sum(i) for i in combinations(b,2)]]
Out[15]: [3, 19]
combinations will give all possible combinations in b and get the list of sum. And just check the value is present in a
Edit
If you don't want to use the itertools wrote a function for it. Like this,
def comb(s):
for i, v1 in enumerate(s):
for j in range(i+1, len(s)):
yield [v1, s[j]]
result = [item for item in a if item in [sum(i) for i in comb(b)]]
Comments on code:
It's very dangerous to delete elements from a list while iterating over it. Perhaps you could append items you want to keep to a new list, and return that.
Your current algorithm is O(nm^2), where n is the size of list_a, and m is the size of list_b. This is pretty inefficient, but a good start to the problem.
Thee's also a lot of unnecessary continue and break statements, which can lead to complicated code that is hard to debug.
You also put everything into one function. If you split up each task into different functions, such as dedicating one function to finding pairs, and one for checking each item in list_a against list_b. This is a way of splitting problems into smaller problems, and using them to solve the bigger problem.
Overall I think your function is doing too much, and the logic could be condensed into much simpler code by breaking down the problem.
Another approach:
Since I found this task interesting, I decided to try it myself. My outlined approach is illustrated below.
1. You can first check if a list has a pair of a given sum in O(n) time using hashing:
def check_pairs(lst, sums):
lookup = set()
for x in lst:
current = sums - x
if current in lookup:
return True
lookup.add(x)
return False
2. Then you could use this function to check if any any pair in list_b is equal to the sum of numbers iterated in list_a:
def remove_first_sum(list_a, list_b):
new_list_a = []
for x in list_a:
check = check_pairs(list_b, x)
if check:
new_list_a.append(x)
return new_list_a
Which keeps numbers in list_a that contribute to a sum of two numbers in list_b.
3. The above can also be written with a list comprehension:
def remove_first_sum(list_a, list_b):
return [x for x in list_a if check_pairs(list_b, x)]
Both of which works as follows:
>>> remove_first_sum([3,19,20], [1,2,17])
[3, 19]
>>> remove_first_sum([3,19,20,18], [1,2,17])
[3, 19, 18]
>>> remove_first_sum([1,2,5,6],[2,3,4])
[5, 6]
Note: Overall the algorithm above is O(n) time complexity, which doesn't require anything too complicated. However, this also leads to O(n) extra auxiliary space, because a set is kept to record what items have been seen.
You can do it by first creating all possible sum combinations, then filtering out elements which don't belong to that combination list
Define the input lists
>>> a = [3,19,20]
>>> b = [1,2,17]
Next we will define all possible combinations of sum of two elements
>>> y = [i+j for k,j in enumerate(b) for i in b[k+1:]]
Next we will apply a function to every element of list a and check if it is present in above calculated list. map function can be use with an if/else clause. map will yield None in case of else clause is successful. To cater for this we can filter the list to remove None values
>>> list(filter(None, map(lambda x: x if x in y else None,a)))
The above operation will output:
>>> [3,19]
You can also write a one-line by combining all these lines into one, but I don't recommend this.
you can try something like that:
a = [3,19,20]
b= [1,2,17,5]
n_m_s=[]
data=[n_m_s.append(i+j) for i in b for j in b if i+j in a]
print(set(n_m_s))
print("after remove")
final_data=[]
for j,i in enumerate(a):
if i not in n_m_s:
final_data.append(i)
print(final_data)
output:
{19, 3}
after remove
[20]
I've got a list in which some items shall be moved into a separate list (by a comparator function). Those elements are pure dicts. The question is how should I iterate over such list.
When iterating the simplest way, for element in mylist, then I don't know the index of the element. There's no .iteritems() methods for lists, which could be useful here. So I've tried to use for index in range(len(mylist)):, which [1] seems over-complicated as for python and [2] does not satisfy me, since range(len()) is calculated once in the beginning and if I remove an element from the list during iteration, I'll get IndexError: list index out of range.
Finally, my question is - how should I iterate over a python list, to be able to remove elements from the list (using a comparator function and put them in another list)?
You can use enumerate function and make a temporary copy of the list:
for i, value in enumerate(old_list[:]):
# i == index
# value == dictionary
# you can safely remove from old_list because we are iterating over copy
Creating a new list really isn't much of a problem compared to removing items from the old one. Similarly, iterating twice is a very minor performance hit, probably swamped by other factors. Unless you have a very good reason to do otherwise, backed by profiling your code, I'd recommend iterating twice and building two new lists:
from itertools import ifilter, ifilterfalse
l1 = list(ifilter(condition, l))
l2 = list(ifilterfalse(condition, l))
You can slice-assign the contents of one of the new lists into the original if you want:
l[:] = l1
If you're absolutely sure you want a 1-pass solution, and you're absolutely sure you want to modify the original list in place instead of creating a copy, the following avoids quadratic performance hits from popping from the middle of a list:
j = 0
l2 = []
for i in range(len(l)):
if condition(l[i]):
l[j] = l[i]
j += 1
else:
l2.append(l[i])
del l[j:]
We move each element of the list directly to its final position without wasting time shifting elements that don't really need to be shifted. We could use for item in l if we wanted, and it'd probably be a bit faster, but when the algorithm involves modifying the thing we're iterating over, I prefer the explicit index.
I prefer not to touch the original list and do as #Martol1ni, but one way to do it in place and not be affected by the removal of elements would be to iterate backwards:
for i in reversed(range(len()):
# do the filtering...
That will affect only the indices of elements that you have tested/removed already
Try the filter command, and you can override the original list with it too if you don't need it.
def cmp(i): #Comparator function returning a boolean for a given item
...
# mylist is the initial list
mylist = filter(cmp, mylist)
mylist is now a generator of suitable items. You can use list(mylist) if you need to use it more than once.
Haven't tried this yet but.. i'll give it a quick shot:
new_list = [old.pop(i) for i, x in reversed(list(enumerate(old))) if comparator(x)]
You can do this, might be one line too much though.
new_list1 = [x for x in old_list if your_comparator(x)]
new_list2 = [x for x in old_list if x not in new_list1]
A multidimensional list like l=[[1,2],[3,4]] could be converted to a 1D one by doing sum(l,[]). How does this happen?
(This doesn't work directly for higher multidimensional lists, but it can be repeated to handle those cases. For example if A is a 3D-list, then sum(sum(A),[]),[]) will flatten A to a 1D list.)
If your list nested is, as you say, "2D" (meaning that you only want to go one level down, and all 1-level-down items of nested are lists), a simple list comprehension:
flat = [x for sublist in nested for x in sublist]
is the approach I'd recommend -- much more efficient than summing would be (sum is intended for numbers -- it was just too much of a bother to somehow make it block all attempts to "sum" non-numbers... I was the original proposer and first implementer of sum in the Python standard library, so I guess I should know;-).
If you want to go down "as deep as it takes" (for deeply nested lists), recursion is the simplest way, although by eliminating the recursion you can get higher performance (at the price of higher complication).
This recipe suggests a recursive solution, a recursion elimination, and other approaches
(all instructive, though none as simple as the one-liner I suggested earlier in this answer).
sum adds a sequence together using the + operator. e.g sum([1,2,3]) == 6. The 2nd parameter is an optional start value which defaults to 0. e.g. sum([1,2,3], 10) == 16.
In your example it does [] + [1,2] + [3,4] where + on 2 lists concatenates them together. Therefore the result is [1,2,3,4]
The empty list is required as the 2nd paramter to sum because, as mentioned above, the default is for sum to add to 0 (i.e. 0 + [1,2] + [3,4]) which would result in unsupported operand type(s) for +: 'int' and 'list'
This is the relevant section of the help for sum:
sum(sequence[, start]) -> value
Returns the sum of a sequence of
numbers (NOT strings) plus the value
of parameter 'start' (which defaults
to 0).
Note
As wallacoloo comented this is not a general solution for flattening any multi dimensional list. It just works for a list of 1D lists due to the behavior described above.
Update
For a way to flatten 1 level of nesting see this recipe from the itertools page:
def flatten(listOfLists):
"Flatten one level of nesting"
return chain.from_iterable(listOfLists)
To flatten more deeply nested lists (including irregularly nested lists) see the accepted answer to this question (there are also some other questions linked to from that question itself.)
Note that the recipe returns an itertools.chain object (which is iterable) and the other question's answer returns a generator object so you need to wrap either of these in a call to list if you want the full list rather than iterating over it. e.g. list(flatten(my_list_of_lists)).
For any kind of multidiamentional array, this code will do flattening to one dimension :
def flatten(l):
try:
return flatten(l[0]) + (flatten(l[1:]) if len(l) > 1 else []) if type(l) is list else [l]
except IndexError:
return []
It looks to me more like you're looking for a final answer of:
[3, 7]
For that you're best off with a list comprehension
>>> l=[[1,2],[3,4]]
>>> [x+y for x,y in l]
[3, 7]
I wrote a program to do multi-dimensional flattening using recursion. If anyone has comments on making the program better, you can always see me smiling:
def flatten(l):
lf=[]
li=[]
ll=[]
p=0
for i in l:
if type(i).__name__=='list':
li.append(i)
else:
lf.append(i)
ll=[x for i in li for x in i]
lf.extend(ll)
for i in lf:
if type(i).__name__ =='list':
#not completely flattened
flatten(lf)
else:
p=p+1
continue
if p==len(lf):
print(lf)
I've written this function:
def make_array_single_dimension(l):
l2 = []
for x in l:
if type(x).__name__ == "list":
l2 += make_array_single_dimension(x)
else:
l2.append(x)
return l2
It works as well!
The + operator concatenates lists and the starting value is [] an empty list.