I am trying to understand the "yield".
First, the code that I can't understand is below.
def permutations(seq):
if len(seq) <= 1: yield seq
else:
for perm in permutations(seq[1:]):
for i in range(len(perm)+1):
yield perm[i:] + seq[0:1] + perm[:i]
print list(permutations(['police', 'buffalo', 'fish']))
The result is as below:
[['fish', 'buffalo', 'police'], ['buffalo', 'police', 'fish'], ['police', 'fish', 'buffalo'], ['buffalo', 'fish', 'police'], ['fish', 'police', 'buffalo'], ['police', 'buffalo', 'fish']]
My undertanding level about "yield" is just it is used for generater. And I can understand the code below.
def reverse(data):
for index in range(len(data) -1, -1, -1):
yield data[index]
for char in reverse('golf'):
print(char)
f
l
o
g
My level is just understanding above.. but with recursion, I can't understand... please explain. Thanks
Yes, yield is for generators. That implies that when called, they return iterators. Generators can be recursive: they will call themselves, get an iterator and the iterate over it, on each Iteration they can yield up as many or few of their own elements as they like.
In your example permutations is a generator which always returns an iterator over lists.
if len(seq) <= 1: yield seq
Is simple enough: in the trivial case, just generate one list, seq itself.
for perm in permutations(seq[1:]):
...
Means "now iterate over different sequence-of-lists", in this case that is the sequence of all permutations of the elements after the first. In each iteration we have a nested loop that inserts the first element into each position of the permutation and yields the result.
I hope that helps. It's a little hard for me because I don't know exactly what you are confused about.
Update: the OP wants to know why the first result is in reverse order. Consider the line:
yield perm[i:] + seq[0:1] + perm[:i]
For the first result (i=0) this is equivalent to yield perm + seq[0:1] -- the first element is sent to the end of the yielded list. By induction, this result is the reverse of seq. If you want the first result to be ['police', 'buffalo', 'fish'] then you can do:
yield perm[:i] + seq[0:1] + perm[i:]
The code you've given works according to this algorithm:
If the sequence has one item or is empty, the only permutation is the sequence itself
Separate the sequence into its first element, and the list of all its remaining elements
Consider all the permutations of the remainder of the sequence, according to this same algorithm. For each permutation:
For each position in the permutation:
Split the permutation at that position
Insert the first element of the original sequence in that split. This is a new permutation of the original sequence.
Done.
The original code is slightly strange in how it achieves point 3.1.2. We can make it this a little bit clearer by using more variables to show the relationship to the algorithm more clearly - this assumes you're using Python 3:
def permutations(seq):
if len(seq) <= 1:
yield seq
else:
first, *rest = seq
for perm in permutations(rest):
for i in range(len(perm)+1):
before = perm[:i]
after = perm[i:]
yield after + [first] + before
As you can see, the final line switches the start and end of the permutation for no real reason. It could just as easily do before + [first] + after, but the author decided not to. This doesn't affect how the algorithm works - it will find all the orderings, including the mirrored ones - but it means the order it produces them in might be a little strange.
You can use a similar recursive generator to implement reverse as well. In this case, the algorithm is:
If the sequence only has one item or is empty, it is its own reverse
Split the sequence into its first element and the list of all remaining elements
Use this algorithm to reverse the remaining elements
Yield each item of the result of the last step
Yield the first element of the original sequence
In Python, it looks like this:
def reverse(seq):
if len(seq) <= 1:
return seq
first, *rest = seq
for item in reverse(rest):
yield item
# Or you could use:
# yield from reverse(rest)
# Instead of the above loop
yield first
Related
def permute2(seq):
if not seq: # Shuffle any sequence: generator
yield seq # Empty sequence
else:
for i in range(len(seq)):
rest = seq[:i] + seq[i+1:] # Delete current node
for x in permute2(rest): # Permute the others
# In some cases x = empty string
yield seq[i:i+1] + x # Add node at front
for x in permute2('abc'):
print('result =',x)
When yielding the first result ('abc') the value of i == 0 and seq == 'abc'. The control flow then takes it to the top of the outer for loop where i == 1, which makes sense; however, seq == 'bc', which completely baffles me.
I'll try to explain it from an inductive standpoint.
Let the length of the sequence be n.
Our base case is n = 0, when we simply return the empty sequence (it is the only permutation).
If n > 0, then:
To define a permutation, we must first begin by choosing the first element. To define all permutations, obviously we must consider all elements of the sequence as possible first elements.
Once we have chosen the first element, we now need to choose the rest of the elements. But this is just the same as finding all the permutations of sequence without the element we chose. This new sequence is of length n-1, and so by induction we can solve it.
Slightly altering the original code to mimic my explanation and hopefully make it clearer:
def permute2(seq):
length = len(seq)
if (len == 0):
# This is our base case, the empty sequence, with only one possible permutation.
yield seq
return
for i in range(length):
# We take the ith element as the first element of our permutation.
first_element_as_single_element_collection = seq[i:i+1]
# Note other_elements has len(other_elements) == len(seq) - 1
other_elements = seq[:i] + seq[i+1:]
for permutation_of_smaller_collection in permute2(other_elements):
yield first_element_as_single_element_collection + permutation_of_smaller_collection
for x in permute2('abc'):
print('result =',x)
Hopefully it is clear that the original code does exactly the same thing as the above code.
I've written a function to create combinations of inputs of an arbitrary length, so recursion seemed to be the obvious way to do it. While it's OK for a small toy example to return a list of the results, I'd like to yield them instead. I've read about yield from, but don't fully understand how it is used, the examples don't appear to cover my use case, and hoping'n'poking it into my code has not yet produced anything that works. Note that writing this recursive code was at the limit of my python ability, hence the copious debug print statements.
This is the working list return code, with my hopeful non-working yield commented out.
def allposs(elements, output_length):
"""
return all zero insertion paddings of elements up to output_length maintaining order
elements - an iterable of length >= 1
output_length >= len(elements)
for instance allposs((3,1), 4) returns
[[3,1,0,0], [3,0,1,0], [3,0,0,1], [0,3,1,0], [0,3,0,1], [0,0,3,1]]
"""
output_list = []
def place_nth_element(nth, start_at, output_so_far):
# print('entering place_nth_element with nth =', nth,
# ', start_at =', start_at,
# ', output_so_far =', output_so_far)
last_pos = output_length - len(elements) + nth
# print('iterating over range',start_at, 'to', last_pos+1)
for pos in range(start_at, last_pos+1):
output = list(output_so_far)
# print('placing', elements[nth], 'at position', pos)
output[pos] = elements[nth]
if nth == len(elements)-1:
# print('appending output', output)
output_list.append(output)
# yield output
else:
# print('making recursive call')
place_nth_element(nth+1, pos+1, output)
place_nth_element(0, 0, [0]*output_length)
return output_list
if __name__=='__main__':
for q in allposs((3,1), 4):
print(q)
What is the syntax to use yield from to get my list generated a combination at a time?
Recursive generators are a powerful tool and I'm glad you're putting in the effort to study them.
What is the syntax to use yield from to get my list generated a combination at a time?
You put yield from in front of the expression from which results should be yielded; in your case, the recursive call. Thus: yield from place_nth_element(nth+1, pos+1, output). The idea is that each result from the recursively-called generator is iterated over (behind the scenes) and yielded at this point in the process.
Note that for this to work:
You need to yield the individual results at the base level of the recursion
To "collect" the results from the resulting generator, you need to iterate over the result from the top-level call. Fortunately, iteration is built-in in a lot of places; for example, you can just call list and it will iterate for you.
Rather than nesting the recursive generator inside a wrapper function, I prefer to write it as a separate helper function. Since there is no longer a need to access output_list from the recursion, there is no need to form a closure; and flat is better than nested as they say. This does, however, mean that we need to pass elements through the recursion. We don't need to pass output_length because we can recompute it (the length of output_so_far is constant across the recursion).
Also, I find it's helpful, when doing these sorts of algorithms, to think as functionally as possible (in the paradigm sense - i.e., avoid side effects and mutability, and proceed by creating new objects). You had a workable approach using list to make copies (although it is clearer to use the .copy method), but I think there's a cleaner way, as shown below.
All this advice leads us to:
def place_nth_element(elements, nth, start_at, output_so_far):
last_pos = len(output_so_far) - len(elements) + nth
for pos in range(start_at, last_pos+1):
output = output_so_far[:pos] + (elements[nth],) + output_so_far[pos+1:]
if nth == len(elements)-1:
yield output
else:
yield from place_nth_element(elements, nth+1, pos+1, output)
def allposs(elements, output_length):
return list(place_nth_element(elements, 0, 0, (0,)*output_length))
HOWEVER, I would not solve the problem that way - because the standard library already offers a neat solution: we can find the itertools.combinations of indices where a value should go, and then insert them. Now that we no longer have to think recursively, we can go ahead and mutate values :)
from itertools import combinations
def place_values(positions, values, size):
result = [0] * size
for position, value in zip(positions, values):
result[position] = value
return tuple(result)
def possibilities(values, size):
return [
place_values(positions, values, size)
for positions in combinations(range(size), len(values))
]
This question already has answers here:
Are Python variables pointers? Or else, what are they?
(9 answers)
Closed 3 years ago.
I am writing a little program to create a list of permutations. I read about the algorithm on wikipedia.
My algorithm basically takes an initially sorted list of numbers, and permutes it in place. It then appends this new permutation to a list. When all permutations are found, it returns the list of lists containing all the permutations. It is very good at printing out the expected results, but when I try to add those results to a list, things get a little funny.
I noticed that every time I find the next permutation and append it to, the previous list elements get updated to the new permutation. So, at the end of it all, what gets returned is a list containing a bunch of copies of the same permutation (exactly the last permutation).
I've read that Python is pass by value, pass by reference and I've also read that it's neither. I'm not smart enough to argue with any of those people, but I am wondering why my program is doing this, and how to remedy it:
def lexi_order(nums):
permutations = []
length = len(nums)
while True:
# find largest index i such that nums[i] < nums[i + 1]
exists = False
for j, elem in enumerate(nums):
# check if last element
if j == length - 1:
break
if elem < nums[j + 1]:
i = j
exists = True
if not exists:
break
# find largest index k, such that i < k AND nums[i] < nums[k]
for j in range(i + 1, length):
if nums[j] > nums[i]:
k = j
# swap order of nums[i] and nums[k]
nums[i], nums[k] = nums[k], nums[i]
# reverse order of elements starting at position i+1
to_reverse = nums[i+1:][::-1]
nums[i+1::] = to_reverse
permutations.append(nums)
print(permutations)
return permutations
You're modifying the input (nums) in place each iteration through the loop, and then you keep adding a reference to the input to permutations. To fix it, make a copy of nums at the beginning of the loop and use it instead of the original everywhere inside it.
When you append nums to permutations, you are appending a reference to it, not copying all of the data over. When you modify nums, it gets modified everywhere. Python is pass by reference. If you make a change to a variable (not to be confused with reassigning it), that change will be reflected everywhere.
You need to make a copy of the passed nums, otherwise you are working on the passed reference. E.g.
def lexi_order(nums):
permutations = []
nums = list(nums) # We are now working on a copy, and won't mutate the original whatsoever.
length = len(nums)
...
using the itertools tool, I have all the possible permutations of a given list of numbers, but if the list is as follows:
List=[0,0,0,0,3,6,0,0,5,0,0]
itertools does not "know" that iterating the zeros is wasted work, for example the following iterations will appear in the results:
List=[0,3,0,0,0,6,0,0,5,0,0]
List=[0,3,0,0,0,6,0,0,5,0,0]
they are the same but itertools just takes the first zero ( for example ) and moves it at the fourth place in the list and vice-versa.
The question is: how can I iterate only some selected numbers and left alone others such like zero ? it can be with or without itertools.
Voilá - it works now - after getting the permutations on the "meat", I further get all possible combnations for the "0"s positions and yield
one permutation for each possible set of "0 positions" for each permutation
of the non-0s:
from itertools import permutations, combinations
def permut_with_pivot(sequence, pivot=0):
pivot_indexes = set()
seq_len = 0
def yield_non_pivots():
nonlocal seq_len
for i, item in enumerate(sequence):
if item != pivot:
yield item
else:
pivot_indexes.add(i)
seq_len = i + 1
def fill_pivots(permutation):
for pivot_positions in combinations(range(seq_len), len(pivot_indexes)):
sequence = iter(permutation)
yield tuple ((pivot if i in pivot_positions else next(sequence)) for i in range(seq_len))
for permutation in permutations(yield_non_pivots()):
for filled_permutation in fill_pivots(permutation):
yield filled_permutation
(I've used Python's 3 "nonlocal" keyword - if you are still on Python 2.7,
you will have to take another approach, like making seq_len be a list with a single item you can then repplace on the inner function)
My second try (the working one is actually the 3rd)
This is a naive approach that just keeps a cache of the already "seen" permutations - it saves on the work done to each permutation but notonthe work to generate all possible permutations:
from itertools import permutations
def non_repeating_permutations(seq):
seen = set()
for permutation in permutations(seq):
hperm = hash(permutation)
if hperm in seen:
continue
seen.add(hperm)
yield permutation
Append each result to a List. Now you'll have every single possible combination and then do the following:
list(set(your_big_list))
Set will narrow down the list down to only the unique permutations.
I'm not wholly sure if this is the problem you're trying to solve or you're worried about performance.
Regardless just made an account so I thought I'd try to contribute something
Your questions is unclear, but if you are trying to list the permutations without having 0 in your output, you can do it as follows:
from itertools import permutations
def perms( listy ):
return permutations( [i for i in listy if i!=0])
I've been having some trouble with recursion, especially with lists. I don't understand the process of 'building a list recursively' very well, since I don't really get where the lists are created. I have gotten this program to print out all the permutations of len(string), but I would like to extend it so that it'll also give me the permutations of length 1 to len(string) - 1. Here is what I have got:
def subset(string):
result = []
if len(string) == 0:
result.append(string)
return result
else:
for i in range(len(string)):
shorterstring = string[ :i] + string[i+1: ]
shortersets = subset(shorterstring)
for s in shortersets:
result.append(string[i] + s)
return result
Which gives:
print(subset("rum"))
['rum', 'rmu', 'urm', 'umr', 'mru', 'mur']
I don't understand why that when I change result.append(string[i] + s), to just result.append(s), I get no output at all.
If you change result.append(string[i] + s) to result.append(s), your code will only add permutations of length len(string) to results when len(string) == 0.
For your code to generate all permutations, the last for loop needs to be:
for s in shortersets:
result.append(string[i] + s)
result.append(s)
Note that when used with the original code, you will actually end up adding multiple instances of the same permutations to the final output. You could fix this by making results a set instead of a list, but you might want to try re-writing your code to avoid this inefficiency altogether.