Related
there is an issue that i am trying to solve which requires me to generate the indices for an n - dimensional list. Eg: [5, 4, 3] is a 3 dimensional list so valid indices are [0, 0, 0], [0, 0, 1], [0, 1, 0] ... [2, 2, 1] ..... [4, 3, 2]. The best I could come up with a recursive algorithm but this isn't constant space
def f1(dims):
def recur(res, lst, depth, dims):
if depth == len(dims):
res.append(lst[::])
return
curr = dims[depth]
for i in range(curr):
lst[depth] = i
recur(res, lst, depth + 1, dims)
res = []
lst = [0] * len(dims)
recur(res, lst, 0, dims)
return res
the dimensions can be any number , ie: 4D, 5D, 15D etc. Each time it would be given in the form of a list . Eg: 5D would be [3,2,1,5,2] and I would need to generate all the valid indices for these while using constant space ( just while loops and indices processing ) . How would I go about generating these efficiently without the help of any in built python functions ( just while, for loops etc )
This is a working solution in constant space (a single loop variable i, and a vector idx of which modified copies are being yielded, and a single temporary length variable n to avoid calling len() on every iteration):
def all_indices(dimensions):
n = len(dimensions)
idx = [0] * n
while True:
for i in range(n):
yield tuple(idx)
if idx[i] + 1 < dimensions[i]:
idx[i] += 1
break
else:
idx[i] = 0
if not any(idx):
break
print(list(all_indices([3, 2, 1])))
Result:
[(0, 0, 0), (1, 0, 0), (2, 0, 0), (0, 0, 0), (0, 1, 0), (1, 1, 0), (2, 1, 0), (0, 1, 0), (0, 0, 0)]
As pointed out in the comments, there's duplicates there, a bit sloppy, this is cleaner:
def all_indices(dimensions):
n = len(dimensions)
idx = [0] * n
yield tuple(idx) # yield the initial 'all zeroes' state
while True:
for i in range(n):
if idx[i] + 1 < dimensions[i]:
idx[i] += 1
yield tuple(idx) # yield new states
break
else:
idx[i] = 0 # no yield, repeated state
if not any(idx):
break
print(list(all_indices([3, 2, 1])))
Alternatively, you could yield before the break instead of at the start of the loop, but I feel having the 'all zeroes' at the start looks cleaner.
The break is there to force a depth first on running through the indices, which ensures the loop never reaches 'all zeroes' again before having passed all possibilities. Try removing the break and then passing in something like [2, 1, 2] and you'll find it is missing a result.
I think a break is actually the 'clean' way to do it, since it allows using a simple for instead of using a while with a more complicated condition and a separate increment statement. You could do that though:
def all_indices3(dimensions):
n = len(dimensions)
idx = [1] + [0] * (n - 1)
yield tuple([0] * n)
while any(idx):
yield tuple(idx)
i = 0
while i < n and idx[i] + 1 == dimensions[i]:
idx[i] = 0
i += 1 % n
if i < n:
idx[i] += 1
This has the same result, but only uses while, if and yields the results in the same order.
There is a task on CodeFights involving building a two-dimensional array:
A spiral matrix is a square matrix of size n × n. It contains all the
integers in range from 1 to n * n so that number 1 is written in the
bottom right corner, and all other numbers are written in increasing
order spirally in the counterclockwise direction.
Given the size of the matrix n, your task is to create a spiral
matrix.
One is supposed to fill only one gap denoted by ... in the following code:
def createSpiralMatrix(n):
dirs = [(-1, 0), (0, -1), (1, 0), (0, 1)]
curDir = 0
curPos = (n - 1, n - 1)
res = ...
for i in range(1, n * n + 1):
res[curPos[0]][curPos[1]] = i
nextPos = curPos[0] + dirs[curDir][0], curPos[1] + dirs[curDir][1]
if not (0 <= nextPos[0] < n and
0 <= nextPos[1] < n and
res[nextPos[0]][nextPos[1]] == 0):
curDir = (curDir + 1) % 4
nextPos = curPos[0] + dirs[curDir][0], curPos[1] + dirs[curDir][1]
curPos = nextPos
return res
When I fill in the following code, all tests are passed:
res = [[0 for item in range(n)] for sublist in range(n)]
However, if I slightly change it to:
res = [[None for item in range(n)] for sublist in range(n)]
I receive the following error message:
Execution error on test 1: Something went wrong when executing the solution - program stopped unexpectedly with an error.
Traceback (most recent call last):
file.py3 on line ?, in getUserOutputs
userOutput = _runiuljw(testInputs[i])
file.py3 on line ?, in _runiuljw
return createSpiralMatrix(*_fArgs_jlosndfelxsr)
file.py3 on line 8, in createSpiralMatrix
res[curPos[0]][curPos[1]] = i
IndexError: list index out of range
Test 1 Input: n: 3 Output: Empty Expected Output: [[5,4,3], [6,9,2],
[7,8,1]] Console Output: Empty
The same result (with the error message) is with the following code:
res = [list(range(n)) for sublist in range(n)]
All three options build arrays of the same size:
n = 3
res = [[0 for item in range(n)] for sublist in range(n)]
[[0, 0, 0], [0, 0, 0], [0, 0, 0]]
res = [[None for item in range(n)] for sublist in range(n)]
[[None, None, None], [None, None, None], [None, None, None]]
res = [list(range(n)) for sublist in range(n)]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
Am I missing something obvious?
If you want it to work with None, then you simply have to change the third condition of your if not ... statement to agree with how you initialized res. Otherwise, nextPos will incorrectly become out of the range of your 2D array.
def createSpiralMatrix(n):
dirs = [(-1, 0), (0, -1), (1, 0), (0, 1)]
curDir = 0
curPos = (n - 1, n - 1)
res = [[None for item in range(n)] for sublist in range(n)]
for i in range(1, n * n + 1):
res[curPos[0]][curPos[1]] = i
nextPos = curPos[0] + dirs[curDir][0], curPos[1] + dirs[curDir][1]
if not (0 <= nextPos[0] < n and
0 <= nextPos[1] < n and
res[nextPos[0]][nextPos[1]] is None): # Changed this line
curDir = (curDir + 1) % 4
nextPos = curPos[0] + dirs[curDir][0], curPos[1] + dirs[curDir][1]
curPos = nextPos
return res
How can I randomly shuffle a list so that none of the elements remains in its original position?
In other words, given a list A with distinct elements, I'd like to generate a permutation B of it so that
this permutation is random
and for each n, a[n] != b[n]
e.g.
a = [1,2,3,4]
b = [4,1,2,3] # good
b = [4,2,1,3] # good
a = [1,2,3,4]
x = [2,4,3,1] # bad
I don't know the proper term for such a permutation (is it "total"?) thus having a hard time googling. The correct term appears to be "derangement".
After some research I was able to implement the "early refusal" algorithm as described e.g. in this paper [1]. It goes like this:
import random
def random_derangement(n):
while True:
v = [i for i in range(n)]
for j in range(n - 1, -1, -1):
p = random.randint(0, j)
if v[p] == j:
break
else:
v[j], v[p] = v[p], v[j]
else:
if v[0] != 0:
return tuple(v)
The idea is: we keep shuffling the array, once we find that the permutation we're working on is not valid (v[i]==i), we break and start from scratch.
A quick test shows that this algorithm generates all derangements uniformly:
N = 4
# enumerate all derangements for testing
import itertools
counter = {}
for p in itertools.permutations(range(N)):
if all(p[i] != i for i in p):
counter[p] = 0
# make M probes for each derangement
M = 5000
for _ in range(M*len(counter)):
# generate a random derangement
p = random_derangement(N)
# is it really?
assert p in counter
# ok, record it
counter[p] += 1
# the distribution looks uniform
for p, c in sorted(counter.items()):
print p, c
Results:
(1, 0, 3, 2) 4934
(1, 2, 3, 0) 4952
(1, 3, 0, 2) 4980
(2, 0, 3, 1) 5054
(2, 3, 0, 1) 5032
(2, 3, 1, 0) 5053
(3, 0, 1, 2) 4951
(3, 2, 0, 1) 5048
(3, 2, 1, 0) 4996
I choose this algorithm for simplicity, this presentation [2] briefly outlines other ideas.
References:
[1] An analysis of a simple algorithm for random derangements. Merlini, Sprugnoli, Verri. WSPC Proceedings, 2007.
[2] Generating random derangements. Martínez, Panholzer, Prodinger.
Such permutations are called derangements. In practice you can just try random permutations until hitting a derangement, their ratio approaches the inverse of 'e' as 'n' grows.
As a possible starting point, the Fisher-Yates shuffle goes like this.
def swap(xs, a, b):
xs[a], xs[b] = xs[b], xs[a]
def permute(xs):
for a in xrange(len(xs)):
b = random.choice(xrange(a, len(xs)))
swap(xs, a, b)
Perhaps this will do the trick?
def derange(xs):
for a in xrange(len(xs) - 1):
b = random.choice(xrange(a + 1, len(xs) - 1))
swap(xs, a, b)
swap(len(xs) - 1, random.choice(xrange(n - 1))
Here's the version described by Vatine:
def derange(xs):
for a in xrange(1, len(xs)):
b = random.choice(xrange(0, a))
swap(xs, a, b)
return xs
A quick statistical test:
from collections import Counter
def test(n):
derangements = (tuple(derange(range(n))) for _ in xrange(10000))
for k,v in Counter(derangements).iteritems():
print('{} {}').format(k, v)
test(4):
(1, 3, 0, 2) 1665
(2, 0, 3, 1) 1702
(3, 2, 0, 1) 1636
(1, 2, 3, 0) 1632
(3, 0, 1, 2) 1694
(2, 3, 1, 0) 1671
This does appear uniform over its range, and it has the nice property that each element has an equal chance to appear in each allowed slot.
But unfortunately it doesn't include all of the derangements. There are 9 derangements of size 4. (The formula and an example for n=4 are given on the Wikipedia article).
This should work
import random
totalrandom = False
array = [1, 2, 3, 4]
it = 0
while totalrandom == False:
it += 1
shuffledArray = sorted(array, key=lambda k: random.random())
total = 0
for i in array:
if array[i-1] != shuffledArray[i-1]: total += 1
if total == 4:
totalrandom = True
if it > 10*len(array):
print("'Total random' shuffle impossible")
exit()
print(shuffledArray)
Note the variable it which exits the code if too many iterations are called. This accounts for arrays such as [1, 1, 1] or [3]
EDIT
Turns out that if you're using this with large arrays (bigger than 15 or so), it will be CPU intensive. Using a randomly generated 100 element array and upping it to len(array)**3, it takes my Samsung Galaxy S4 a long time to solve.
EDIT 2
After about 1200 seconds (20 minutes), the program ended saying 'Total Random shuffle impossible'. For large arrays, you need a very large number of permutations... Say len(array)**10 or something.
Code:
import random, time
totalrandom = False
array = []
it = 0
for i in range(1, 100):
array.append(random.randint(1, 6))
start = time.time()
while totalrandom == False:
it += 1
shuffledArray = sorted(array, key=lambda k: random.random())
total = 0
for i in array:
if array[i-1] != shuffledArray[i-1]: total += 1
if total == 4:
totalrandom = True
if it > len(array)**3:
end = time.time()
print(end-start)
print("'Total random' shuffle impossible")
exit()
end = time.time()
print(end-start)
print(shuffledArray)
Here is a smaller one, with pythonic syntax -
import random
def derange(s):
d=s[:]
while any([a==b for a,b in zip(d,s)]):random.shuffle(d)
return d
All it does is shuffles the list until there is no element-wise match. Also, be careful that it'll run forever if a list that cannot be deranged is passed.It happens when there are duplicates. To remove duplicates simply call the function like this derange(list(set(my_list_to_be_deranged))).
import random
a=[1,2,3,4]
c=[]
i=0
while i < len(a):
while 1:
k=random.choice(a)
#print k,a[i]
if k==a[i]:
pass
else:
if k not in c:
if i==len(a)-2:
if a[len(a)-1] not in c:
if k==a[len(a)-1]:
c.append(k)
break
else:
c.append(k)
break
else:
c.append(k)
break
i=i+1
print c
A quick way is to try to shuffle your list until you reach that state. You simply try to shuffle your list until you are left with a list that satisfies your condition.
import random
import copy
def is_derangement(l_original, l_proposal):
return all([l_original[i] != item for i, item in enumerate(l_proposal)])
l_original = [1, 2, 3, 4, 5]
l_proposal = copy.copy(l_original)
while not is_derangement(l_original, l_proposal):
random.shuffle(l_proposal)
print(l_proposal)
I am trying to implement ternary heap sort, where every parent of the heap has 3 children, however I can not get my code to return a sorted list. I think I may be making a logical error somewhere but can not spot it, any help fixing this code would be much appreciated.
def swap(i, j):
sqc[i], sqc[j] = sqc[j], sqc[i]
def heapify(end,i):
l=3 * (i + 1)
m=3 * (i + 2)
r=3 * (i + 3)
max=i
if l < end and sqc[i] < sqc[l]:
max = l
if r < end and sqc[max] < sqc[r]:
max = r
if m < end and sqc[max] < sqc[m]:
max = m
if max != i:
swap(i, max)
heapify(end, max)
def heap_sort():
end = len(sqc)
start = end / 2 - 1
for i in range(start, -1, -1):
heapify(end, i)
for i in range(end-1, 0, -1):
swap(i, 0)
heapify(i, 0)
sqc = [2, 7, 1, -2, 56, 5, 3]
heap_sort()
print(sqc)
[7, 1, -2, 56, 5, 2, 3] is what is returned, completely out of order.
Code
def heapSort3(a):
def sift(start, count):
root = start
while root * 3 + 1 < count:
r3 = root * 3
upper = min(count - r3, 4)
children = list(range(r3 + 1, r3 + upper))
min_child = min((a[i], i) for i in children)
v, i = min_child
if a[root] > a[i]:
a[root], a[i] = a[i], a[root]
root = i
else:
break
count = len(a)
for start in reversed(range(count // 3 + 2)):
sift(start, count)
for end in reversed(range(count)):
a[end], a[0] = a[0], a[end]
sift(0, end)
return a
Test on sorted and reverse-sorted
for i in range(2, 25):
print '-' * 30
data = list(range(i))
sorted_data = heapSort3(data)
print i, sorted_data
data = list(reversed(range(i)))
sorted_data = heapSort3(data)
print i, sorted_data
Test on shuffled data
from random import shuffle
for i in range(1, 100):
print '-' * 30, i
expected = list(reversed(range(i)))
for _ in range(5000):
data = list(range(i))
shuffle(data)
sorted_data = heapSort3(data)
assert sorted_data == expected
Reference
Adapted from this code. Sort of.
Here is a solution that merely corrects your heapify function:
def swap(series, i, j):
series[i], series[j] = series[j], series[i]
def heapify(series, end, i):
firstchild = 3*i + 1
lastchild = min(firstchild + 3, end)
children = list(range(firstchild, lastchild))
maxelem = max((series[j], j) for j in [i] + children)
maxval, maxindex = maxelem
if maxindex != i:
swap(series, i, maxindex)
heapify(series, end, maxindex)
def heap_sort(series):
end = len(series)
start = end / 2 - 1
for i in range(start, -1, -1):
heapify(series, end, i)
for i in range(end-1, 0, -1):
swap(series, i, 0)
heapify(series, i, 0)
return series
heapsort changes
You can also change your heapsort code to this, but it is not necessary:
def heap_sort(series):
end = len(series)
start = end // 2
for i in reversed(range(start)):
heapify(series, end, i)
for i in reversed(range(end)):
swap(series, i, 0)
heapify(series, i, 0)
return series
Test on your data
>>> sqc = [2, 7, 1, -2, 56, 5, 3]
>>> print heap_sort(sqc)
[-2, 1, 2, 3, 5, 7, 56]
Test on random shuffled data
from random import shuffle
for i in range(1, 100):
print '-' * 30, i
expected = list(range(i))
for _ in range(5000):
data = list(range(i))
shuffle(data)
sorted_data = heap_sort(data)
assert sorted_data == expected
HI,
I'm try to find a general expression to obtain exponents of a multivariate polynomial of order order and with n_variables, like the one presented in this reference in equation (3).
Here is my current code, which uses an itertools.product generator.
def generalized_taylor_expansion_exponents( order, n_variables ):
"""
Find the exponents of a multivariate polynomial expression of order
`order` and `n_variable` number of variables.
"""
exps = (p for p in itertools.product(range(order+1), repeat=n_variables) if sum(p) <= order)
# discard the first element, which is all zeros..
exps.next()
return exps
The desired out is this:
for i in generalized_taylor_expansion_exponents(order=3, n_variables=3):
print i
(0, 0, 1)
(0, 0, 2)
(0, 0, 3)
(0, 1, 0)
(0, 1, 1)
(0, 1, 2)
(0, 2, 0)
(0, 2, 1)
(0, 3, 0)
(1, 0, 0)
(1, 0, 1)
(1, 0, 2)
(1, 1, 0)
(1, 1, 1)
(1, 2, 0)
(2, 0, 0)
(2, 0, 1)
(2, 1, 0)
(3, 0, 0)
Actually this code executes fast, because the generator object is only created. If i want to fill a list with values from this generator execution really starts to be slow, mainly because of the high number of calls to sum. Tipical value for order and n_variables is 5 and 10, respectively.
How can i significantly improve execution speed?
Thanks for any help.
Davide Lasagna
Actually your biggest performance issue is that most of the tuples you're generating are too big and need to be thrown away. The following should generate exactly the tuples you want.
def generalized_taylor_expansion_exponents( order, n_variables ):
"""
Find the exponents of a multivariate polynomial expression of order
`order` and `n_variable` number of variables.
"""
pattern = [0] * n_variables
for current_sum in range(1, order+1):
pattern[0] = current_sum
yield tuple(pattern)
while pattern[-1] < current_sum:
for i in range(2, n_variables + 1):
if 0 < pattern[n_variables - i]:
pattern[n_variables - i] -= 1
if 2 < i:
pattern[n_variables - i + 1] = 1 + pattern[-1]
pattern[-1] = 0
else:
pattern[-1] += 1
break
yield tuple(pattern)
pattern[-1] = 0
I would try writing it recursively so as to generate only the desired elements:
def _gtee_helper(order, n_variables):
if n_variables == 0:
yield ()
return
for i in range(order + 1):
for result in _gtee_helper(order - i, n_variables - 1):
yield (i,) + result
def generalized_taylor_expansion_exponents(order, n_variables):
"""
Find the exponents of a multivariate polynomial expression of order
`order` and `n_variable` number of variables.
"""
result = _gtee_helper(order, n_variables)
result.next() # discard the first element, which is all zeros
return result