We Keep Coding

Neater strategy to drop down the numbers in the 2D list

I have a problem. It is a 2D list of non-negative integers will be given like
0, 0, 2, 0, 1
0, 2, 1, 1, 0
3, 0, 2, 1, 0
0, 0, 0, 0, 0
I have to drop the numbers, number columns. e.g. drop down the 1's down 1 column, the 2's down 2 columns, the 3's down 3 columns, and so on. If the number can't be moved down enough, wrap it around the top. (e. g If there is a 3 in the second-to-last row, it should wrap around to the first row.) If two numbers map to the same slot, the biggest number takes that slot.
After this transformation the given matrix above will end up like:
0, 0, 2, 0, 0
3, 0, 0, 0, 1
0, 0, 2, 1, 0
0, 2, 0, 1, 0
Here's my trivial solution to the problem (Assumes a list l is pre-set):
new = [[0] * len(l[0]) for _ in range(len(l))]
idx = sorted([((n + x) % len(l), m, x) for n, y in enumerate(l) for m, x in enumerate(y)], key=lambda e: e[2])
for x, y, z in idx:
new[x][y] = z
print(new)
The strategy is:
Build a list new with 0s of the shape of l
Save the new indices of each number in l and each number as tuple pairs in idx
Sort idx by each number
Assign indices from idx to the respective numbers to new list
Print new
I am not satisfied with this strategy. Is there a neater/better way to do this? I can use numpy.

Let's say you have
a = np.array([
[0,0,2,0,1],
[0,2,1,1,0],
[3,0,2,1,0],
[0,0,0,0,0]])
You can get the locations of the elements with np.where or np.nonzero:
r, c = np.nonzero(a)
And the elements themselves with the index:
v = a[r, c]
Incrementing the row is simple now:
new_r = (r + v) % a.shape[0]
To settle collisions, sort the arrays so that large values come last:
i = v.argsort()
Now you can assign to a fresh matrix of zeros directly:
result = np.zeros_like(a)
result[new_r[i], c[i]] = v[i]
The result is
[[0 0 2 0 0]
[3 0 0 0 1]
[0 0 2 1 0]
[0 2 0 1 0]]

I suggest doing it like this if only because it's more readable :-
L = [[0, 0, 2, 0, 1],
[0, 2, 1, 1, 0],
[3, 0, 2, 1, 0],
[0, 0, 0, 0, 0]]
R = len(L)
NL = [[0]*len(L[0]) for _ in range(R)]
for i, r in enumerate(L):
for j, c in enumerate(r):
_r = (c + i) % R
if c > NL[_r][j]:
NL[_r][j] = c
print(NL)

Find Distance to Nearest Zero in NumPy Array

Let's say I have a NumPy array:
x = np.array([0, 1, 2, 0, 4, 5, 6, 7, 0, 0])
At each index, I want to find the distance to nearest zero value. If the position is a zero itself then return zero as a distance. Afterward, we are only interested in distances to the nearest zero that is to the right of the current position. The super naive approach would be something like:
out = np.full(x.shape[0], x.shape[0]-1)
for i in range(x.shape[0]):
j = 0
while i + j < x.shape[0]:
if x[i+j] == 0:
break
j += 1
out[i] = j
And the output would be:
array([0, 2, 1, 0, 4, 3, 2, 1, 0, 0])
I'm noticing a countdown/decrement pattern in the output in between the zeros. So, I might be able to do use the locations of the zeros (i.e., zero_indices = np.argwhere(x == 0).flatten())
What is the fastest way to get the desired output in linear time?

Approach #1 : Searchsorted to the rescue for linear-time in a vectorized manner (before numba guys come in)!
mask_z = x==0
idx_z = np.flatnonzero(mask_z)
idx_nz = np.flatnonzero(~mask_z)
# Cover for the case when there's no 0 left to the right
# (for same results as with posted loop-based solution)
if x[-1]!=0:
idx_z = np.r_[idx_z,len(x)]
out = np.zeros(len(x), dtype=int)
idx = np.searchsorted(idx_z, idx_nz)
out[~mask_z] = idx_z[idx] - idx_nz
Approach #2 : Another with some cumsum -
mask_z = x==0
idx_z = np.flatnonzero(mask_z)
# Cover for the case when there's no 0 left to the right
if x[-1]!=0:
idx_z = np.r_[idx_z,len(x)]
out = idx_z[np.r_[False,mask_z[:-1]].cumsum()] - np.arange(len(x))
Alternatively, last step of cumsum could be replaced by repeat functionality -
r = np.r_[idx_z[0]+1,np.diff(idx_z)]
out = np.repeat(idx_z,r)[:len(x)] - np.arange(len(x))
Approach #3 : Another with mostly just cumsum -
mask_z = x==0
idx_z = np.flatnonzero(mask_z)
pp = np.full(len(x), -1)
pp[idx_z[:-1]] = np.diff(idx_z) - 1
if idx_z[0]==0:
pp[0] = idx_z[1]
else:
pp[0] = idx_z[0]
out = pp.cumsum()
# Handle boundary case and assigns 0s at original 0s places
out[idx_z[-1]:] = np.arange(len(x)-idx_z[-1],0,-1)
out[mask_z] = 0

You could work from the other side. Keep a counter on how many non zero digits have passed and assign it to the element in the array. If you see 0, reset the counter to 0
Edit: if there is no zero on the right, then you need another check
x = np.array([0, 1, 2, 0, 4, 5, 6, 7, 0, 0])
out = x
count = 0
hasZero = False
for i in range(x.shape[0]-1,-1,-1):
if out[i] != 0:
if not hasZero:
out[i] = x.shape[0]-1
else:
count += 1
out[i] = count
else:
hasZero = True
count = 0
print(out)

You can use the difference between the indices of each position and the cumulative max of zero positions to determine the distance to the preceding zero. This can be done forward and backward. The minimum between forward and backward distance to the preceding (or next) zero will be the nearest:
import numpy as np
indices = np.arange(x.size)
zeroes = x==0
forward = indices - np.maximum.accumulate(indices*zeroes) # forward distance
forward[np.cumsum(zeroes)==0] = x.size-1 # handle absence of zero from edge
forward = forward * (x!=0) # set zero positions to zero
zeroes = zeroes[::-1]
backward = indices - np.maximum.accumulate(indices*zeroes) # backward distance
backward[np.cumsum(zeroes)==0] = x.size-1 # handle absence of zero from edge
backward = backward[::-1] * (x!=0) # set zero positions to zero
distZero = np.minimum(forward,backward) # closest distance (minimum)
results:
distZero
# [0, 1, 1, 0, 1, 2, 2, 1, 0, 0]
forward
# [0, 1, 2, 0, 1, 2, 3, 4, 0, 0]
backward
# [0, 2, 1, 0, 4, 3, 2, 1, 0, 0]
Special case where no zeroes are present on outer edges:
x = np.array([3, 1, 2, 0, 4, 5, 6, 0,8,8])
forward: [9 9 9 0 1 2 3 0 1 2]
backward: [3 2 1 0 3 2 1 0 9 9]
distZero: [3 2 1 0 1 2 1 0 1 2]
also works with no zeroes at all
[EDIT] non-numpy solutions ...
if you're looking for an O(N) solution that doesn't require numpy, you can apply this strategy using the accumulate function from itertools:
x = [0, 1, 2, 0, 4, 5, 6, 7, 0, 0]
from itertools import accumulate
maxDist = len(x) - 1
zeroes = [maxDist*(v!=0) for v in x]
forward = [*accumulate(zeroes,lambda d,v:min(maxDist,(d+1)*(v!=0)))]
backward = accumulate(zeroes[::-1],lambda d,v:min(maxDist,(d+1)*(v!=0)))
backward = [*backward][::-1]
distZero = [min(f,b) for f,b in zip(forward,backward)]
print("x",x)
print("f",forward)
print("b",backward)
print("d",distZero)
output:
x [0, 1, 2, 0, 4, 5, 6, 7, 0, 0]
f [0, 1, 2, 0, 1, 2, 3, 4, 0, 0]
b [0, 2, 1, 0, 4, 3, 2, 1, 0, 0]
d [0, 1, 1, 0, 1, 2, 2, 1, 0, 0]
If you don't want to use any library, you can accumulate the distances manually in a loop:
x = [0, 1, 2, 0, 4, 5, 6, 7, 0, 0]
forward,backward = [],[]
fDist = bDist = maxDist = len(x)-1
for f,b in zip(x,reversed(x)):
fDist = min(maxDist,(fDist+1)*(f!=0))
forward.append(fDist)
bDist = min(maxDist,(bDist+1)*(b!=0))
backward.append(bDist)
backward = backward[::-1]
distZero = [min(f,b) for f,b in zip(forward,backward)]
print("x",x)
print("f",forward)
print("b",backward)
print("d",distZero)
output:
x [0, 1, 2, 0, 4, 5, 6, 7, 0, 0]
f [0, 1, 2, 0, 1, 2, 3, 4, 0, 0]
b [0, 2, 1, 0, 4, 3, 2, 1, 0, 0]
d [0, 1, 1, 0, 1, 2, 2, 1, 0, 0]

My first intuition would be to use slicing. If x can be a normal list instead of a numpy array, then you could use
out = [x[i:].index(0) for i,_ in enumerate(x)]
if numpy is necessary then you can use
out = [np.where(x[i:]==0)[0][0] for i,_ in enumerate(x)]
but this is less efficient because you are finding all zero locations to the right of the value and then pulling out just the first. Almost definitely a better way to do this in numpy.

Edit: I am sorry, I misunderstood. This will give you the distance to the nearest zeros - may it be at left or right. But you can use d_right as intermediate result. This does not cover the edge case of not having any zero to the right though.
import numpy as np
x = np.array([0, 1, 2, 0, 4, 5, 6, 7, 0, 0])
# Get the distance to the closest zero from the left:
zeros = x == 0
zero_locations = np.argwhere(x == 0).flatten()
zero_distances = np.diff(np.insert(zero_locations, 0, 0))
temp = x.copy()
temp[~zeros] = 1
temp[zeros] = -(zero_distances-1)
d_left = np.cumsum(temp) - 1
# Get the distance to the closest zero from the right:
zeros = x[::-1] == 0
zero_locations = np.argwhere(x[::-1] == 0).flatten()
zero_distances = np.diff(np.insert(zero_locations, 0, 0))
temp = x.copy()
temp[~zeros] = 1
temp[zeros] = -(zero_distances-1)
d_right = np.cumsum(temp) - 1
d_right = d_right[::-1]
# Get the smallest distance from both sides:
smallest_distances = np.min(np.stack([d_left, d_right]), axis=0)
# np.array([0, 1, 1, 0, 1, 2, 2, 1, 0, 0])

How to filter Python list while keeping filtered values zero

With
input = [0,0,5,9,0,4,10,3,0]
as list
I need an output, which is going to be two highest values in input while setting other list elements to zero.
output = [0,0,0,9,0,0,10,0,0]
The closest I got:
from itertools import compress
import numpy as np
import operator
input= [0,0,5,9,0,4,10,3,0]
top_2_idx = np.argsort(test)[-2:]
test[top_2_idx[0]]
test[top_2_idx[1]]
Can you please help?

You can sort, find the two largest values, and then use a list comprehension:
input = [0,0,5,9,0,4,10,3,0]
*_, c1, c2 = sorted(input)
result = [0 if i not in {c1, c2} else i for i in input]
Output:
[0, 0, 0, 9, 0, 0, 10, 0, 0]

Not as pretty as Ajax's solution but a O(n) solution and a little more dynamic:
from collections import deque
def zero_non_max(lst, keep_top_n):
"""
Returns a list with all numbers zeroed out
except the keep_top_n.
>>> zero_non_max([0, 0, 5, 9, 0, 4, 10, 3, 0], 3)
>>> [0, 0, 5, 9, 0, 0, 10, 0, 0]
"""
lst = lst.copy()
top_n = deque(maxlen=keep_top_n)
for index, x in enumerate(lst):
if len(top_n) < top_n.maxlen or x > top_n[-1][0]:
top_n.append((x, index))
lst[index] = 0
for val, index in top_n:
lst[index] = val
return lst
lst = [0, 0, 5, 9, 0, 4, 10, 3, 0]
print(zero_non_max(lst, 2))
Output:
[0, 0, 0, 9, 0, 0, 10, 0, 0]

Pure numpy approach:
import numpy as np
arr = np.array([0, 0, 5, 9, 0, 4, 10, 3, 0])
top_2_idx = np.argsort(arr)[-2:]
np.put(arr, np.argwhere(~np.isin(arr, arr[top_2_idx])), 0)
print(arr)
The output:
[ 0 0 0 9 0 0 10 0 0]
Numpy.put

It's possible to achieve this with a single list traversal, making the algorithm O(n):
First find the two highest values with a single traversal;
Then create a list of zeros and add in the found maxima.
Code
def two_max(lst):
# Find two highest values in a single traversal
max_i, max_j = 0, 1
for i in range(len(lst)):
_, max_i, max_j = sorted((max_i, max_j, i), key=lst.__getitem__)
# Make a new list with zeros and replace both maxima
new_lst = [0] * len(lst)
new_lst[max_i], new_lst[max_j] = lst[max_i], lst[max_j]
return new_lst
lst = [0, 0, 5, 9, 0, 4, 10, 3, 0]
print(two_max(lst)) # [0, 0, 0, 9, 0, 0, 10, 0, 0]
Note that if the maximum value in the list appears more than twice, only the two left-most values will appear.
As a sidenote, do not use names such as input in your code as this overshadows the built-in function of the same name.

Here is another numpy-based solution that avoids sorting the entire array, which takes O(nlogn) time.
import numpy as np
arr = np.array([0,0,5,9,0,4,10,3,0])
arr[np.argpartition(arr,-2)[:-2]] = 0
If you want to create a new array as output:
result = np.zeros_like(arr)
idx = np.argpartition(arr,-2)[-2:]
result[idx] = arr[idx]
A corresponding Python-native solution is to use heap.nlargest, which also avoids sorting the entire array.
import heapq
arr = [0,0,5,9,0,4,10,3,0]
l = len(arr)
idx1, idx2 = heapq.nlargest(2, range(l), key=arr.__getitem__)
result = [0] * l
result[idx1] = arr[idx1]
result[idx2] = arr[idx2]

Generate unique binary permutations in python

Please, how can I get all these binary permutations, but without repetition in Python?
a = list(itertools.permutations([1, 1, 0, 0]))
for i in range(len(a)):
print a[i]
(1, 1, 0, 0)
(1, 1, 0, 0)
(1, 0, 1, 0)
...
It would be great if it would be roughly efficient since I'll have to do that with a list of even 30 elements like this.

As #Antti said in a comment, this is equivalent to looking for combinations of positions of the input list which determine which bits in the output are 1.
from itertools import combinations
def binary_permutations(lst):
for comb in combinations(range(len(lst)), lst.count(1)):
result = [0] * len(lst)
for i in comb:
result[i] = 1
yield result
for perm in binary_permutations([1, 1, 0, 0]):
print(perm)
Output:
[1, 1, 0, 0]
[1, 0, 1, 0]
[1, 0, 0, 1]
[0, 1, 1, 0]
[0, 1, 0, 1]
[0, 0, 1, 1]

Here's the algorithm from the accepted answer to the generic algorithm question, adapted into Python 3 (should work in Python 2.7+). The function generate(start, n_bits) will generate all n-bit integers starting from start lexicographically.
def generate(start, n_bits):
# no ones to permute...
if start == 0:
yield 0
return
# fastest count of 1s in the input value!!
n_ones = bin(start).count('1')
# the minimum value to wrap to when maxv is reached;
# all ones in LSB positions
minv = 2 ** n_ones - 1
# this one is just min value shifted left by number of zeroes
maxv = minv << (n_bits - n_ones)
# initialize the iteration value
v = start
while True:
yield v
# the bit permutation doesn't wrap after maxv by itself, so,
if v == maxv:
v = minv
else:
t = ((v | ((v - 1))) + 1)
v = t | (((((t & -t)) // ((v & -v))) >> 1) - 1)
# full circle yet?
if v == start:
break
for i in generate(12, 4):
print('{:04b}'.format(i))
Prints
1100
0011
0101
0110
1001
1010
If list output is generated, this can then be decorated:
def generate_list(start):
n_bits = len(start)
start_int = int(''.join(map(str, start)), 2)
# old and new-style formatting in one
binarifier = ('{:0%db}' % n_bits).format
for i in generate(start_int, n_bits):
yield [int(j) for j in binarifier(i)]
for i in generate_list([1, 1, 0, 0]):
print(i)
prints
[1, 1, 0, 0]
[0, 0, 1, 1]
[0, 1, 0, 1]
[0, 1, 1, 0]
[1, 0, 0, 1]
[1, 0, 1, 0]
What is nice about this algorithm is that you can resume it at any point. If you find a way to calculate good starting points, it is possible to parallelize too. And the numbers should be more compact than lists, so you could use them if possible.

What you are trying to do is choose two positions at which the element will be 1.
Code
from itertools import combinations
def bit_patterns(size, ones):
for pos in map(set, combinations(range(size), ones)):
yield [int(i in pos) for i in range(size)]
Output
>>> print(*bit_patterns(4, 2), sep='\n')
[1, 1, 0, 0]
[1, 0, 1, 0]
[1, 0, 0, 1]
[0, 1, 1, 0]
[0, 1, 0, 1]
[0, 0, 1, 1]
Alternative
A fun alternative is to see the desired output as the binary representations which have only two ones. We can use this definition to get the output you want.
from itertools import combinations
def bit_patterns(size, ones):
for t in combinations([1 << i for i in range(size)], ones):
yield [int(n) for n in f'{sum(t):0{size}b}']

Here is a recursive solution:
def bin_combs_iter(ones, zeros):
if not zeros:
yield [1] * ones
elif not ones:
yield [0] * zeros
else:
for x in bin_combs_iter(ones - 1, zeros):
x.append(1)
yield x
for x in bin_combs_iter(ones, zeros - 1):
x.append(0)
yield x
def bin_combs(ones, zeros):
return list(bin_combs_iter(ones, zeros))

counting 2*2 squares in n*n binary matrix

I've got an n*n binary matrix (only 1 and 0), how can I go about counting 2*2 squares (squares are made by 1)
for example A=[[1,1],[1,1]] is considered to make one 2*2 square. or
A = [[1, 1, 0, 1],
[1, 1, 1, 1],
[1, 1, 1, 0],
[0, 1, 1, 1]]
is considered to make four 2*2 squares.
here's my code for this , but I just don't know why it doesn't work.
A = [[1, 1, 0, 1] , [1, 1, 1, 1], [1, 1, 1, 0], [0, 1, 1, 1]]
result=[]
for x in range(len(A)-1):
for y in range(len(A)-1):
if A[x][y]==1:
if A[x+1][y]==1:
if A[x][y+1]==1 or A[x][y-1]==1 and A[x+1][y] or A[x+1][y-1]==1:
result.append(1)
if A[x-1][y]==1:
if A[x][y+1]==1 or A[x][y-1]==1 and A[x-1][y] or A[x-1][y-1]==1:
result.append(1)
print(len(result))
`

Generate indices for width - 1 by height - 1; itertools.product() can do this for us.
Test 4 coordinates for each generated index using all() to only test as many as needed to disprove a square exists.
Use sum() with a generator to count the number of squares found; faster than manually counting with a list or a counter.
Together with lambda to test for squares, this then becomes:
from itertools import product
def count_squares(A):
width, height = len(A[0]), len(A)
indices = product(range(width - 1), range(height - 1))
is_square = lambda x, y: all(A[a][b] == 1 for a, b in product((x, x + 1), (y, y + 1)))
return sum(1 for x, y in indices if is_square(x, y))
Demo:
>>> from itertools import product
>>> count_squares([[1,1],[1,1]])
>>> def count_squares(A):
... width, height = len(A[0]), len(A)
... indices = product(range(width - 1), range(height - 1))
... is_square = lambda x, y: all(A[a][b] == 1 for a, b in product((x, x + 1), (y, y + 1)))
... return sum(1 for x, y in indices if is_square(x, y))
...
>>> count_squares([[1,1],[1,1]])
1
>>> count_squares([[1, 1, 0, 1] , [1, 1, 1, 1], [1, 1, 1, 0], [0, 1, 1, 1]])
4

To get the column count use len(A[x]) so
for y in range(len(A)-1)
becomes
for y in range(len(A[x])-1)
Change
if A[x][y]==1:
if A[x+1][y]==1:
if A[x][y+1]==1 or A[x][y-1]==1 and A[x+1][y] or A[x+1][y-1]==1:
result.append(1)
if A[x-1][y]==1:
if A[x][y+1]==1 or A[x][y-1]==1 and A[x-1][y] or A[x-1][y-1]==1:
result.append(1)
To
if A[x][y]==1 and A[x+1][y]==1 and a[x+1][y+1]==1 and a[x][y+1]:
result.append(1)
Unless you want to count squares multiple times.

Using scipy.signal there is a simple solution that finds the correlation between your target and the input. This is nice since it generalizes to "almost matches" and arbitrary shapes!
import numpy as np
from scipy import signal
A = np.array([[1,1,0,1] ,[1,1,1,1],[1,1,1,0],[0,1,1,1]],dtype=int)
b = np.ones((2,2),dtype=int)
c = signal.correlate(A, b, 'valid')
idx = np.where(c==4)
count = sum(idx[0])
print count
This gives 4 as expected. If you find this interesting, there is a (longer) answer that uses this same idea:
Finding matching submatrices inside a matrix

I multipliply the values of every 2*2-submatrix and sum up:
A = [[1, 1, 0, 1],
[1, 1, 1, 1],
[1, 1, 1, 0],
[0, 1, 1, 1]]
sum( A[x][y]*A[x+1][y]*A[x][y+1]*A[x+1][y+1]
for y in range(len(A)-1)
for x in range(len(A[y])-1)
)
Out[79]: 4