We Keep Coding

Vectorised index of arrays

Originally I had something like this:
a = 1 # Some randomly generated positive integer
b = -1 # Some randomly generated negative integer
c = 0 # Constant 0
i = 0 # Randomly picked from (0, 1, 2)
d = [a, b, c][i]
I would like to vectorise this so that many samples can be generated
So I have three arrays of length N, an index array of length N, and would like to use that index array to pick one of the three arrays
a = np.array([1, 2, 3, 4])
b = np.array([-1, -2, -3, -4])
c = np.array([0, 0, 0, 0])
i = np.array([2, 1, 2, 0])
d = np.array([a, b, c])[i] # Doesn't work
# Would like the result:
d = np.array([0, -2, 0, 4])
d = a * (i == 0) + b * (i == 1) + c * (i == 2) works, but surely there is a way that looks more like the unvectorised code

Make a 2-d array from the three arrays then use Integer indexing
>>> e = np.vstack([a,b,c])
>>> i = np.array([2, 1, 2, 0])
>>> e[(i,np.arange(i.shape[0]))]
array([ 0, -2, 0, 4])
>>>

Notice that your answer is on the diagonal of
np.array([a, b, c])[i]
so you can go:
np.array([a, b, c])[i].diagonal()

How can I create or change polynomial coefficients

I have this code and I want to edit it to do something else:
def pol(poly, n, x):
result = poly[0]
#Using Horner's method
for i in range(1, n):
result = result * x + poly[i]
return result
#Let us evaluate value of
#ax^3 - bx^2 - x - 10 for x = 1
poly = [5, 9, -1, -10]
x = 1
n = len(poly)
print("Value of polynomial is: ", pol(poly, n, x))
I wonder how can I can change the coefficients of the polynomial. And this code just calculates:
x^3 and x^2
How can I make this code calculate for example this polynomial:
p(x) = 5x^10 + 9x - 7x - 10
or any polynomial in Python?

Your code should work, you just need to present the correct input. For
p(x) = 5x^10 + 9x - 7x - 10
you should provide:
poly2 = [5, 0, 0, 0, 0, 0, 0, 0, 0, 9-7, 10]
Alternate pol - implementation:
def pol(poly, x):
n = len(poly) # no need to provide it at call
rp = poly[::-1] # [-10, -1, 9, 5] so they correlate with range(n) as power
print("Poly:",poly, "for x =",x)
result = 0
for i in range(n):
val = rp[i] * x**i
print(rp[i],' * x^', i, ' = ', val, sep='') # debug output
result += val
return result
x = 2 # 1 is a bad test candidate - no differences for 1**10 vs 1**2
# 5x^3 + 9x^2 - x - 10 for x = 1
poly = [5, 9, -1, -10]
print("Value of polynomial is: ", pol(poly, x))
# p(x) = 5x^10 + 9x - 7x - 10
poly2 = [5, 0, 0, 0, 0, 0, 0, 0, 0, 9-7, 10]
print("Value of polynomial is: ", pol(poly2, x))
Output:
Poly: [5, 9, -1, -10] for x = 2
-10 * x^0 = -10
-1 * x^1 = -2
9 * x^2 = 36
5 * x^3 = 40
Value of polynomial is: 64
Poly: [5, 0, 0, 0, 0, 0, 0, 0, 0, 2, -10] for x = 2
-10 * x^0 = -10
2 * x^1 = 4
0 * x^2 = 0
0 * x^3 = 0
0 * x^4 = 0
0 * x^5 = 0
0 * x^6 = 0
0 * x^7 = 0
0 * x^8 = 0
0 * x^9 = 0
5 * x^10 = 5120
Value of polynomial is: 5114

Replace all -1 in numpy array with values from another array

I have two numpy arrays, e. g.:
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
I would like to replace all -1 in x with numbers in y, that are not in x. First -1 with first number in y that is not in x (3), second -1 with second number in y that is not in x (5), ... So final x should be:
[4 3 1 5 6 2]
I created this function:
import numpy as np
import time
start = time.time()
def fill(x, y):
x_i = 0
y_i = 0
while x_i < len(x) and y_i < len(y):
if x[x_i] != -1: # If current value in x is not -1.
x_i += 1
continue
if y[y_i] in x: # If current value in y is already in x.
y_i += 1
continue
x[x_i] = y[y_i] # Replace current -1 in x by current value in y.
for i in range(10000):
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
fill(x, y)
end = time.time()
print(end - start) # 0.296
It's working, but I need run this function many times (e. g. milion times), so I would like to optimize it. Is there any way?

You could do:
import numpy as np
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
# create set of choices
sx = set(x)
choices = np.array([yi for yi in y if yi not in sx])
# assign new values
x[x == -1] = choices[:(x == -1).sum()]
print(x)
Output
[4 3 1 5 6 2]

y_not_in_x = np.setdiff1d(y, x)
x_eq_neg1 = x == -1
n_neg1s = np.sum(x_eq_neg1)
x[x_eq_neg1] = y_not_in_x[:n_neg1s]

Finding biggest negative submatrix in python

I want find the biggest submatrix which contains only negative numbers in a matrix, for example:
In
[[1, -9, -2, 8, 6, 1],
[8, -1,-11, -7, 6, 4],
[10, 12, -1, -9, -12, 14],
[8, 10, -3, -5, 17, 8],
[6, 4, 10, -13, -16, 19]]
the biggest submatrix containing only negative numbers is
[[-11, -7],
[-1, -9],
[-3,-5]]
(left upper corner coordinates: 1,2, right lower corner coordinates: 3,3).
What's the most effective way to do it?

A brute force solution. Will work, but may be considered too slow for a bigger matrix:
mOrig = [[1, -9, -2, 8, 6, 1],
[8, -1,-11, -7, 6, 4],
[10, 12, -1, -9, -12, 14],
[8, 10, -3, -5, 17, 8],
[6, 4, 10, -13, -16, 19]]
# reduce the problem
# now we have a matrix that contains only 0 and 1
# at the place where there was a negative number
# there is now a 1 and at the places where a positive
# number had been there is now a 0. 0s are considered
# to be negative numbers, if you want to change this,
# change the x < 0 to x <= 0.
m = [[1 if x < 0 else 0 for x in z] for z in mOrig]
# now we have the problem to find the biggest submatrix
# consisting only 1s.
# first a function that checks if a submatrix only contains 1s
def containsOnly1s(m, x1, y1, x2, y2):
for i in range(x1, x2):
for j in range(y1, y2):
if m[i][j] == 0:
return False
return True
def calculateSize(x1, y1, x2, y2):
return (x2 - x1) * (y2 - y1)
best = (-1, -1, -1, -1, -1)
for x1 in range(len(m)):
for y1 in range(len(m[0])):
for x2 in range(x1, len(m)):
for y2 in range(y1, len(m[0])):
if containsOnly1s(m, x1, y1, x2, y2):
sizeOfSolution = calculateSize(x1, y1, x2, y2)
if best[4] < sizeOfSolution:
best = (x1, y1, x2, y2, sizeOfSolution)
for x in range(best[0], best[2]):
print("\t".join([str(mOrig[x][y]) for y in range(best[1], best[3])]))
Will output
-11 -7
-1 -9
-3 -5
In case something else is meant with "biggest submatrix", the only function that needs to get changed is the following:
def calculateSize(x1, y1, x2, y2):
return (x2 - x1) * (y2 - y1)
which is calculating the size of a submatrix.
Edit 1 ... first speedup
best = (-1, -1, -1, -1, -1)
for x1 in range(len(m)):
for y1 in range(len(m[0])):
if m[x1][y1] == 1: # The starting point must contain a 1
for x2 in range(x1 + 1, len(m)): # actually can start with x1 + 1 here
for y2 in range(y1 + 1, len(m[0])):
if containsOnly1s(m, x1, y1, x2, y2):
sizeOfSolution = calculateSize(x1, y1, x2, y2)
if best[4] < sizeOfSolution:
best = (x1, y1, x2, y2, sizeOfSolution)
else:
# There is at least one 0 in the matrix, so every greater
# matrix will also contain this 0
break
Edit 2
Ok, after converting the matrix into a matrix of 0 and 1 (as I do via the line m = [[1 if x < 0 else 0 for x in z] for z in mOrig] the problem is the same as what is called the maximal rectangle problem in literature. So I googled a bit about known algorithms for this kind of problem and came across this site here http://www.drdobbs.com/database/the-maximal-rectangle-problem/184410529 which is describing a very fast algorithm to solve this kind of problem. To summarize the points of this website, the algorithm is exploiting the structure. This can be done by using a stack in order to remember the structure profile which allows us to recalculate the width in case a narrow rectangle gets reused when an wider one gets closed.

Here is my pretty fast solution using convolutions from OpenCV. Usage of float32 is required as it is much faster than integers. Takes 135 ms for 1000 x 1000 matrix on my 2 cores. There's space for further code optimization though.
import cv2
import numpy as np
data = """1 -9 -2 8 6 1
8 -1 -11 -7 6 4
10 12 -1 -9 -12 14
8 10 -3 -5 17 8
6 4 10 -13 -16 19"""
# matrix = np.random.randint(-128, 128, (1000, 1000), dtype=np.int32)
matrix = np.int32([line.split() for line in data.splitlines()])
def find_max_kernel(matrix, border=cv2.BORDER_ISOLATED):
max_area = 0
mask = np.float32(matrix < 0)
ones = np.ones_like(mask)
conv_x = np.zeros_like(mask)
conv_y = np.zeros_like(mask)
max_h, max_w = mask.shape
for h in range(1, max_h + 1):
cv2.filter2D(mask, -1, ones[:h, None, 0], conv_y, (0, 0), 0, border)
for w in range(1, max_w + 1):
area = h * w
if area > max_area:
cv2.filter2D(conv_y, -1, ones[None, 0, :w], conv_x, (0, 0), 0, border)
if conv_x.max() == area:
max_area, shape = area, (h, w)
else:
if w == 1:
max_h = h - 1
if h == 1:
max_w = w - 1
break
if h >= max_h:
break
cv2.filter2D(mask, -1, np.ones(shape, np.float32), conv_x, (0, 0), 0, border)
p1 = np.array(np.unravel_index(conv_x.argmax(), conv_x.shape))
p2 = p1 + shape - 1
return p1, p2
print(*find_max_kernel(matrix), sep='\n')

Below is a function that executes in much less than a second for a 5000x5000 matrix. It relies only on basic np-functions.
It could be improved by returning the first index instead of all indices. Several other optimizations can be done but for many uses it is fast enough.
from numpy import roll, where
def gidx(X):
Wl = X & roll(X, 1, axis=1)
T = X & Wl & roll(X, -1, axis=1)
if T[1:-1][1:-1].any():
N = T & roll(T, -1, axis=0) & roll(T, 1, axis=0)
if N.any(): return gidx(N)
W = Wl & roll(Wl, 1, axis=0)
if W.any(): return where(W)
return where(X)
#%% Example
import numpy as np
#np.random.seed(0)
M = 100
X = np.random.randn(M, M) - 2
X0 = (X < 0)
X0[[0, -1]], X0[:, [0, -1]] = False, False
jx, kx = gidx(X0)

find weight matrix for Temporal Associative Memories

I am finding weight matrix for Temporal Associative Memories my vectors are
x1 = [ 1 –1 1 –1 1 –1 1 –1 ]
x2 = [ 1 1 –1 –1 1 1 –1 –1 ]
x3 = [ 1 1 1 1 –1 –1 –1 –1 ]
and equation to find weight matrix is W = T(X1)*X2 + T(X2)*X3 + T(X3)*X1 where T is transpose
x1 = [1, -1, 1, -1, 1, -1, 1, -1]
x2 = [1, 1, -1, -1, 1, 1, -1, -1]
x3 = [1, 1, 1, 1, -1, -1, -1, -1]
x = pd.DataFrame(data=[x1, x2, x3]).T
w = np.zeros(np.max(x.shape), np.max(x.shape))
for n in range(x.shape[0] - 1):
w1 = np.dot(x[n], x[n + 1])
w += w1
w += np.dot([-1], [1])
print(w)

First of all, there are brackets missing in initialization of w, it should be w = np.zeros((np.max(x.shape), np.max(x.shape))) to get proper shape.
Second, you want to iterate over columns of dataframe, so you need to use
for n in range(x.shape[1]):
Note that this will give you n values [0,1,2] not [1,2,3] (as in the book you're referring to), but this is ok since column names in pandas are by default numbered from zero as well.
In the last iteration this will give you n+1=3 and you want to address first column (indexed with 0) instead. This can be corrected with if statement.
When it comes to vector multiplication - np.dot, which you are using, performs inner product. It will treat the first vector as horizontal and second as vertical and then perform matrix multiplication. Instead, you should use np.outer, to get reversed alignment of vectors.
Last thing - I do not know what the last line inside the loop is doing
w += np.dot([-1], [1])
this does not seem to be needed (based on the book you're referring to)
Putting this all together gives following code:
x1 = [1, -1, 1, -1, 1, -1, 1, -1]
x2 = [1, 1, -1, -1, 1, 1, -1, -1]
x3 = [1, 1, 1, 1, -1, -1, -1, -1]
x = pd.DataFrame(data=[x1, x2, x3]).T
w = np.zeros((np.max(x.shape), np.max(x.shape)))
for n in range(x.shape[1]):
next_idx = n+1
if next_idx == x.shape[1]:
next_idx = 0
w1 = np.outer(x[n], x[next_idx])
w += w1
print(w)