Related
I want to implement the Gradient Descent Algorithm on this simple data but I am facing problems. It would be great if someone points me in the right direction. The answer should be 7 for x=6 but I'm not getting there.
X = [1, 2, 3, 4]
Y = [2, 3, 4, 5]
m_gradient = 0
b_gradient = 0
m, b = 0, 0
learning_rate = 0.1
N = len(Y)
for p in range(100):
for idx in range(len(Y)):
x = X[idx]
y = Y[idx]
hyp = (m * x) + b
m_gradient += -(2/N) * x * (y - hyp)
b_gradient += -(2/N) * (y - hyp)
m = m - (m_gradient * learning_rate)
b = b - (b_gradient * learning_rate)
print(b+m*6)
You are calculating the gradients incorrectly for all but the first iteration. You need to set both gradients to 0 in the outer for loop.
X = [1, 2, 3, 4]
Y = [2, 3, 4, 5]
m_gradient = 0
b_gradient = 0
m, b = 0, 0
learning_rate = 0.1
N = len(Y)
for p in range(100):
for idx in range(len(Y)):
x = X[idx]
y = Y[idx]
hyp = (m * x) + b
m_gradient += -(2/N) * x * (y - hyp)
b_gradient += -(2/N) * (y - hyp)
m = m - (m_gradient * learning_rate)
b = b - (b_gradient * learning_rate)
m_gradient, b_gradient = 0, 0
print(b+m*6)
For example consider b_gradient. Before first iteration b_gradient = 0 and is calculated as 0 + -0.5*(y0 - (m*x0 +b)) + -0.5(y1 - (m*x1 +b)) + -0.5(y2 - (m*x2 + b)) + -0.5(y3 - (m*x3 + b)), where x0 and y0 are X[0] and Y[0], respectively.
After the first iteration the value of b_gradient is -7, this is correct.
The problem starts with the second iteration. Instead of calculating b_gradient as the sum of (-0.5(yn - (m*xn + b)) for 0 <= n <= 3, you calculated it as the previous value of b_gradient plus the sum of (-0.5(yn - (m*xn + b)) for 0 <= n <= 3.
After the second iteration the value of b_gradient is -2.6, this is incorrect. The correct value is 4.4, note that 4.4 - 7 = -2.6.
It seems you want coefficients for Linear Regression using Gradient Descent. Some more data points, a slightly smaller learning rate, training for more epochs by looking at the loss will help reduce error.
As input size gets larger the code below will give slightly off results. The above mentioned methods such as training for more epoch will give correct results for larger range of numbers.
Vectorized Version
import numpy as np
X = np.array([1, 2, 3, 4, 5, 6, 7])
Y = np.array([2, 3, 4, 5, 6, 7, 8])
w_gradient = 0
b_gradient = 0
w, b = 0.5, 0.5
learning_rate = .01
loss = 0
EPOCHS = 2000
N = len(Y)
for i in range(EPOCHS):
# Predict
Y_pred = (w * X) + b
# Loss
loss = np.square(Y_pred - Y).sum() / (2.0 * N)
if i % 100 == 0:
print(loss)
# Backprop
grad_y_pred = (2 / N) * (Y_pred - Y)
w_gradient = (grad_y_pred * X).sum()
b_gradient = (grad_y_pred).sum()
# Optimize
w -= (w_gradient * learning_rate)
b -= (b_gradient * learning_rate)
print("\n\n")
print("LEARNED:")
print(w, b)
print("\n")
print("TEST:")
print(np.round(b + w * (-2)))
print(np.round(b + w * 0))
print(np.round(b + w * 1))
print(np.round(b + w * 6))
print(np.round(b + w * 3000))
# Expected: 30001, but gives 30002.
# Training for 3000 epochs will give expected result.
# For simple demo with less training data and small input range 2000 in enough
print(np.round(b + w * 30000))
Output
LEARNED:
1.0000349103409163 0.9998271260509328
TEST:
-1.0
1.0
2.0
7.0
3001.0
30002.0
Loop Version
import numpy as np
X = np.array([1, 2, 3, 4, 5, 6, 7])
Y = np.array([2, 3, 4, 5, 6, 7, 8])
w_gradient = 0
b_gradient = 0
w, b = 0.5, 0.5
learning_rate = .01
loss = 0
EPOCHS = 2000
N = len(Y)
for i in range(EPOCHS):
w_gradient = 0
b_gradient = 0
loss = 0
for j in range(N):
# Predict
Y_pred = (w * X[j]) + b
# Loss
loss += np.square(Y_pred - Y[j]) / (2.0 * N)
# Backprop
grad_y_pred = (2 / N) * (Y_pred - Y[j])
w_gradient += (grad_y_pred * X[j])
b_gradient += (grad_y_pred)
# Optimize
w -= (w_gradient * learning_rate)
b -= (b_gradient * learning_rate)
# Print loss
if i % 100 == 0:
print(loss)
print("\n\n")
print("LEARNED:")
print(w, b)
print("\n")
print("TEST:")
print(np.round(b + w * (-2)))
print(np.round(b + w * 0))
print(np.round(b + w * 1))
print(np.round(b + w * 6))
print(np.round(b + w * 3000))
# Expected: 30001, but gives 30002.
# Training for 3000 epochs will give expected result.
# For simple demo with less training data and small input range 2000 in enough
print(np.round(b + w * 30000))
Output
LEARNED:
1.0000349103409163 0.9998271260509328
TEST:
-1.0
1.0
2.0
7.0
3001.0
30002.0
I have this code and I want to edit it to do something else:
def pol(poly, n, x):
result = poly[0]
#Using Horner's method
for i in range(1, n):
result = result * x + poly[i]
return result
#Let us evaluate value of
#ax^3 - bx^2 - x - 10 for x = 1
poly = [5, 9, -1, -10]
x = 1
n = len(poly)
print("Value of polynomial is: ", pol(poly, n, x))
I wonder how can I can change the coefficients of the polynomial. And this code just calculates:
x^3 and x^2
How can I make this code calculate for example this polynomial:
p(x) = 5x^10 + 9x - 7x - 10
or any polynomial in Python?
Your code should work, you just need to present the correct input. For
p(x) = 5x^10 + 9x - 7x - 10
you should provide:
poly2 = [5, 0, 0, 0, 0, 0, 0, 0, 0, 9-7, 10]
Alternate pol - implementation:
def pol(poly, x):
n = len(poly) # no need to provide it at call
rp = poly[::-1] # [-10, -1, 9, 5] so they correlate with range(n) as power
print("Poly:",poly, "for x =",x)
result = 0
for i in range(n):
val = rp[i] * x**i
print(rp[i],' * x^', i, ' = ', val, sep='') # debug output
result += val
return result
x = 2 # 1 is a bad test candidate - no differences for 1**10 vs 1**2
# 5x^3 + 9x^2 - x - 10 for x = 1
poly = [5, 9, -1, -10]
print("Value of polynomial is: ", pol(poly, x))
# p(x) = 5x^10 + 9x - 7x - 10
poly2 = [5, 0, 0, 0, 0, 0, 0, 0, 0, 9-7, 10]
print("Value of polynomial is: ", pol(poly2, x))
Output:
Poly: [5, 9, -1, -10] for x = 2
-10 * x^0 = -10
-1 * x^1 = -2
9 * x^2 = 36
5 * x^3 = 40
Value of polynomial is: 64
Poly: [5, 0, 0, 0, 0, 0, 0, 0, 0, 2, -10] for x = 2
-10 * x^0 = -10
2 * x^1 = 4
0 * x^2 = 0
0 * x^3 = 0
0 * x^4 = 0
0 * x^5 = 0
0 * x^6 = 0
0 * x^7 = 0
0 * x^8 = 0
0 * x^9 = 0
5 * x^10 = 5120
Value of polynomial is: 5114
I have two numpy arrays, e. g.:
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
I would like to replace all -1 in x with numbers in y, that are not in x. First -1 with first number in y that is not in x (3), second -1 with second number in y that is not in x (5), ... So final x should be:
[4 3 1 5 6 2]
I created this function:
import numpy as np
import time
start = time.time()
def fill(x, y):
x_i = 0
y_i = 0
while x_i < len(x) and y_i < len(y):
if x[x_i] != -1: # If current value in x is not -1.
x_i += 1
continue
if y[y_i] in x: # If current value in y is already in x.
y_i += 1
continue
x[x_i] = y[y_i] # Replace current -1 in x by current value in y.
for i in range(10000):
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
fill(x, y)
end = time.time()
print(end - start) # 0.296
It's working, but I need run this function many times (e. g. milion times), so I would like to optimize it. Is there any way?
You could do:
import numpy as np
x = np.array([4, -1, 1, -1, -1, 2])
y = np.array([1, 2, 3, 4, 5, 6])
# create set of choices
sx = set(x)
choices = np.array([yi for yi in y if yi not in sx])
# assign new values
x[x == -1] = choices[:(x == -1).sum()]
print(x)
Output
[4 3 1 5 6 2]
y_not_in_x = np.setdiff1d(y, x)
x_eq_neg1 = x == -1
n_neg1s = np.sum(x_eq_neg1)
x[x_eq_neg1] = y_not_in_x[:n_neg1s]
I want to achieve an algorithm, which allows to return some results according to inputs.
I pass those values ( Changeable, values varies)
b, c, a, li = 200, 30, 3 , [3,3]
into a function with algorithm, and my result should return as a list.
[x1, x2, x3, ...]
x1, xn is just for illustration.
Sample of code:
b, c, a, li = 200, 30, 3 , [1,1]
def func(b,c,a,li):
for index, item in enumerate(li):
if index == 0:
if a %2 == 0:
Result = []
start= 0
for num in range(a):
Result[:0]=[start+num*c]
Result[-1:]=[b-num*c]
else:
Result = []
start= 0
for num in range(a):
Result[:0]=[start+num*c]
Result[-1:]=[b-num*c]
return Result
g = func(b,c,a,li)
print(g)
Output:
[60, 30, 140]
I expect to get an output as:
[0, 100 ,200]
What I want to achieve is as below:
For instance in above example:
A for-loop inside function first checks first value of list which is 1. then place x1 and second value of list which is 1 and place x2, the remaining is 1 , this should be placed as x3, se figur, and xn values are calulated as x1=b-b=0, x2=b, x3= x1 + b/(a-len(li)+1)
Another example with even numbers:
b, c, a, li = 300, 35, 4 , [2,2]
Output:
[ 0, 35, 265, 300]
Another example with odd numbers:
b, c, a, li = 400, 25, 5 , [2,2]
Output:
[ 0, 25, 200, 375, 400]
Another example with some other numbers:
b, c, a, li = 350, 40, 3 , [4,2]
Output:
[ 0, 40, 350]
I really find it difficult to write an algorithm pattern which solves the puzzle. I appreciate much any help. Thanks.
I hope I understood the question right, I'm working only with examples provided in the text.
The main method is get_points(), which accepts parameters b, c, a, li as in question and returns list of computed values:
from itertools import cycle
def get_points(b, c, a, li):
num_points_left, num_points_right = li
final_left = [*range(0, num_points_left * c, c)]
final_right = [*range(b, b - num_points_right * c, -c)][::-1]
final_center = []
points_left = a - (len(final_left) + len(final_right))
if points_left > 0:
step = (final_right[0] - final_left[-1]) // (points_left+1)
final_center = [final_left[-1] + step*i for i in range(1, points_left+1)]
elif points_left < 0:
cycle_list = [lambda: final_left.pop() if len(final_left) > 1 else None,
lambda: final_right.pop(0) if len(final_right) > 1 else None]
if len(final_left) > len(final_right):
remove_cycle = cycle(cycle_list)
else:
remove_cycle = cycle(cycle_list[::-1])
while len(final_left) + len(final_right) > a:
next(remove_cycle)()
return final_left + final_center + final_right
def test(b, c, a, li, expected):
print('Testing parameters:', b, c, a, li)
print('Expected:', expected)
returned = get_points(b, c, a, li)
print('Returned:', returned)
if returned == expected:
print('* PASSED! *')
else:
print('!!! Fail !!!')
test(400, 25, 5 , [2,2], [ 0, 25, 200, 375, 400])
test(200, 30, 3 , [1,1], [0, 100 ,200])
test(300, 35, 4 , [2,2], [ 0, 35, 265, 300])
test(350, 40, 3 , [4,2], [ 0, 40, 350])
Prints:
Testing parameters: 400 25 5 [2, 2]
Expected: [0, 25, 200, 375, 400]
Returned: [0, 25, 200, 375, 400]
* PASSED! *
Testing parameters: 200 30 3 [1, 1]
Expected: [0, 100, 200]
Returned: [0, 100, 200]
* PASSED! *
Testing parameters: 300 35 4 [2, 2]
Expected: [0, 35, 265, 300]
Returned: [0, 35, 265, 300]
* PASSED! *
Testing parameters: 350 40 3 [4, 2]
Expected: [0, 40, 350]
Returned: [0, 40, 350]
* PASSED! *
I want find the biggest submatrix which contains only negative numbers in a matrix, for example:
In
[[1, -9, -2, 8, 6, 1],
[8, -1,-11, -7, 6, 4],
[10, 12, -1, -9, -12, 14],
[8, 10, -3, -5, 17, 8],
[6, 4, 10, -13, -16, 19]]
the biggest submatrix containing only negative numbers is
[[-11, -7],
[-1, -9],
[-3,-5]]
(left upper corner coordinates: 1,2, right lower corner coordinates: 3,3).
What's the most effective way to do it?
A brute force solution. Will work, but may be considered too slow for a bigger matrix:
mOrig = [[1, -9, -2, 8, 6, 1],
[8, -1,-11, -7, 6, 4],
[10, 12, -1, -9, -12, 14],
[8, 10, -3, -5, 17, 8],
[6, 4, 10, -13, -16, 19]]
# reduce the problem
# now we have a matrix that contains only 0 and 1
# at the place where there was a negative number
# there is now a 1 and at the places where a positive
# number had been there is now a 0. 0s are considered
# to be negative numbers, if you want to change this,
# change the x < 0 to x <= 0.
m = [[1 if x < 0 else 0 for x in z] for z in mOrig]
# now we have the problem to find the biggest submatrix
# consisting only 1s.
# first a function that checks if a submatrix only contains 1s
def containsOnly1s(m, x1, y1, x2, y2):
for i in range(x1, x2):
for j in range(y1, y2):
if m[i][j] == 0:
return False
return True
def calculateSize(x1, y1, x2, y2):
return (x2 - x1) * (y2 - y1)
best = (-1, -1, -1, -1, -1)
for x1 in range(len(m)):
for y1 in range(len(m[0])):
for x2 in range(x1, len(m)):
for y2 in range(y1, len(m[0])):
if containsOnly1s(m, x1, y1, x2, y2):
sizeOfSolution = calculateSize(x1, y1, x2, y2)
if best[4] < sizeOfSolution:
best = (x1, y1, x2, y2, sizeOfSolution)
for x in range(best[0], best[2]):
print("\t".join([str(mOrig[x][y]) for y in range(best[1], best[3])]))
Will output
-11 -7
-1 -9
-3 -5
In case something else is meant with "biggest submatrix", the only function that needs to get changed is the following:
def calculateSize(x1, y1, x2, y2):
return (x2 - x1) * (y2 - y1)
which is calculating the size of a submatrix.
Edit 1 ... first speedup
best = (-1, -1, -1, -1, -1)
for x1 in range(len(m)):
for y1 in range(len(m[0])):
if m[x1][y1] == 1: # The starting point must contain a 1
for x2 in range(x1 + 1, len(m)): # actually can start with x1 + 1 here
for y2 in range(y1 + 1, len(m[0])):
if containsOnly1s(m, x1, y1, x2, y2):
sizeOfSolution = calculateSize(x1, y1, x2, y2)
if best[4] < sizeOfSolution:
best = (x1, y1, x2, y2, sizeOfSolution)
else:
# There is at least one 0 in the matrix, so every greater
# matrix will also contain this 0
break
Edit 2
Ok, after converting the matrix into a matrix of 0 and 1 (as I do via the line m = [[1 if x < 0 else 0 for x in z] for z in mOrig] the problem is the same as what is called the maximal rectangle problem in literature. So I googled a bit about known algorithms for this kind of problem and came across this site here http://www.drdobbs.com/database/the-maximal-rectangle-problem/184410529 which is describing a very fast algorithm to solve this kind of problem. To summarize the points of this website, the algorithm is exploiting the structure. This can be done by using a stack in order to remember the structure profile which allows us to recalculate the width in case a narrow rectangle gets reused when an wider one gets closed.
Here is my pretty fast solution using convolutions from OpenCV. Usage of float32 is required as it is much faster than integers. Takes 135 ms for 1000 x 1000 matrix on my 2 cores. There's space for further code optimization though.
import cv2
import numpy as np
data = """1 -9 -2 8 6 1
8 -1 -11 -7 6 4
10 12 -1 -9 -12 14
8 10 -3 -5 17 8
6 4 10 -13 -16 19"""
# matrix = np.random.randint(-128, 128, (1000, 1000), dtype=np.int32)
matrix = np.int32([line.split() for line in data.splitlines()])
def find_max_kernel(matrix, border=cv2.BORDER_ISOLATED):
max_area = 0
mask = np.float32(matrix < 0)
ones = np.ones_like(mask)
conv_x = np.zeros_like(mask)
conv_y = np.zeros_like(mask)
max_h, max_w = mask.shape
for h in range(1, max_h + 1):
cv2.filter2D(mask, -1, ones[:h, None, 0], conv_y, (0, 0), 0, border)
for w in range(1, max_w + 1):
area = h * w
if area > max_area:
cv2.filter2D(conv_y, -1, ones[None, 0, :w], conv_x, (0, 0), 0, border)
if conv_x.max() == area:
max_area, shape = area, (h, w)
else:
if w == 1:
max_h = h - 1
if h == 1:
max_w = w - 1
break
if h >= max_h:
break
cv2.filter2D(mask, -1, np.ones(shape, np.float32), conv_x, (0, 0), 0, border)
p1 = np.array(np.unravel_index(conv_x.argmax(), conv_x.shape))
p2 = p1 + shape - 1
return p1, p2
print(*find_max_kernel(matrix), sep='\n')
Below is a function that executes in much less than a second for a 5000x5000 matrix. It relies only on basic np-functions.
It could be improved by returning the first index instead of all indices. Several other optimizations can be done but for many uses it is fast enough.
from numpy import roll, where
def gidx(X):
Wl = X & roll(X, 1, axis=1)
T = X & Wl & roll(X, -1, axis=1)
if T[1:-1][1:-1].any():
N = T & roll(T, -1, axis=0) & roll(T, 1, axis=0)
if N.any(): return gidx(N)
W = Wl & roll(Wl, 1, axis=0)
if W.any(): return where(W)
return where(X)
#%% Example
import numpy as np
#np.random.seed(0)
M = 100
X = np.random.randn(M, M) - 2
X0 = (X < 0)
X0[[0, -1]], X0[:, [0, -1]] = False, False
jx, kx = gidx(X0)