So, I checked other questions and answers about this and I didn't understand the reason, could you please help me with this, please
I'm learning linear regression and I implemented the code for Linear Regression with two variables.
This implementation should predict the sum of two numbers but it's giving an error
Implemented code here:
import matplotlib.pyplot as plt
import numpy as np
x = np.array([[1,1],[1,2],[2,3],[3,4],[4,5],[5,6]])
y = np.array([2,3,5,7,9,11])
#hypothesis
def hypothesis(theta, x):
return theta[0] + theta[1]*x[0] + theta[2]*x[1]
#cost function J(t0, t1, t2)
def cost(theta, x, y):
m = x.shape[0]
error = 0
for i in range(m):
d = x[i]
hx = hypothesis(theta, d)
error = error + (hx - y[i])**2
return error
#differentiation of cost function
def diffGradient(theta, x, y):
grad = np.zeros((3,))
m = x.shape[0]
for i in range(m):
hx = hypothesis(theta, x)
grad[0] = grad[0] + (hx - y[i])
grad[1] = grad[1] + (hx - y[i])*x[0]
grad[2] = grad[2] + (hx - y[i])*x[1]
return 0
#gradient descent funtion
def gradientDescent(x, y, learning_rate = 0.001):
theta = [-2.0,0.0,1.0]
iter = 100
error_list = []
theta_list = []
for i in range(iter):
d = x[i]
grad = diffGradient(theta, d, y)
e = cost(theta, d, y)
error_list.append(e)
theta_list.append((theta[0],theta[1],theta[2]))
#simultaneous update
theta[0] = theta[0] - learning_rate*grad[0]
theta[1] = theta[1] - learning_rate*grad[1]
theta[2] = theta[2] - learning_rate*grad[2]
return theta, theta_list, error_list
final_theta, theta_list, error_list = gradientDescent(x,y)
After the above line, I'm getting this error
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-56-bda7687d0af9> in <module>
----> 1 final_theta, theta_list, error_list = gradientDescent(x,y)
<ipython-input-55-033133fbfbd5> in gradientDescent(x, y, learning_rate)
8 d = x[i]
9 grad = diffGradient(theta, d, y)
---> 10 e = cost(theta, d, y)
11 error_list.append(e)
12 theta_list.append((theta[0],theta[1],theta[2]))
<ipython-input-41-6a07f4b81c9c> in cost(theta, x, y)
5 for i in range(m):
6 d = x[i]
----> 7 hx = hypothesis(theta, d)
8 error = error + (hx - y[i])**2
9 return error
<ipython-input-27-43ce9d7c567b> in hypothesis(theta, x)
1 #hypothesis
2 def hypothesis(theta, x):
----> 3 return theta[0] + theta[1]*x[0] + theta[2]*x[1]
IndexError: invalid index to scalar variable.
I don't know what I'm doing wrong. Any help would be appreciated.
What's the x that you pass to
gradientDescent(x,y)
Look at the traceback and try to figure out what variable has the problem - the problem is that you can't index a scalar, a number. It has to be a list or array. Trace the problem variable back up through your code.
In the traceback:
in the problem line you use x[0] and x[1]. What's x at this point?
In the calling function it's d, which is set with d = x[i]
In gradientDescent the passed variable is again called d, and set as d = x[i]
So you have 3 levels of indexing. Does the original x support that?
You have to understand the problem before you try a fix.
Related
def gradient_descent(epoch, x, y, alpha):
cost = np.ones(epoch) #Tracking Costs
features = x.shape[1]
samplesize = x.shape[0]
theta = np.ones(features)
hypo = np.dot(x, theta) #Hypothesis Function
for i in range(0, epoch):
theta[0] = theta[0] - (alpha/ samplesize) * sum(hypo - y) #Updating Bias theta
for j in range(1, features + 1):
theta[j] = theta[j] - (alpha /samplesize) * sum(hypo - y) * x[:, j] #Updating feature weights
hypo = np.dot(x, theta)
cost = (1/2*x.shape[0]) * np.sum(hypo - y) **2 #MSE Function
return(cost, theta)
cost, theta = gradient_descent(1000, x1, y,.01)
Why am I getting this error?
TypeError Traceback (most recent call last)
TypeError: only size-1 arrays can be converted to Python scalars
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
<ipython-input-193-22d4ca0f3896> in <module>
17
18
---> 19 cost, theta = gradient_descent(1000, x1, y,.01)
<ipython-input-193-22d4ca0f3896> in gradient_descent(epoch, x, y, alpha)
10 theta[0] = theta[0] - (alpha/ samplesize) * sum(hypo - y) #Updating Bias theta
11 for j in range(1, features + 1):
---> 12 theta[j] = theta[j] - (alpha /samplesize) * sum(hypo - y) * x[:, j] #Updating feature weights
13 hypo = np.dot(x, theta)
14 cost = (1/2*x.shape[0]) * np.sum(hypo - y) **2 #MSE Function
ValueError: setting an array element with a sequence.
x[:, j] is a sequence, so the result of right expression is a sequence, and you assign it to a element in array theta, which result in the error reported
Basically, I am trying to sample x and y from a proposal distribution function and then find the weights for the sample, and create 10 different sets of n = 10,000 samples. I was given an array of 55 values for radius and mu and sigma, and I have 10,000 values of x and y as samples.
here is my code:
import numpy as np
import matplotlib.pyplot as plt
GN = 4.302**-6
A = 8445.264
mu = np.array([1,1])
C = [[1,0],[0,1]]
num = 10**4
def chi_sq(x,y):
r = np.zeros(len(radius))
for s in range(len(radius)):
f_xy = np.sqrt( GN * 10**y * radius[s] / (radius[s] + 10**x )**3 )
chi_sqtotal = np.sum((f_xy - mu[s] )**2 / (sigma[s])**2)
r += chi_sqtotal
return r
def P(x,y):
return A*np.exp(-0.5*chi_sq(x,y))
for i in range(10):
def chi_sq(x,y):
r = np.zeros(len(radius))
for s in range(len(radius)):
f_xy = np.sqrt( GN * 10**y * radius[s] / (radius[s] + 10**x )**3 )
chi_sqtotal = np.sum((f_xy - mu[s] )**2 / (sigma[s])**2)
r += chi_sqtotal
return r
def P(x,y):
return A*np.exp(-0.5*chi_sq(x,y))
detC = np.linalg.det(C)
Cinv = np.linalg.inv(C)
def Q(x,y):
r = np.array([x,y]) - mu
prefactor = 1/(2*np.pi*np.sqrt(detC))
exponent = 0.5*(r # Cinv # r)
return prefactor*np.exp(-exponent)
r_samples = np.random.multivariate_normal(mu,C,num)
x_samples = r_samples[:,0]
y_samples = r_samples[:,1]
w = np.zeros(num)
for j in range(num):
x, y = r_samples[j]
w[j] = P(x,y)/Q(x,y)
Here is the error I am getting :
IndexError Traceback (most recent call last)
<ipython-input-114-638cc1a1b2e2> in <module>()
41 x, y = r_samples[j]
42
---> 43 w[j] = P(x,y)/Q(x,y)
1 frames
<ipython-input-114-638cc1a1b2e2> in chi_sq(x, y)
22 for s in range(len(radius)):
23 f_xy = np.sqrt( GN * 10**y * radius[s] / (radius[s] + 10**x )**3 )
---> 24 chi_sqtotal = np.sum((f_xy - mu[s] )**2 / (sigma[s])**2)
25 r += chi_sqtotal
26 return r
IndexError: index 2 is out of bounds for axis 0 with size 2
The error is very descriptive:
IndexError: index 2 is out of bounds for axis 0 with size 2
If I have a numpy.ndarray, let's call it x, then the axes are as follows:
x[axis_0][axis_1]...
They are saying that the size of the axis is 2, and you're indexing it with 2, which is out of bounds. Notice
mu = np.array([1,1])
and the line with mu has the error. In this chunk:
for s in range(len(radius)):
f_xy = np.sqrt( GN * 10**y * radius[s] / (radius[s] + 10**x )**3 )
chi_sqtotal = np.sum((f_xy - mu[s] )**2 / (sigma[s])**2)
r += chi_sqtotal
s goes above 1, causing the error. You need to make a correction somewhere and not loop as high as len(radius), because mu only has 2 elements.
I'm trying to plot the result of a trapezoidal method into a graph, I figured out that to get the exact value of the integral you can use integrate.trapz(f(x)) from the scypy libary. This is my second month learning comp math, not really sure about libraries usage yet.
Here is my code
from scipy import integrate
def f(x):
return 1 - x - 4*x**3 + 2*x**5
def trapezoid(a, b, n):
h = (b - a) / n
s = (f(a) + f(b))
i = 1
while i < n:
s += 2 * f(a + i * h)
i += 1
area = ((h / 2) * s)
return area
def graphTrapezoid():
val = []
err = []
exact_sum = integrate.trapz(f(b))-integrate.trapz(f(a))
for i in range (2,100):
val.append(trapezoid(a,b,i))
errVal = abs((trapezoid(a,b,i) - exact_sum)/exact_sum)
err.append(errVal)
plt.plot(val, err)
a = 1
b = 5
n = 1
graphTrapezoid()
Below is the error statement that I got
IndexError Traceback (most recent call last)
<ipython-input-134-504c0acc46bf> in <module>
39
40
---> 41 graphTrapezoid()
<ipython-input-134-504c0acc46bf> in graphTrapezoid()
29 val = []
30 err = []
---> 31 exact_sum = integrate.trapz(f(b))-integrate.trapz(f(a))
32 for i in range (2,100):
33 val.append(trapezoid(a,b,i))
~\Anaconda3\lib\site-packages\numpy\lib\function_base.py in trapz(y, x, dx, axis)
4059 slice1 = [slice(None)]*nd
4060 slice2 = [slice(None)]*nd
-> 4061 slice1[axis] = slice(1, None)
4062 slice2[axis] = slice(None, -1)
4063 try:
IndexError: list assignment index out of range
From the scipy.integrate.trapz documentation:
The function definition is:
scipy.integrate.trapz(y, x=None, dx=1.0, axis=-1)
Where y should be "array_like". However, you are passing two integer values to the trapz function.
If you want to integrate f betweeen b and a, maybe you could first define a list of x values:
a = 1
b = 5
step = 0.1 # A smaller step will give you a more precise result
x = []
for i in range(int((b-a)/step + 1)):
x.append(a + i*step)
Then, sample your function:
y = [f(val) for val in x]
And lastly, you could integrate your samples:
result = integrate.trapz(y, x, dx=step)
I made a gradient descent algorithm in Python and it doesn't work. My m and b values keep increasing and never stop until I get the -inf error or the overflow encountered in square error.
import numpy as np
x = np.array([2,3,4,5])
y = np.array([5,7,9,5])
m = np.random.randn()
b = np.random.randn()
error = 0
lr = 0.0001
for q in range(1000):
for i in range(len(x)):
ypred = m*x[i] + b
error += (ypred - y[i]) **2
m = m - (x * error) *lr
b = b - (lr * error)
print(b,m)
I expected my algorithm to return the best m and b values for my data (x and y) but it didn't work. What is going wrong?
import numpy as np
x = np.array([2,3,4,5])
y = 0.3*x+0.6
m = np.random.randn()
b = np.random.randn()
lr = 0.001
for q in range(100000):
ypred = m*x + b
error = (1./(2*len(x))) * np.sum(np.square(ypred - y)) #eq 1
m = m - lr * np.sum((ypred - y)*x)/len(x) # eq 2 and eq 4
b = b - lr * np.sum(ypred - y)/len(x) # eq 3 and eq 5
print (m , b)
Output:
0.30007724168011807 0.5997039817571881
Math behind it
Use numpy vectorized operations to avoid loops.
I think you implemented the formula incorrectly:
Use summation on x - error
divide by length of x
See below code:
import numpy as np
x = np.array([2,3,4,5])
y = np.array([5,7,9,11])
m = np.random.randn()
b = np.random.randn()
error = 0
lr = 0.1
print(b, m)
for q in range(1000):
ypred = []
for i in range(len(x)):
temp = m*x[i] + b
ypred.append(temp)
error += temp - y[i]
m = m - np.sum(x * (ypred-y)) *lr/len(x)
b = b - np.sum(lr * (ypred-y))/len(x)
print(b,m)
Output:
-1.198074371762264 0.058595039571115955 # initial weights
0.9997389097653074 2.0000681277214487 # Final weights
I'm trying to complete week 4 the Machine Learning course on Coursera. The assingment uses the MINST data for multi-class classification.
The dimensions are X (5000,401), y (5000,1), theta (10,401), which start off as arrays. X was inserted with 1's on the first feature column.
My cost and gradient functions are below:
def sigmoid(z):
g = 1 / (1 + np.exp(-z))
return g
def lrCostFunction(theta, X, y, my_lambda):
m = float(len(X))
theta = np.matrix(theta)
X = np.matrix(X)
y = np.matrix(y)
#cost function:
term1 = np.multiply(-y,np.log(sigmoid(X*theta.T)))
term2 = np.multiply((1-y),np.log(1-sigmoid(X*theta.T)))
reg = np.power(theta[:,1:theta.shape[1]],2)
J = np.sum(term1-term2)/m + (my_lambda/(2.0*m) * np.sum(reg))
return J
def gradient (theta, X, y, my_lambda):
m = float(len(X))
theta = np.matrix(theta)
X = np.matrix(X)
y = np.matrix(y)
#gradient:
error = sigmoid(X * theta.T) - y
g = (X.T * error /(m)).T + ((my_lambda/m) * theta)
g[0,0] = np.sum(np.multiply(error, X[:,0])) / m
return g
Here is my One vs All classification function with the TNC optimization:
def oneVsAll(X, y, num_labels, my_lambda):
m = float(X.shape[0])
n = float(X.shape[1])-1
all_theta = np.zeros((num_labels,n+1))
for K in range(1, num_labels + 1):
theta = np.zeros(n+1)
y_logical = np.array([1 if j == K else 0 for j in y]).reshape(m,1)
opt_theta = opt.minimize(fun=lrCostFunction, x0=theta, \
args=(X,y_logical,my_lambda), \
method='TNC', jac=gradient).x
all_theta[K-1,:] = opt_theta
return all_theta
When I try to run CG however, it returns the error at line 8: "shapes (1,401) and (1,401) not aligned: 401 (dim 1) != 1 (dim 0)":
def oneVsAll(X, y, num_labels, my_lambda):
m = float(X.shape[0])
n = float(X.shape[1])-1
all_theta = np.zeros((num_labels,n+1))
for K in range(1, num_labels + 1):
theta = np.zeros(n+1)
y_logical = np.array([1 if j == K else 0 for j in y]).reshape(m,1)
opt_theta = opt.fmin_cg(f=lrCostFunction, x0=theta, \
fprime=gradient, \
args=(X,y_logical,my_lambda))
all_theta[K-1,:] = opt_theta
return all_theta
I saw elsewhere that CG only likes 1-d vectors from y. If I try to flatten y or reduce its dimension, however, everything else breaks. Is it generally a bad idea to use np.matrix as oppose to use np.dot with arrays? I like being able to easily transpose with matrixes.
Any help would be greatly appreciated.