We Keep Coding

Problem with a perceptron. ValueError: operands could not be broadcast together with shapes (20,2) (20,)

I have a Perceptron code (Raschka). We got an exercise to rebuilt the code. I've tried to generate 2 gaussian distributions with the linear separability (that is the task). Instead of basic irises. I've read some threads here with the same error but unfortunately it doesn't seem like the same thing.
Here I have my perceptron model.
class Perceptron(object):
def __init__(self, eta = 0.0001, n_iter = 20, random_state = 1):
self.eta = eta
self.n_iter = n_iter
self.random_state = random_state
def fit(self, X, y):
rgen = np.random.RandomState(self.random_state)
self.w_ = rgen.normal(loc = 0.0, scale = 0.1, size = 1 + X.shape[2])
self.errors_ = []
for _ in range(self.n_iter):
errors = 0
for xi, target in zip(X, y):
update = self.eta * (target - self.predict(xi))
self.w_[1:] += update * xi
self.w_[0] += update
errors += int(update != 0.0 )
self.errors_.append(errors)
return self
def net_input(self, X):
return np.dot(X, self.w_[1:]) + self.w_[0]
def predict(self, X):
return np.where(self.net_input(X) != 0.0 , 1, -1)
However here I bump into the problem like matrices incompatibility:
I guess this is because of my wrong generated distributions:
import numpy as np
import matplotlib.pyplot as plt
# creating 2 gaussian distributions if x axis and y axis
x1 = 1 + 1 * np.random.randn(20,2)
x2 = 5.5 + 1 * np.random.randn(20,2)
y = np.array([x1, x2]) # target values
y = np.where(y == x1, -1, 1) # giving a class label
y.resize(40,2)
X = np.array([x1, x2]) # parameters
X.resize(40,2)
plt.scatter(x1[:, 0], x1[:, 1], color = 'blue', label = 'класс 1', marker = 'x')
plt.scatter(x2[:, 0], x2[:, 1], color = 'red', label = 'класс 2', marker = 'o')
plt.xlabel('распределение по оси x')
plt.ylabel('распределение по оси y')
plt.legend(loc = 'upper left')
plt.show()
I've played with transposing matrices in different ways but it has no sense. I've also checked my X and y shapes. It is the same: (2, 20, 2) and (2, 20, 2). Both matrices mutually multiply. Dtypes are int32 (y), float64 (X).

How to count points inside a contour obtained from svm.SVC?

I generated a data-set from a bivariate gaussian to simulate a signal and a background. Then I tried to separate the two components using svm.SVC with kernel 'rbf'. Now I'd like to count how many background points lay inside the acceptance region, how many signal points lay outside the region etc. in order to study the efficiency of the separation method.
I have to say that I got parts of this code from an online example because it is the first time for me doing such type of analysis; so I'm not sure I really understand how the contour line is created, consequently I'm not able to handle it and to analyse its position respect to the points.
Thanks
mu_vec1 = np.array([0,0])
cov_mat1 = np.array([[0.3**2,0.5*0.3*0.3],[0.5*0.3*0.3,0.3**2]])
x1_samples = np.random.multivariate_normal(mu_vec1, cov_mat1, 1000)
#mu_vec1 = mu_vec1.reshape(1,2).T
mu_vec2 = np.array([2,1])
cov_mat2 = np.array([[1,0.4],[0.4,1]])
x2_samples = np.random.multivariate_normal(mu_vec2, cov_mat2, 1000)
#mu_vec2 = mu_vec2.reshape(1,2).T
X = np.concatenate((x1_samples,x2_samples), axis = 0)
Y = np.array([0]*1000 + [1]*1000)
plt.scatter(x1_samples[:,0],x1_samples[:,1], label='Signal')
plt.scatter(x2_samples[:,0],x2_samples[:,1],label='Background')
C = 1.0 # SVM regularization parameter
clf = svm.SVC(kernel = 'rbf', C=C, probability = True )
clf.fit(X, Y)
h = .02 # step size in the mesh
# create a mesh to plot in
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
print(clf.score(X,Y))
# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, m_max]x[y_min, y_max].
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.contour(xx, yy, Z, cmap=plt.cm.Paired)
plt.xlabel('x')
plt.ylabel('y')
plt.legend()

I actually solve it using the method 'predict' this way:
false_pos=0
true_pos=0
true_neg=0
false_neg=0
pred_sig = clf.predict(np.c_[x1_samples[:,0], x1_samples[:,1]])
for i in range(0,len(pred_sig)):
if pred_sig[i] == 1:
false_neg=false_neg + 1
if pred_sig[i] == 0:
true_pos=true_pos + 1
pred_back = clf.predict(np.c_[x2_samples[:,0], x2_samples[:,1]])
for i in range(0,len(pred_back)):
if pred_back[i] == 1:
true_neg=true_neg + 1
if pred_back[i] == 0:
false_pos=false_pos + 1
print(true_pos+false_pos+true_neg+false_neg)
purezza = true_pos/(true_pos+false_pos)
rig_fondo = true_neg/(true_neg+false_neg)

How can I use multiple dimensional polynomials with numpy.polynomial?

I'm able to use numpy.polynomial to fit terms to 1D polynomials like f(x) = 1 + x + x^2. How can I fit multidimensional polynomials, like f(x,y) = 1 + x + x^2 + y + yx + y x^2 + y^2 + y^2 x + y^2 x^2? It looks like numpy doesn't support multidimensional polynomials at all: is that the case? In my real application, I have 5 dimensions of input and I am interested in hermite polynomials. It looks like the polynomials in scipy.special are also only available for one dimension of inputs.
# One dimension of data can be fit
x = np.random.random(100)
y = np.sin(x)
params = np.polynomial.polynomial.polyfit(x, y, 6)
np.polynomial.polynomial.polyval([0, .2, .5, 1.5], params)
array([ -5.01799432e-08, 1.98669317e-01, 4.79425535e-01,
9.97606096e-01])
# When I try two dimensions, it fails.
x = np.random.random((100, 2))
y = np.sin(5 * x[:,0]) + .4 * np.sin(x[:,1])
params = np.polynomial.polynomial.polyvander2d(x, y, [6, 6])
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-13-5409f9a3e632> in <module>()
----> 1 params = np.polynomial.polynomial.polyvander2d(x, y, [6, 6])
/usr/local/lib/python2.7/site-packages/numpy/polynomial/polynomial.pyc in polyvander2d(x, y, deg)
1201 raise ValueError("degrees must be non-negative integers")
1202 degx, degy = ideg
-> 1203 x, y = np.array((x, y), copy=0) + 0.0
1204
1205 vx = polyvander(x, degx)
ValueError: could not broadcast input array from shape (100,2) into shape (100)

I got annoyed that there is no simple function for a 2d polynomial fit of any number of degrees so I made my own. Like the other answers it uses numpy lstsq to find the best coefficients.
import numpy as np
from scipy.linalg import lstsq
from scipy.special import binom
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
def _get_coeff_idx(coeff):
idx = np.indices(coeff.shape)
idx = idx.T.swapaxes(0, 1).reshape((-1, 2))
return idx
def _scale(x, y):
# Normalize x and y to avoid huge numbers
# Mean 0, Variation 1
offset_x, offset_y = np.mean(x), np.mean(y)
norm_x, norm_y = np.std(x), np.std(y)
x = (x - offset_x) / norm_x
y = (y - offset_y) / norm_y
return x, y, (norm_x, norm_y), (offset_x, offset_y)
def _unscale(x, y, norm, offset):
x = x * norm[0] + offset[0]
y = y * norm[1] + offset[1]
return x, y
def polyvander2d(x, y, degree):
A = np.polynomial.polynomial.polyvander2d(x, y, degree)
return A
def polyscale2d(coeff, scale_x, scale_y, copy=True):
if copy:
coeff = np.copy(coeff)
idx = _get_coeff_idx(coeff)
for k, (i, j) in enumerate(idx):
coeff[i, j] /= scale_x ** i * scale_y ** j
return coeff
def polyshift2d(coeff, offset_x, offset_y, copy=True):
if copy:
coeff = np.copy(coeff)
idx = _get_coeff_idx(coeff)
# Copy coeff because it changes during the loop
coeff2 = np.copy(coeff)
for k, m in idx:
not_the_same = ~((idx[:, 0] == k) & (idx[:, 1] == m))
above = (idx[:, 0] >= k) & (idx[:, 1] >= m) & not_the_same
for i, j in idx[above]:
b = binom(i, k) * binom(j, m)
sign = (-1) ** ((i - k) + (j - m))
offset = offset_x ** (i - k) * offset_y ** (j - m)
coeff[k, m] += sign * b * coeff2[i, j] * offset
return coeff
def plot2d(x, y, z, coeff):
# regular grid covering the domain of the data
if x.size > 500:
choice = np.random.choice(x.size, size=500, replace=False)
else:
choice = slice(None, None, None)
x, y, z = x[choice], y[choice], z[choice]
X, Y = np.meshgrid(
np.linspace(np.min(x), np.max(x), 20), np.linspace(np.min(y), np.max(y), 20)
)
Z = np.polynomial.polynomial.polyval2d(X, Y, coeff)
fig = plt.figure()
ax = fig.gca(projection="3d")
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, alpha=0.2)
ax.scatter(x, y, z, c="r", s=50)
plt.xlabel("X")
plt.ylabel("Y")
ax.set_zlabel("Z")
plt.show()
def polyfit2d(x, y, z, degree=1, max_degree=None, scale=True, plot=False):
"""A simple 2D polynomial fit to data x, y, z
The polynomial can be evaluated with numpy.polynomial.polynomial.polyval2d
Parameters
----------
x : array[n]
x coordinates
y : array[n]
y coordinates
z : array[n]
data values
degree : {int, 2-tuple}, optional
degree of the polynomial fit in x and y direction (default: 1)
max_degree : {int, None}, optional
if given the maximum combined degree of the coefficients is limited to this value
scale : bool, optional
Wether to scale the input arrays x and y to mean 0 and variance 1, to avoid numerical overflows.
Especially useful at higher degrees. (default: True)
plot : bool, optional
wether to plot the fitted surface and data (slow) (default: False)
Returns
-------
coeff : array[degree+1, degree+1]
the polynomial coefficients in numpy 2d format, i.e. coeff[i, j] for x**i * y**j
"""
# Flatten input
x = np.asarray(x).ravel()
y = np.asarray(y).ravel()
z = np.asarray(z).ravel()
# Remove masked values
mask = ~(np.ma.getmask(z) | np.ma.getmask(x) | np.ma.getmask(y))
x, y, z = x[mask].ravel(), y[mask].ravel(), z[mask].ravel()
# Scale coordinates to smaller values to avoid numerical problems at larger degrees
if scale:
x, y, norm, offset = _scale(x, y)
if np.isscalar(degree):
degree = (int(degree), int(degree))
degree = [int(degree[0]), int(degree[1])]
coeff = np.zeros((degree[0] + 1, degree[1] + 1))
idx = _get_coeff_idx(coeff)
# Calculate elements 1, x, y, x*y, x**2, y**2, ...
A = polyvander2d(x, y, degree)
# We only want the combinations with maximum order COMBINED power
if max_degree is not None:
mask = idx[:, 0] + idx[:, 1] <= int(max_degree)
idx = idx[mask]
A = A[:, mask]
# Do the actual least squares fit
C, *_ = lstsq(A, z)
# Reorder coefficients into numpy compatible 2d array
for k, (i, j) in enumerate(idx):
coeff[i, j] = C[k]
# Reverse the scaling
if scale:
coeff = polyscale2d(coeff, *norm, copy=False)
coeff = polyshift2d(coeff, *offset, copy=False)
if plot:
if scale:
x, y = _unscale(x, y, norm, offset)
plot2d(x, y, z, coeff)
return coeff
if __name__ == "__main__":
n = 100
x, y = np.meshgrid(np.arange(n), np.arange(n))
z = x ** 2 + y ** 2
c = polyfit2d(x, y, z, degree=2, plot=True)
print(c)

It doesn't look like polyfit supports fitting multivariate polynomials, but you can do it by hand, with linalg.lstsq. The steps are as follows:
Gather the degrees of monomials x**i * y**j you wish to use in the model. Think carefully about it: your current model already has 9 parameters, if you are going to push to 5 variables then with the current approach you'll end up with 3**5 = 243 parameters, a sure road to overfitting. Maybe limit to the monomials of __total_ degree at most 2 or three...
Plug the x-points into each monomial; this gives a 1D array. Stack all such arrays as columns of a matrix.
Solve a linear system with aforementioned matrix and with the right-hand side being the target values (I call them z because y is confusing when you also use x, y for two variables).
Here it is:
import numpy as np
x = np.random.random((100, 2))
z = np.sin(5 * x[:,0]) + .4 * np.sin(x[:,1])
degrees = [(i, j) for i in range(3) for j in range(3)] # list of monomials x**i * y**j to use
matrix = np.stack([np.prod(x**d, axis=1) for d in degrees], axis=-1) # stack monomials like columns
coeff = np.linalg.lstsq(matrix, z)[0] # lstsq returns some additional info we ignore
print("Coefficients", coeff) # in the same order as the monomials listed in "degrees"
fit = np.dot(matrix, coeff)
print("Fitted values", fit)
print("Original values", y)

I believe you have misunderstood what polyvander2d does and how it should be used. polyvander2d() returns the pseudo-Vandermonde matrix of degrees deg and sample points (x, y).
Here, y is not the value(s) of the polynomial at point(s) x but rather it is the y-coordinate of the point(s) and x is the x-coordinate. Roughly speaking, the returned array is a set of combinations of (x**i) * (y**j) and x and y are essentially 2D "mesh-grids". Therefore, both x and y must have identical shapes.
Your x and y, however, arrays have different shapes:
>>> x.shape
(100, 2)
>>> y.shape
(100,)
I do not believe numpy has a 5D-polyvander of the form polyvander5D(x, y, z, v, w, deg). Notice, all the variables here are coordinates and not the values of the polynomial p=p(x,y,z,v,w). You, however, seem to be using y (in the 2D case) as f.
It appears that numpy does not have 2D or higher equivalents for the polyfit() function. If your intention is to find the coefficients of the best-fitting polynomial in higher-dimensions, I would suggest that you generalize the approach described here: Equivalent of `polyfit` for a 2D polynomial in Python

The option isn't there because nobody wants to do that. Combine the polynomials linearly (f(x,y) = 1 + x + y + x^2 + y^2) and solve the system of equations yourself.

How to implement Perceptron in Python?

I am trying to follow the book by Daume
http://ciml.info/dl/v0_99/ciml-v0_99-ch04.pdf (page 43).
To fit a model for vanilla perceptron in python using numpy and without
using sciki-learn library.
The algorithm is given in the book
How can we implement this model in practice?
So far I have learned how to read the data and labels:
def read_data(infile):
data = np.loadtxt(infile)
X = data[:,:-1]
Y = data[:,-1]
return X, Y
The help will be appreciated!!

One way I figured out is this:
(Better ideas are always welcome!!)
#!python
# -*- coding: utf-8 -*-#
"""
Perceptron Algorithm.
#author: Bhishan Poudel
#date: Oct 31, 2017
"""
# Imports
import numpy as np
import matplotlib.pyplot as plt
from numpy.linalg import norm
import os, shutil
np.random.seed(100)
def read_data(infile):
data = np.loadtxt(infile)
X = data[:,:-1]
Y = data[:,-1]
return X, Y
def plot_boundary(X,Y,w,epoch):
try:
plt.style.use('seaborn-darkgrid')
# plt.style.use('ggplot')
#plt.style.available
except:
pass
# Get data for two classes
idxN = np.where(np.array(Y)==-1)
idxP = np.where(np.array(Y)==1)
XN = X[idxN]
XP = X[idxP]
# plot two classes
plt.scatter(XN[:,0],XN[:,1],c='b', marker='_', label="Negative class")
plt.scatter(XP[:,0],XP[:,1],c='r', marker='+', label="Positive class")
# plt.plot(XN[:,0],XN[:,1],'b_', markersize=8, label="Negative class")
# plt.plot(XP[:,0],XP[:,1],'r+', markersize=8, label="Positive class")
plt.title("Perceptron Algorithm iteration: {}".format(epoch))
# plot decision boundary orthogonal to w
# w is w2,w1, w0 last term is bias.
if len(w) == 3:
a = -w[0] / w[1]
b = -w[0] / w[2]
xx = [ 0, a]
yy = [b, 0]
plt.plot(xx,yy,'--g',label='Decision Boundary')
if len(w) == 2:
x2=[ w[0], w[1], -w[1], w[0]]
x3=[ w[0], w[1], w[1], -w[0]]
x2x3 =np.array([x2,x3])
XX,YY,U,V = list(zip(*x2x3))
ax = plt.gca()
ax.quiver(XX,YY,U,V,scale=1, color='g')
# Add labels
plt.xlabel('X')
plt.ylabel('Y')
# limits
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
plt.xlim(x_min,x_max)
plt.ylim(y_min,y_max)
# lines from origin
plt.axhline(y=0, color='k', linestyle='--',alpha=0.2)
plt.axvline(x=0, color='k', linestyle='--',alpha=0.2)
plt.grid(True)
plt.legend(loc=1)
plt.show()
# Always clost the plot
plt.close()
def predict(X,w):
return np.sign(np.dot(X, w))
def plot_contour(X,Y,w,mesh_stepsize):
try:
plt.style.use('seaborn-darkgrid')
# plt.style.use('ggplot')
#plt.style.available
except:
pass
# Get data for two classes
idxN = np.where(np.array(Y)==-1)
idxP = np.where(np.array(Y)==1)
XN = X[idxN]
XP = X[idxP]
# plot two classes with + and - sign
fig, ax = plt.subplots()
ax.set_title('Perceptron Algorithm')
plt.xlabel("X")
plt.ylabel("Y")
plt.plot(XN[:,0],XN[:,1],'b_', markersize=8, label="Negative class")
plt.plot(XP[:,0],XP[:,1],'y+', markersize=8, label="Positive class")
plt.legend()
# create a mesh for contour plot
# We first make a meshgrid (rectangle full of pts) from xmin to xmax and ymin to ymax.
# We then predict the label for each grid point and color it.
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
# Get 2D array for grid axes xx and yy (shape = 700, 1000)
# xx has 700 rows.
# xx[0] has 1000 values.
xx, yy = np.meshgrid(np.arange(x_min, x_max, mesh_stepsize),
np.arange(y_min, y_max, mesh_stepsize))
# Get 1d array for x and y axes
xxr = xx.ravel() # shape (700000,)
yyr = yy.ravel() # shape (700000,)
# ones vector
# ones = np.ones(xxr.shape[0]) # shape (700000,)
ones = np.ones(len(xxr)) # shape (700000,)
# Predict the score
Xvals = np.c_[ones, xxr, yyr]
scores = predict(Xvals, w)
# Plot contour plot
scores = scores.reshape(xx.shape)
ax.contourf(xx, yy, scores, cmap=plt.cm.Paired)
# print("xx.shape = {}".format(xx.shape)) # (700, 1000)
# print("scores.shape = {}".format(scores.shape)) # (700, 1000)
# print("scores[0].shape = {}".format(scores[0].shape)) # (1000,)
# show the plot
plt.savefig("Perceptron.png")
plt.show()
plt.close()
def perceptron_sgd(X, Y,epochs):
"""
X: data matrix without bias.
Y: target
"""
# add bias to X's first column
ones = np.ones(X.shape[0]).reshape(X.shape[0],1)
X1 = np.append(ones, X, axis=1)
w = np.zeros(X1.shape[1])
final_iter = epochs
for epoch in range(epochs):
print("\n")
print("epoch: {} {}".format(epoch, '-'*30))
misclassified = 0
for i, x in enumerate(X1):
y = Y[i]
h = np.dot(x, w)*y
if h <= 0:
w = w + x*y
misclassified += 1
print('misclassified? yes w: {} '.format(w,i))
else:
print('misclassified? no w: {}'.format(w))
pass
if misclassified == 0:
final_iter = epoch
break
return w, final_iter
def gen_lin_separable_data(data, data_tr, data_ts,data_size):
mean1 = np.array([0, 2])
mean2 = np.array([2, 0])
cov = np.array([[0.8, 0.6], [0.6, 0.8]])
X1 = np.random.multivariate_normal(mean1, cov, size=int(data_size/2))
y1 = np.ones(len(X1))
X2 = np.random.multivariate_normal(mean2, cov, size=int(data_size/2))
y2 = np.ones(len(X2)) * -1
with open(data,'w') as fo, \
open(data_tr,'w') as fo1, \
open(data_ts,'w') as fo2:
for i in range( len(X1)):
line = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X1[i][0], X1[i][1], y1[i])
line2 = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X2[i][0], X2[i][1], y2[i])
fo.write(line)
fo.write(line2)
for i in range( len(X1) - 20):
line = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X1[i][0], X1[i][1], y1[i])
line2 = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X2[i][0], X2[i][1], y2[i])
fo1.write(line)
fo1.write(line2)
for i in range((len(X1) - 20), len(X1) ):
line = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X1[i][0], X1[i][1], y1[i])
line2 = '{:5.2f} {:5.2f} {:5.0f} \n'.format(X2[i][0], X2[i][1], y2[i])
fo2.write(line)
fo2.write(line2)
def main():
"""Run main function."""
# generate linearly separable data
data = 'data.txt'
data_tr = 'data_train.txt'
data_ts = 'data_test.txt'
data_size = 200
gen_lin_separable_data(data, data_tr, data_ts,data_size)
# read data
epochs = 20
X_train, Y_train = read_data(data_tr)
X_test, Y_test = read_data(data_ts)
# fit perceptron
w, final_iter = perceptron_sgd(X_train,Y_train,epochs)
print('w = ', w)
plot_boundary(X_test,Y_test,w,final_iter)
# contour plot
mesh_stepsize = 0.01
plot_contour(X_test,Y_test,w,mesh_stepsize)
if __name__ == "__main__":
main()
The decision boundary looks like this:

There is an implementation of perceptron in my recent repo: NP_ML. The example result is:

Logistic regression: plotting decision boundary from theta

I have the following code:
x1 = np.random.randn(100)
y1 = np.random.randn(100) + 3
x2 = np.random.randn(100) + 3
y2 = np.random.randn(100)
plt.plot(x1, y1, "+", x2, y2, "x")
plt.axis('equal')
plt.show()
which results in the following image
I have implemented my own logistic regression, and this returns a theta, and I want to use this theta to plot the decision boundary, but I'm not sure how to do this.
X = np.matrix(np.vstack((np.hstack((x1,x2)), np.hstack((y1,y2)))).T)
X = np.concatenate((np.ones((X.shape[0], 1)), X), axis=1)
Y = np.matrix(1.0 * np.hstack((np.zeros(100), np.ones(100)))).T
learning_rate = 0.0001
iterations = 3000
theta = np.matrix([[0.5], [0.5], [0.5]])
theta = logistic_regression(theta, X, Y, learning_rate, iterations)
and this gives theta =
[[ 0.40377942]
[ 0.53696461]
[ 0.1398419 ]]
for example. How can I use this to plot the decision boundary?

You want to plot θTX = 0, where X is the vector containing (1, x, y). That is, you want to plot the line defined by theta[0] + theta[1]*x + theta[2]*y = 0. Solve for y:
y = -(theta[0] + theta[1]*x)/theta[2]
So, something like:
theta = theta[:,0] # Make theta a 1-d array.
x = np.linspace(-6, 6, 50)
y = -(theta[0] + theta[1]*x)/theta[2]
plt.plot(x, y)
Something doesn't look right, though, because you have theta[1] > 0 and theta[2] > 0, which results in a line with a negative slope.