I would like to perform a simple linear regression using statsmodels and I've tried several different methods by now but I just don't get it to work. The code that I have constructed now doesn't give me any errors but it also doesn't show me the result
I am trying to create a model for the variable "Direction" which takes the value 0 if the return for the corresponding date was negative and 1 if it was positive. The explinatory variables are the (5) lags of the returns. The df13 contains the lags and also the direction for each observed date. I tried this code and as I mentioned it doesn't give an error but says " Optimization terminated successfully.
Current function value: 0.682314
Iterations 5
However, I would like to see the typical table with all the beta values, their significance etc.
Also, what would you say, since Direction is a binary variable may it be better to use a logit instead of a linear model? However, in the assignment it appeared as a linear model.
And lastly, I am sorry its not displayed here correctly but I don't know how to write as code or insert my dataframe
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
import os
import itertools
from sklearn import preprocessing
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
import statsmodels.api as sm
import matplotlib.pyplot as plt
from statsmodels.sandbox.regression.predstd import wls_prediction_std
...
X = df13[['Lag1', 'Lag2', 'Lag3', 'Lag4', 'Lag5']]
Y = df13['Direction']
X = sm.add_constant(X)
model = sm.Logit(Y.astype(float), X.astype(float)).fit()
predictions = model.predict(X)
print_model = model.summary
print(print_model)
Edit: I'm sure it has to be a logit regression so I updated that part
I don't know if this is unintentional, but it looks like you need to define X and Y separately:
X = df13[['Lag1', 'Lag2', 'Lag3', 'Lag4', 'Lag5']]
Y = df13['Direction']
Secondly, I'm not familiar with statsmodel, but I would try converting your dataframes to numpy arrays. You can do this with
Xnum = X.to_numpy()
ynum = y.to_numpy()
And try passing those to the regressors.
Related
I'm attempting to perform a lasso regression for a larger than main memory dataset by using Dask, but there doesn't seem to be a cleanly documented way to do so.
I did previously find a somewhat related question but no actual answer.
Looking into how scikit sets up the Lasso regression, I thought I could set it up the same way. For example, here is one approach I tried
from dask_ml.datasets import make_regression
import dask_glm.families
import dask_glm.regularizers
import dask_glm.algorithms
import pandas as pd
# dask dataframes
X, y = make_regression(n_samples=1000, chunks=100)
# pandas dataframes
df_X = X.compute()
df_y = y.compute()
family = dask_glm.families.Normal()
regularizer = dask_glm.regularizers.ElasticNet(weight=1)
b = dask_glm.algorithms.gradient_descent(X=X, y=y, max_iter=100000, family=family, regularizer=regularizer, alpha=0.01, normalize=False, fit_intercept=False)
print(b)
reg = linear_model.Lasso(alpha=0.01, fit_intercept=False)
reg.fit(df_X, df_y)
print(reg.coef_)
However, the coefficients don't match up at all, and the dask version's coefficients seem more unstable than scikit's.
Here's another approach I tried, this time based on a comment from this GH issue
from dask_ml.datasets import make_regression
from dask_glm.regularizers import L1
from dask_glm.estimators import LinearRegression
X, y = make_regression(n_samples=1000, chunks=100)
lr = LinearRegression(regularizer=L1())
lr.fit(X, y)
print(lr.coef_)
Again, the coefficients seem very unstable.
Ideally there would already be an implementation of Lasso using Dask for this, but I can't seem to find much on the internet except for running LassoCV with dask as the backend to joblib, which is a little different than I want.
I am trying to run a regression without predictor, just constant and error term. The model is y = a + error.
I have tried as follows:
import numpy as np
import statsmodels.api as sm
import matplotlib.pyplot as plt
y = np.random.normal(size=50)
sm.OLS(y, sm.add_constant(), missing='drop').fit()
However, this does not work.
As denoted here, using regression without predictors is not a major data analysis tool. Logistic regression is not a classifier, while this was already discussed as "Linear vs. Logistic Regression on Classification Problems" and "Regression for Binary Classification". However, it can still be a requirement for any reason like the one you pointed in this question. Thus, we will try to provide a proper solution as an answer to your case.
Similar to this question, you can use DummyRegressor from Sklearn as follows:
import numpy as np
import statsmodels.api as sm
import matplotlib.pyplot as plt
from sklearn.dummy import DummyRegressor
X = np.random.normal(size=50)
y = np.random.normal(size=50)
dummy_regr = DummyRegressor(strategy="mean")
dummy_regr.fit(X, y)
...
I want to get the marginal effects of a logistic regression from a sklearn model
I know you can get these for a statsmodel logistic regression using '.get_margeff()'. Is there nothing for sklearn? I want to avoid doing the calculation my self as I feel there would be a lot of room for error.
import statsmodels.formula.api as sm
from statsmodels.tools.tools import add_constant
from sklearn.datasets import load_breast_cancer
import pandas as pd
import numpy as np
data = load_breast_cancer()
x = data.data
y= data.target
x=add_constant(x,has_constant='add')
model = sm.Logit(y, x).fit_regularized()
margeff = model.get_margeff(dummy=True,count=True)
##print the margal effect
print(margeff.margeff)
>> [ 6.73582136e-02 2.15779589e-04 1.28857837e-02 -1.06718136e-03
-1.96032750e+00 1.36137385e+00 -1.16303369e+00 -1.37422595e+00
8.14539021e-01 -1.95330095e+00 -4.86235558e-01 4.84260993e-02
7.16675627e-02 -2.89644712e-03 -5.18982198e+00 -5.93269894e-01
3.22934080e+00 -1.28363008e+01 3.07823155e+00 5.84122170e+00
1.92785670e-02 -9.86284081e-03 -7.53298463e-03 -3.52349287e-04
9.13527446e-01 1.69938656e-01 -2.89245493e-01 -4.65659522e-01
-8.32713335e-01 -1.15567833e+00]
# manual calculation, doing this as you can get the coef_ from a sklearn model and use in the function
def PDF(XB):
var1 = np.exp(XB)
var2 = np.power((1+np.exp(XB)),2)
var3 = (var1 / var2)
return var3
arrPDF = PDF(np.dot(x,model.params))
ME=pd.DataFrame(np.dot(arrPDF[:,None],model.params[None,:]))
print(ME.iloc[:,1:].mean().to_list())
>>
[0.06735821358791198, 0.0002157795887363032, 0.012885783711597246, -0.0010671813611730326, -1.9603274961356965, 1.361373851981879, -1.1630336876543224, -1.3742259536619654, 0.8145390210646809, -1.9533009514684947, -0.48623555805230195, 0.04842609927469917, 0.07166756271689229, -0.0028964471200298475, -5.189821981601878, -0.5932698935239838, 3.229340802910038, -12.836300822253634, 3.0782315528664834, 5.8412217033605245, 0.019278567008384557, -0.009862840813512401, -0.007532984627259091, -0.0003523492868714151, 0.9135274456151128, 0.16993865598225097, -0.2892454926120402, -0.46565952159093893, -0.8327133347971125, -1.1556783345783221]
the custom function gives the same as ".get_margeff()" but there might be a lot of room for error when using the sklearn ceof_ in the custom function above.
Is there some method/function/Attribute in sklearn that can give me
the marginal effects
If there is not, is there another library get from the ceof_ and
data to the marginal effects
if the answer to both the above is no, are there any circumstances
in which the custom function will not work (e.g. with a particular solver or
penalty in sklearn)
I just hit this demand a few days ago.
My supervisor gave me this information that I want to share. Hope this can help you.
partial_dependence: This method can get the partial dependence or marginal effects you meant.
plot_partial_dependence: This method can plot the partial dependence.
Here is the sample code from the API Reference.
scikit-learn version: 0.21.2
from sklearn.inspection import plot_partial_dependence, partial_dependence
from sklearn.datasets import make_friedman1
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor
%matplotlib inline
X, y = make_friedman1()
# case1: linear model
lm = LinearRegression().fit(X, y)
# plot the partial dependence
plot_partial_dependence(lm, X, [0, (0, 1)])
# get the partial dependence
partial_dependence(lm, X, [0])
# case2: classifier
clf = GradientBoostingRegressor(n_estimators=10).fit(X, y)
# plot the partial dependence
plot_partial_dependence(clf, X, [0, (0, 1)])
# get the partial dependence
partial_dependence(clf, X, [0])
I am trying to evaluate an sklearn predictor which I have made over a larger than memory dask array of inputs. I have read over the parallel post fit documentation https://dask-ml.readthedocs.io/en/latest/modules/generated/dask_ml.wrappers.ParallelPostFit.html and am still having some problems. The following code illustrates the kind issue that I am running into:
from dask.base import tokenize
import numpy as np
import dask.array as da
from dask.array import Array
from sklearn.linear_model import LinearRegression
from dask_ml.wrappers import ParallelPostFit
"""
for stack overflow question
"""
x = np.linspace(0,100,100,dtype=np.int32)
y = np.linspace(0,100,100,dtype=np.int32)
z = np.linspace(0,100,100,dtype=np.int32)
Y = np.random.normal(size=(100,))
X = np.stack([x,y,z],axis=1)
reg = LinearRegression().fit(X,Y)
#now try to compute on dask arrays over the whole space
x= da.linspace(0,100,100,chunks=(10,)).astype(np.int32)
y= da.linspace(0,100,100,chunks=(10,)).astype(np.int32)
z= da.linspace(0,100,100,chunks=(10,)).astype(np.int32)
x,y,z = da.meshgrid(x,y,z,sparse=False,indexing='ij')
stacked = da.stack([x.flatten(),y.flatten(),z.flatten()],axis=1)
clf = ParallelPostFit(estimator=reg)
clf.predict(stacked)
Excecuting clf.predict throws a value error Can't drop an axis with more than 1 block. Please use atop instead.
which I dont understand how to correct.
Thank You for any help.
I am new to machine learning.
I created a data, random numbers in two sets. I am trying how to find a sample, however when doing following, I receive very low accuracy score:
from random import randint as R
from matplotlib import pyplot as plt
import numpy as np
from sklearn.neighbors import KNeighborsClassifier as KNC
from sklearn.cross_validation import train_test_split as tts
from sklearn.metrics import accuracy_score
a = [R(100,200) for x in range(100)]
b = [R(1000,2000) for x in range(100)]
c = a+b
X = np.array(c).reshape(len(c),1)
y = np.arange(len(c))
train_X, test_X, train_y,test_y = tts(X,y,test_size=0.4)
mimi = KNC()
mimi.fit(train_X, train_y)
y__pred = mimi.predict(train_X)
print(accuracy_score(train_y,y__pred))
print(mimi.score(train_X,train_y))
I receive a result of 0.18... What exactly does this mean? That a prediction score is just 18%? Please, can you explain to me in most simple way. I would really appreciate it.
By doing y = np.arange(len(c)) you have c different classes (here 200 classes) with only one example for each class. Learning the nearest neighbors on such a setup does not have any sense.
What you want (If I'm guessing right) is to have one class for data a and another class for data b.
Change y to:
y = np.concatenate([[0] *len(a), [1] *len(b)])
You'll see that you obtain an accuracy score of 1.0, which means that you successfully classify all your testing example.