Please help me. After splitting my data into
X_train, y_train, X_test, y_test = train_test_split(X,y)
then passing it to my linear regression model I.e
linereg = LinearRegression().fit(X_train, y_train)
It brings out an error saying array must be 2D not 1D array. How can I make it a 2D array.
first split the data correctly
X_train, x_test, Y_train,y_test=train_test_split(features,labels,train_size=0.7, test_size=0.3, random_state=2)
try reshaping the x_train and x_test using reshape method.
x_test=x_test.reshape(-1,1)
x_train=x_train.reshape(-1,1)
Related
i have a problem when i try to concatenate train set and validation set. I split my dataset into train set, validation set and test set. Then i scale them with 'StandardScaler()':
X_train, X_test, t_train, t_test = train_test_split(x, t, test_size=0.20, random_state=1)
X_train, X_valid, t_train, t_valid = train_test_split(X_train, t_train, test_size=0.25, random_state=1)
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_valid = sc.transform(X_valid)
X_test = sc.transform(X_test)
Then after model selection i want concatenate training and validation set:
X_train = pd.concat([X_train, X_valid])
t_train = pd.concat([t_train, t_valid])
But it doesn't work. I give me that error:
cannot concatenate object of type '<class 'numpy.ndarray'>'; only Series and DataFrame objs are valid
Can someone help me? Thanks
X_train, X_valid, t_train, t_valid are all numpy arrays so they need to be concatenated using numpy:
X_train = np.concatenate([X_train, X_valid])
t_train = np.concatenate([t_train, t_valid])
As suggested in the comments it is most likely not a good idea to merge train and validation sets together. Make sure you understand why datasets are split into training, testing and validation parts. You can apply cross validation to use all the data for train/test/valid in multiple steps.
Im trying to make a confusion matrix to determine how well my model performed. I split my model into x and y testing and training set however, to make my confusion matrix, I need the y_test data(the predicted data) and the actual data. Is there a way I can see the actual results of the y_test data.
Heres a little snippet of my cod:
x_train, x_test, y_train, y_test = train_test_split(a, yy, test_size=0.2, random_state=1)
model = MultinomialNB() #don forget these brackets here
model.fit(x_train,y_train.ravel())
#CONFUSION MATRIX
confusion = confusion_matrix(y_test, y_test)
print(confusion)
print(len(y_test))
Your y_test is the actual data and the results from the predict method will be the predicted data.
y_pred = model.predict(x_test)
confusion = confusion_matrix(y_test, y_pred)
I have data about Parkinson patients stored in the dataframe X and whether a patient has Parkinson indicated by y (0 or 1). This is retrieved by:
X=pd.read_csv('parkinsons.data',index_col=0)
y=X['status']
X=X.drop(['status'],axis=1)
Then, I create training and test samples:
X_train, y_train, X_test, y_test = train_test_split(X,y,test_size=0.3,random_state=7)
I want to use SVC on this training data:
svc=SVC()
svc.fit(X_train,y_train)
Then, I get the error:
ValueError: bad input shape (59, 22).
What did I do wrong and how can I get rid of this error?
You have problems with the definition of train_test_split Careful! train_test_split outputs the X part first followed by the Y part. You are actually naming y_train what is X_test. Change this and it should work:
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3,random_state=7)
Either use this
X_train, y_train, X_test, y_test =train_test_split(X,y,test_size=0.3,random_state=7)
svc=SVC()
svc.fit(X_train,X_test)
Or this
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3,random_state=7)
svc=SVC()
svc.fit(X_train,y_train)
I prefer using the second one
I have two dataframes. One with 1065*75000 which is my training matrix and one with 1065*1 which is my target matrix. I want to split them into training and testing so i can fed them to my CNN model. Any help would be appreciated.
I have tried following
y = target_dataframe["Target_column"]
X_train, X_test, Y_train, Y_test = (df_training, y, test_size = 0.2)
and
x_train, X_test = train_test_split(df_training)
y_train, y_test = train_test_split(df_target)
The cnn is working on both of them but how would i know which one is better approach and if there is something new please share it with me.
l want to split data into train and test and also a vector that contains names (it serves me as an index and reference).
name_images has a shape of (2440,)
My data are :
data has a shape of (2440, 3072)
labels has a shape of (2440,)
from sklearn.cross_validation import train_test_split
x_train, x_test, y_train, y_test= train_test_split(data, labels, test_size=0.3)
but l want also to split my name_images into name_images_train and name_images_test with respect to the split of data and labels
l tried
x_train, x_test, y_train, y_test,name_images_train,name_images_test= train_test_split(data, labels,name_images, test_size=0.3)
it doesn't preserve the order
Any suggestions
thank you
EDIT1:
x_train, x_test, y_train, y_test= train_test_split(data, labels,test_size=0.3, random_state=42)
name_images_train, name_images_test=train_test_split(name_images,
test_size=0.3,
random_state=42)
EDIT1 don't preserve the order
There are multiple ways to accomplish this.
The most straight forward is to use random_state parameter of train_test_split. As the documentation states:
random_state : int or RandomState :-
Pseudo-random number generator state used for random sampling.
When you fix the random_state, the indices which are generated for splitting the arrays into train and test are exact same each time.
So change your code to:
x_train, x_test,
y_train, y_test,
name_images_train, name_images_test=train_test_split(data, labels, name_images,
test_size=0.3,
random_state=42)
For more understanding on random_state, see my answer here:
https://stackoverflow.com/a/42197534/3374996
In my case, I realize that my input arrays were not in proper order in the first place. So for future Googlers--you may want to double-check if (data, labels) are in the same order or not.