We Keep Coding

How to determine if data point was placed in training or testing set?

I'm using train_test_split to split image data for a convolutional neural network in Python:
x_train, x_test, y_train, y_test = train_test_split(X, Y)
For each image in X, how can I figure out whether it was sent to the x_train or x_test set? Since all the data in the x_train or x_test datasets are in tensor form and randomized, I'm not sure how to relate a given instance in x_train/x_test back to its original place in X. My confusion matrix is printing inconsistent information, so I'm trying to figure out if the way the data is split being training and testing is the reason.
Edit 1: Folder Structure
All the images are in one array (X = np.array(X_images)) which I derived from collecting image from folders such that:
Data
Class_1
Class_2
...
Class_n
I then used: Y = np_utils.to_categorical(labels, num_classes) to get the Y values

If you are able to re-run the experiment, try generating indicates instead of raw arrays. Then use indicates to extract train and dev sets.
# this is slightly modified example from the sklearn documentation:
import numpy as np
from sklearn.model_selection import KFold
X = np.array([["a", "b"], ["c", "d"], ["e", "f"], ["g", "h"]])
y = np.array(["a1", "c1", "e1", "g1"])
kf = KFold(n_splits=2)
for train_index, test_index in kf.split(X):
print("indicates", "TRAIN:", train_index, "TEST:", test_index)
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]

train_test_split takes as arguments an arbitrary number of arrays/vectors, so you could just pass an additional list/array containing some identifier to that call, e.g. X_train, X_test, y_train, y_test, id_train, id_test = train_test_split(X, y, ids), where ids is some list/array containing the identifiers corresponding to each element in X/y. Then, the data point at index i in X_train/y_train will correspond to the identifier at id_train[i], and so on for the "test" data. If you don't have a row-identifier column handy, you could just use the index of X, e.g. ids = list(range(X.shape[0])).

The following solved my problem. I created a numpy array from the original image data:
indices = np.arange(X.shape[0])
Fed this into the train_test_split function call and added two more that created an index corresponding to the image's respective X index:
x_train, x_test, y_train, y_test, x_train_ind, x_test_ind = train_test_split(X, Y, indices, test_size=0.2, random_state=2)
After getting the index of an image in the x_train dataset, we can plug that into x_train_ind to get the index in the original X dataset

How to partition a dataset into three equal parts?

I am trying to divide my dataset into three equal parts by using scikit-learn. But when I use StratifiedKFold (on sklearn) to do it, it only shows me the command that I did for partition the dataset, rather than the result:
from sklearn.model_selection import StratifiedKFold
partition = StratifiedKFold(n_splits = 3, shuffle = True, random_state = None)
print(partition)
I am still new with Python libraries, so I am not sure about how to do it.

The second line of your code creates a StratifiedKFold object, it does not really partition your data. It is this object that you should use to split your data (see example below)
partition = StratifiedKFold(n_splits = 3, shuffle = True, random_state = 1)
for train_index, test_index in partition.split(x, y):
x_train_f, x_test_f = x[train_index], x[test_index]
y_train_f, y_test_f = y[train_index], y[test_index]
Your answer for splitting your data in 3 parts has been answered here
X_train, X_test, X_validate = np.split(X, [int(.7*len(X)), int(.8*len(X))])
y_train, y_test, y_validate = np.split(y, [int(.7*len(y)), int(.8*len(y))])

TensorFlow: How to validate on 1 specific row of data and train on the rest?

I have 11 rows of data, and my goal is to train the network on 10, and validate on 1 specific row (not random).
The aim is to work through validating on each single row while training on the other 10, until I have a prediction for all 11 rows.
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.1)
The train/test split as shown above doesn't seem like it will work as it is random, is there a way to specify exactly which rows are to be used for training and testing?

What you are looking for seems to be k-fold cross validation. This will use each row as a validation set, and train on the remaining k - 1 rows and so forth. I would suggest using sklearn's built-in method.
from sklearn.model_selection import KFold
n_splits = 11
for train_idx, test_idx in KFold(n_splits).split(x):
x_train, x_test = x[train_idx], x[test_idx]
y_train, y_test = y[train_idx], y[test_idx]
# do your stuff

Split dataset into training and test by month

I was not able to find the answer to this anywhere. I have data for three months, where I would like to split it into the first two months('Jan-19', 'Feb-19') as training set and the last month as the test ('Mar-19').
Previously I have done random sampling with simple code like this:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30,random_state=109)
and before that assigned y as the label and x as the columns to use to predict. I'm not sure how to assign the test and training to the months I want.
Thank you

If your data is in a pandas dataframe, you can use subsetting like this:
X_train = X[X['month'] != 'Mar-19']
y_train = y[X['month'] != 'Mar-19']
X_test = X[X['month'] == 'Mar-19']
y_test = y[X['month'] == 'Mar-19']

You try this option and see if it helps.
dataset_train = df['2004-02-12 11:02:39':'2004-02-13 23:52:39']
dataset_test = df['2004-02-13 23:52:39':]

create training validation split using sklearn

I have a training set consisting of X and Y, The X is of shape (4000,32,1) and Y is of shape (4000,1).
I would like to create a training/validation set based on split. Here is what I have been trying to do
from sklearn.model_selection import StratifiedShuffleSplit
sss = StratifiedShuffleSplit(test_size=0.1, random_state=23)
for train_index, valid_index in sss.split(X, Y):
X_train, X_valid = X[train_index], X[valid_index]
y_train, y_valid = Y[train_index], Y[valid_index]
Running the program gives the following error message related to the above code segment
for train_index, valid_index in sss.split(X, Y):
ValueError: The least populated class in y has only 1 member, which is too few. The minimum number of groups for any class cannot be less than 2.
I am not very clear about the above error message, what's the right way to create a training/validation split for the training set as above?

It's a little bit weird because I copy/pasted your code with sklearn's breast cancer dataset as follow
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
X, Y = cancer.data, cancer.target
from sklearn.model_selection import StratifiedShuffleSplit
sss = StratifiedShuffleSplit(test_size=0.1, random_state=23)
for train_index, valid_index in sss.split(X, Y):
X_train, X_valid = X[train_index], X[valid_index]
y_train, y_valid = Y[train_index], Y[valid_index]
Here X.shape = (569, 30) and Y.shape = (569,) and I had no error, for example y_valid.shape = 57 or one tenth of 569.
I suggest you to reshape X into (4000,32) (and so Y into (4000)), because Python may see it as a list of ONE big element (I am using python 2-7 by the way).
To answer your question, you can alternatively use train_test_split
from sklearn.model_selection import train_test_split
which according to the help
Split arrays or matrices into random train and test subsets Quick utility that wraps input validation and
``next(ShuffleSplit().split(X, y))`
Basically a wrapper of what you wanted to do. You can then specify the training and the test sizes, the random_state, if you want to stratify your data or to shuffle it etc.
It's easy to use for example:
X_train, X_valid, y_train, y_valid = train_test_split(X,Y, test_size = 0.1, random_state=0)