I've trained a decision tree on a dataset (handwritten) which contains 8 x-y points sampled along the length of the number (number digit dataset). The test dataset given (assignment), is the MNIST dataset, which is the pixel intensities in a 28x28 bitmap image. I need to sample 8 points and along the trajectory of the number so that it performs well.
I'm doing this in Python. I don't know what to do with the image to sample those points. Any package/procedure will help.
Simply index the array as you would any other array. The pixel intensities are merely ints. e.g val = arr[3,9].
You do not have the stroke direction in mnist.
Hence, you cannot reliably infer such positions.
You can do the opposite though: render the stroke information as pixel image, train a classifier on that, and then test it with mnist.
There's an MNIST sequence dataset from Edwin de Jong:
Paper: https://arxiv.org/pdf/1611.03068.pdf
Github: https://edwin-de-jong.github.io/blog/mnist-sequence-data/
Blog: https://github.com/edwin-de-jong/mnist-digits-as-stroke-sequences/
and an MNIST classification using RNN by Ryan Epp:
https://www.ryanepp.com/blog/mnist-classification-using-stroke-paths
In both projects, the direction strokes will take at a T-junction depends on the algorithm and is often counterintuitive. This means that there's more to learn for sequences since many stroke patterns will produce the same image.
Related
My goal is to identify true 4K images from images that have been upscaled from a lower resolution with methods such as Bicubic, Billinear, Lanczos, EDSR and so on.
I have a dataset of 200 4K images that I have downscaled to 1080p and back to 4K with Lanczos and Bicubic interpolation.
My idea is to use the BRISQUE features of these images to identify if the images have been upscaled or not.
This is what I currently have:
I used the python implementation of BRISQUE to get the BRISQUE features of 3 same images and scaled them;
import brisque
brisq = BRISQUE()
features = brisq.get_feature('path_to_img')
features = brisq._scale_feature(features)
# This gives me a vector of [36,], some examples shown in code block below
2 of them have been upscaled from a resolution of 1080p as labelled. From the density plot, we can clearly see that there is a difference between true 4K images and upscaled images.
My question is; Now what? I have these features, and we can see the distinction from the density plot, but I have not been able to find a way to classify these images. I am open to any alternative methods that I can use. I will greatly appreciate any help/suggestions!
In the actual BRISQUE implementation, they have used a Support Vector Regression model to return a quality score of [0,100]. However, what I want to be able to do is for a model to be able to classify the image as upscaled/original based on these features; a vector of [36,]
What I've tried:
I tried using a Support Vector Classifier to classify these images, but I must have done something wrong as all images were classified as Upscaled. My guess is that it is due to the features being too close to one another and not dense in different areas for upscaled/original. So it seems like SVC is definitely not the way to go.
I tried adding the abs() of all the features, hoping that this would allow me to have a scatter plot of 2 dense clusters, but the features are still scattered around.
clf = svm.SVC(gamma='auto')
X = np.asarray(X)
X = X.squeeze()
y = np.asarray(y)
clf = clf.fit(X,y.ravel())
print(X)
# These are the scaled feature vectors of [36,] y contains my categories Upscaled/Original
[[-0.65742082 -0.60651108 -0.38406828 ... 0.13391201 -0.77064699
-0.66440166]
[-0.67936245 -0.66312799 -0.40825036 ... -0.04571298 -0.75259527
-0.72149044]
[-0.6176775 -0.3162819 -0.25604552 ... -0.08188693 -0.22914459
-0.04314284]
...
[-0.58745601 -0.65824511 -0.31152205 ... 0.53725558 -0.73736713
-0.40638184]
[-0.65079694 -0.84827717 -0.41251778 ... 0.40268912 -0.94145548
-0.83813568]
[-0.64831298 -0.74385767 -0.41820768 ... 0.38536 -0.83109257
-0.6435719 ]]
I am sort of having a problem on how I should represent network for training my CNN. The input data consists of different sized images, in which the number of rows is constant (441 rows) and the dimension of the color is constant (RGB), but the number of columns is different. The CNN are supposed to generate a feature vector, in which the length of feature is dependent of the number of columns each image has.
Example:
An image shaped (441,300,3) should generate a feature vector of length 98
An image shaped (441,1209,3) should generate a feature vector of length 398
So nearly each column should generate 3 features.
Is it possible to make convolution using kernel that fill the whole column and give out 3 features? The reason why I am seeking for a kernel that fill the whole column, as I want to apply more importance on some areas of the columns, instead of giving all the area equal importance.
If so, how would go by designing such kernel or such a network in keras?
I have a twitter corpus which I am using to build sentiment analysis application. The corpus has 5k tweets which have been hand labelled as - negative, neutral or positive
To represent the text - I am using gensim word2vec pretrained vectors. Each word is mapped to 300 dimensions. For a tweet, I add all the word vectors to get a single 300 dim vectors. Thus every tweet is mapped to a single vector of 300 dimension.
I am visualizing my data using t-SNE (tsne python package). See attached image 1 - Red points = negative tweets, Blue points = neutral tweets and Green points = Positive tweets
Question:
In the plot there no clear separation (boundary) among the data points. Can I assume this will also be the case with the original points in 300 Dimensions ?
i.e if points overlap in t-SNE graph then they also overlap in original space and vice-versa ?
Question: In the plot there no clear separation (boundary) among the data points. Can I assume this will also be the case with the original points in 300 Dimensions ?
In most cases NO. By reducing dimensions you will probably loose some information.
The case where you may reduce dimension without losing information is when or data in some dimensions is zero(for example line in 3dimensional space) or when some dimensions linearly dependent on other.
There are few tricks to test how good some dimensionality reductions techniques works. For example:
You may use PCA to reduce dimension form 300 to for example 10. You can calculate sum of 300 eigenvalues(original space) and sum of 10 biggest eigenvalues(these 10 eigenvalues represent eigenvectors that will be used for dimension reduction) and calculate percentage of lost information sum(top-10-eigenvalues)/sum(300-eigenvalues) .This value is not exactly "information" lost, but it is close to that.
I'm implementing a CNN with Theano. In the paper, I have to do this image preprocess before train the CNN
We extracted RGB patches of 61x61 dimensions associated with each poselet activation, subtracted the mean and used this data to train the convnet model shown in Table 1
Can you tell me what does it mean with "subtracted the mean"? Tell me if these steps are correct (it is what I understood)
1) Compute the mean for Red Channel, Green Channel and Blue Channel for the whole image
2) For each pixel, subtract from red value the mean of red channel, from green value the mean of green channel and the same for the blue channel
3) Is it correct to have negative value or do I have use the abs?
Thanks all!!
You should read paper carefully, but what is the most probable is that they mean mean of the patches, so you have N matrices 61x61 pixels, which is equivalent of a vector of length 61^2 (if there are three channels then 3*61^2). What they do - they simple compute mean of each dimension, so they calculate mean over these N vectors in respect to each of the 3*61^2 dimensions. As the result they obtain a mean vector of length 3*61^2 (or mean matrix/mean patch if you prefer) and they substract it from all of these N patches. Resulting patches will have negatives values, it is perfectly fine, you should not take abs value, neural networks prefer this kind of data.
I would assume the mean mentioned in the paper is the mean over all images used in the training set (computed separately for each channel).
Several indications:
Caffe is a lib for ConvNets. In their tutorial they mention the compute image mean part: http://caffe.berkeleyvision.org/gathered/examples/imagenet.html
For this they use the following script: https://github.com/BVLC/caffe/blob/master/examples/imagenet/make_imagenet_mean.sh
which does what I indicated.
Google played around with ConvNets and published their code here: https://github.com/google/deepdream/blob/master/dream.ipynb and they do also use the mean of the training set.
This is of course only indirect evidence since I can not explain you why this happens. In fact I stumbled over this question while trying to figure out precisely that.
//EDIT:
In the mean time I found a source confirming my claim (Highlighting added by me):
There are three common forms of data preprocessing a data matrix X [...]
Mean subtraction is the most common form of preprocessing. It
involves subtracting the mean across every individual feature in the
data, and has the geometric interpretation of centering the cloud of
data around the origin along every dimension. In numpy, this operation
would be implemented as: X -= np.mean(X, axis = 0). With images
specifically, for convenience it can be common to subtract a single
value from all pixels (e.g. X -= np.mean(X)), or to do so separately
across the three color channels.
As we can see, the whole data is used to compute the mean.
I want to train a SVM for object detection. At this point I have a python script which detects FAST keypoints and extracts BRIEF features at that location.
Now I don't know how to use these descriptors to train a SVM.
Would you tell me please:
How to use the descriptors to train the SVM (As far as I know these descriptors should be my train data)?
What are labels used for and how I can get them?
To train a SVM you would need a matrix X with your features and a vector y with your labels. It should look like this for 3 images and two features:
>>> from sklearn import svm
>>> X = [[0, 0], <- negative 0
[1, 3], <- positive 1
2, 5]] <- negative 0
>>> y = [0,
1,
0]
>>> model = svm.SVC()
>>> model.fit(X, y)
The training set would consist of several images, each image would be a row of X and y.
Labels:
For the labels y you need positive and negative examples (0 or 1):
Positive Samples
You can specify positive samples in two ways. One way is to specify
rectangular regions in a larger image. The regions contain the objects
of interest. The other approach is to crop out the object of interest
from the image and save it as a separate image. Then, you can specify
the region to be the entire image. You can also generate more positive
samples from existing ones by adding rotation or noise, or by varying
brightness or contrast.
Negative Samples
Images that do not contain objects of interest.
[slightly edited from here]
Feature matrix X:
Here you can get creative but I will mention a simple idea. Make height * width features, one for each pixel of each image, but make them all 0 except in a small region around the FAST keypoints. In the end your X matrix will have dimension (n_images, height*width).
Another commonly used idea is Bag of Words. The X matrix must have a fixed number of features/columns and the number of keypoints is variable. This is a representation problem but it can be solved binning them in a histogram with a fixed number of bins. For details see for example this paper.
You will have to consult the specialized literature to come up with more ways to incorporate the BRIEF features but I hope this will give you an idea on how to get started.