I am a beginner in JAX and I am trying to learn how to train a neural network. I saw some blogs, but as I understood there isn't a library that you can train it easily, like 'fit' as in sklearn. I am interested about classification task, could you please reccommend me any blogs in order to adopt his/her algorithm into my problem?
JAX is an array manipulation library, not a deep learning library: in that respect, you should think of it as more similar to NumPy than similar to scikit-learn. If you want neural networks built on JAX, there are several other good projects available such as haiku and flax.
Related
I am new to machine learning, but I have decent experience in python. I am faced with a problem: I need to find a machine learning model that would work well to predict the speed of a boat given current environmental and physical conditions. I have looked into Scikit-Learn, Pytorch, and Tensorflow, but I am having trouble finding information on what type of model I should use. I am almost certain that linear regression models would be useless for this task. I have been told that non-parametric regression models would be ideal for this, but I am unable to find many in the Scikit Library. Should I be trying to use regression models at all, or should I be looking more into Neural Networks? I'm open to any suggestions, thanks in advance.
I think multi-linear regression model would work well for your case. I am assuming that the input data is just a bunch of environmental parameters and you have a boat speed corresponding to that. For such problems, regression usually works well. I would not recommend you to use neural networks unless you have a lot of training data and the size of one input data is also quite big.
I cannot get a satisfying answer to this question. As I understand it, TensorFlow is a library for numerical computations, often used in deep learning applications, and Scikit-learn is a framework for general machine learning.
But what is the exact difference between them, what is the purpose and function of TensorFlow? Can I use them together, and does it make any sense?
Your understanding is pretty much spot on, albeit very, very basic. TensorFlow is more of a low-level library. Basically, we can think of TensorFlow as the Lego bricks (similar to NumPy and SciPy) that we can use to implement machine learning algorithms whereas Scikit-Learn comes with off-the-shelf algorithms, e.g., algorithms for classification such as SVMs, Random Forests, Logistic Regression, and many, many more. TensorFlow really shines if we want to implement deep learning algorithms, since it allows us to take advantage of GPUs for more efficient training. TensorFlow is a low-level library that allows you to build machine learning models (and other computations) using a set of simple operators, like “add”, “matmul”, “concat”, etc.
Makes sense so far?
Scikit-Learn is a higher-level library that includes implementations of several machine learning algorithms, so you can define a model object in a single line or a few lines of code, then use it to fit a set of points or predict a value.
Tensorflow is mainly used for deep learning while Scikit-Learn is used for machine learning.
Here is a link that shows you how to do Regression and Classification using TensorFlow. I would highly suggest downloading the data sets and running the code yourself.
https://stackabuse.com/tensorflow-2-0-solving-classification-and-regression-problems/
Of course, you can do many different kinds of Regression and Classification using Scikit-Learn, without TensorFlow. I would suggesting reading through the Scikit-Learn documentation when you have a chance.
https://scikit-learn.org/stable/user_guide.html
It's going to take a while to get through everything, but if yo make it to the end, you will have learned a ton!!! Finally, you can get the 2,600+ page user guide for Scikit-Learn from the link below.
https://scikit-learn.org/stable/_downloads/scikit-learn-docs.pdf
The Tensorflow is a library for constructing Neural Networks. The scikit-learn contains ready to use algorithms. The TF can work with a variety of data types: tabular, text, images, audio. The scikit-learn is intended to work with tabular data.
Yes, you can use both packages. But if you need only classic Multi-Layer implementation then the MLPClassifier and MLPRegressor available in scikit-learn is a very good choice. I have run a comparison of MLP implemented in TF vs Scikit-learn and there weren't significant differences and scikit-learn MLP works about 2 times faster than TF on CPU. You can read the details of the comparison in my blog post.
Below the scatter plots of performance comparison:
Both are 3rd party machine learning modules, and both are good at it.
Tensorflow is the more popular of the two.
Tensorflow is typically used more in Deep Learning and Neural Networks.
SciKit learn is more general Machine Learning.
And although I don't think I've come across anyone using both simultaneously, no one is saying you can't.
R's package 'forecast' has a function nnetar, which uses feed-forward neural networks with a single hidden layer to predict in time series.
Now I am using Python to do the similar analysis. I want to use neural network which does not need to be as complex as deep learning. Maybe 2 layers and a couple of nodes are good enough for my case.
So, does Python have a model of simple neural networks which can be used in time series lik nnetar? If not, how to deal with this problem?
Any NN model that uses 1 or more hidden layers is a multi-layer perceptron model, and for that case it is trivial to make it extendable to N layers. So any library that you pick will support it. My guess for you not picking a complex library like pytorch/Tensorflow is its size.
Tensorflow does have TF-Lite which can work for smaller IOT devices.
Sklearn does have MLPRegressor that can train NNs if that is more to your liking.
You can always write your model. There are plenty of examples for this that use numpy and are plenty fast for cpu computation.( Single Hidden layer NN I am guessing will be more memory bound than computation bound)
Use another ML algorithm. Single Hidden layer NNs will not perform nearly as well as other other simpler algorithms.
If there are other reasons for not using a standard library like tensorflow/pytorch then you should mention them.
Are there any Deep Learning literature/references where they performed clustering in structured data?
I know it can be done using Kmeans, GMM etc. But is there any chance that cluster analysis to be done using Deep Neural Nets and the like? Thanks.
Neural networks can be used in a clustering pipeline. For example, you can use Self-organizing maps (SOMs) for dimensionality reduction and k-means for clustering. Also, auto-encoders directly pop to my mind. But then, again, it is rather compression / dimensionality reduction than clustering. The real clustering is done by something else.
The problem with clustering is the missing optimization goal. The problem is not well-defined.
Deep learning refers to the depth of the neural nets in and the huge number of parameters applied to learn how to recognize features related to a certain object, and neural nets in essence need a loss function to learn, and the loss should be in the form of an equation that can by applying calculus give an estimate of how much each parameter we need to correct to get better result (Basically forward propogation to predict and backward propogation to update parameters), and such a loss function as of now does not exist, so we don't use neural nets for clustering. And if no neural nets, no deep learning.
If any part of that seems confusing comment below.
To read more about clustering algorithm have a look at this https://towardsdatascience.com/the-5-clustering-algorithms-data-scientists-need-to-know-a36d136ef68
Again you will find no neural nets. :)
Yes.
If you do a little bit of literature research yourself you will find that people have repeatedly published clustering with deep neural networks.
Except that it doesn't seem to work anywhere but on MNIST data...
Two other Potential Methods:
KMeans + Autoencoder (a simple deep learning architecture with
kmeans and reducing the dimensionality of the data using autoencoders).
Deep Embedded Clustering algorithm (advanced deep learning)
I am using Boston Housing Data, I am using various model provided by keras and Sklearn to predict the house prices.
I want to know from an expert here on Stack Overflow, if I can use keras-rl or Re-inforcement learning to build a prediction model.
If so, how can i do so and integrate that in my NN code. I am looking over papers and kernels over kaggle and browsing, but unfortunately, could not find such implementation. The provided examples are solving games and balancing poles only, like playing games using RL.
TIA