Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I recently got interested in soccer statistics. Right now I want to implement the famous Dixon-Coles Model in Python 3.5 (paper-link).
The basic problem is, that from the model described in the paper a Likelihood function with numerous parameters results, which needs to be maximized.
For example: The likelihood function for one Bundesliga season would result in 37 parameters. Of course I do the minimization of the corresponding negative log-likelihood function. I know that this log function is strictly convex so the optimization should not be too difficult. I also included the analytic gradient, but as the number of parameters exceeds ~10 the optimization methods from the SciPy-Package fail (scipy.optimize.minimize()).
My question:
Which other optimization techniques are out there and are mostly suited for optimization problems involving ~40 independent parameters?
Some hints to other methods would be great!
You may want to have a look at convex optimization packages like https://cvxopt.org/ or https://www.cvxpy.org/. It's Python-based, hence easy to use!
You can make use of Metaheuristic algorithms which work both on convex and non-convex spaces. Probably the most famous one of them is Genetic algorithm. It is also easy to implement and the concept is straightforward. The beautiful thing about Genetic algorithm is that you can adapt it to solve most of the optimization problems.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
Recently I have completed ML course in coursera by Andrews Ng. It's an awesome course. I was working with octave through out the course. But, python is much popular when compared to octave. So, I have started to learn python now. I was implementing linear regression using python. In that I am doing nothing. Simply calling the predefined function for linear regression. But, in octave I used to write the code from scratch. I have to find parameters using gradient descent algorithm. But, no such things in python. I have referred the following link:
https://towardsdatascience.com/linear-regression-python-implementation-ae0d95348ac4
My question is, won't we use any algorithms like gradient descent to learn parameter Theta? Is everything is predefined in python?
Thanks.
Python is a programming language, just like Octave. So everything that can be done in Octave can be done using Python too. If you want to implement Linear Regression algorithm from scratch using Python in order to validate your understanding, of course you can do it (I have done it too). Why stop at Linear Regression, you can implement SVM, Decision Trees or even Deep Neural Networks from scratch in Python. And it is a good way to gain concrete understanding of these algorithms.
However, over the years all these have been implemented in Python in libraries like Sklearn etc. So as the complexity and volume of data increases, you would want to use one of these libraries or frameworks. Why? Because these are highly optimized implementations. To get high level feeling - implement Linear Regression using simple list and for loops, and then vectorize it with Numpy, you will see the difference in performance.
So to summarize - if you are curious, go ahead and implement the algorithms from scratch to gain solid understanding. As complexity and data volume will increase, start using the libraries and frameworks. Hope this helps.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I'm trying put my optimization problem into Pyomo, but it is strongly dependent upon standard linear algebra operations - qr, inverse, transpose, product. Actually, this is Kalman filter problem; recursive linear algebra for long time series. I failed to find pyomo functions to implement it like I could in tensor flow. Is it possible?
Connected questions:
Am I right that numpy target function is practically not usable in pyomo?
Is there a better free optimization solution for the purpose? (scipy cannot approach efficiency of Matlab by far, tensor flow is extremely slow for particular problem, though I do not see why, algorithmic differentiation in Matlab was reasonably fast though not fast enough)
Many thanks,
Vladimir
Pyomo is mainly a package for optimization. i.e. specifying data -> building problem -> sending to the solver -> wait for solver's results -> retrieving solution. Even if it can handle matrix-like data, it cannot manipulate it with matrix operations. This should be done using a good external library, before you send your data to Pyomo. Once you have all your matrixes ready to be used as data in your optimization model, then you can use Pyomo for optimization.
That being said, you should look into finding a library that fits your needs to build your data, since your data values must be static, once you provide it as an input to your model.
Also, keep in mind that Pyomo, like any optimization tools, is deterministic. It is not meant to do data analysis or data description, but to provide a way to find one optimal solution of a mathematical problem. In your case, Pyomo is not meant to do the Kalman filter problem, but to give you the solution of minimizing the mean square error.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
Is it better to implement my own K-means Algorithm in Python or use the pre-implemented K-mean Algorithm in Python libraries like for example Scikit-Learn?
Before answering which is better, here is a quick reminder of the algorithm:
"Choose" the number of clusters K
Initiate your first centroids
For each point, find the closest centroid
according to a distance function D
When all points are attributed to a cluster, calculate the barycenter of the cluster which become its new centroid
Repeat step 3. and step 4. until convergence
As stressed previously, the algorithm depends on various parameters:
The number of clusters
Your initial centroid positions
A distance function to calculate distance between any point and centroid
A function to calculate the barycenter of each new cluster
A convergence metric
...
If none of the above is familiar to you, and you want to understand the role of each parameter, I would recommend to re-implement it on low-dimensional data-sets. Moreover, the implemented Python libraries might not match your specific requirements - even though they provide good tuning possibilities.
If your point is to use it quickly with a big-picture understanding, you can use existing implementation - scikit-learn would be a good choice.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I wanted to apply one of the minimizations methods within sicpy.minimize to a function which may not always provide smooth derivatives. I've gotten comfortable with the Nelder-Mead implementation of the Simplex method, but it does not appear to accept the bounds argument: (...,bounds=[xmin, xmax],...). Reading this documentation it seems only L-BFGS-B, TNC and SLSQP methods accept bounds, and all three of those are based in some way upon Newton's method, and will either calculate a numerical derivative or accept one.
I don't know the exact term, but I'm looking for a 'Simplex-like' or 'derivativeless' method in scipy that accepts bounds, but will also be forgiving of functions that will not provide a smooth derivative (one example being staircase-like behavior). For now, I'm doing 1d. Later I may add dimensions, but that's not critical right now.
I would give lmfit a try (http://cars9.uchicago.edu/software/python/lmfit/).
While not part of scipy, but based on, it offers bounded minimization. I use it for curve-fitting and parameters extraction. Nevertheless, I couldn't tell how it would perform on your specific function.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
for example:
a=1500
b=[500,400,200]
One answer is:
ans=[1,2,1]
because 1*500+2*400+1*200=1500 I want to write a program with genetic algorithm with best evaluation function to solve this problem with this array with pyevolve python evolutionary tool.
Assuming that the coefficients in the answer must be integers, what you're describing is a linear Diophantine equation. It's not a good fit for a genetic algorithm, as the solution space is neither continuous nor smooth. (That is, there is not always a possible input between any two other inputs, and the "correct" answer will not necessarily be anywhere near other nearly-correct inputs.)
(If the coefficients in the answer can be real numbers, finding a solution is trivial to the point that a genetic algorithm would be overkill.)