I've recently begun to use the scipy.optimize minimisation library, and am experimenting with different algorithms.
I see there's good documentation for each algo individually on the docs page, but does anybody know of any intercomparison work that's been done (particularly with reference to those available to scipy)?
For instance, I'm finding that SLSQP is much faster than L-BFGS-B but often doesn't do quite as well..
Related
I'm trying to perform an evaluation of total floating-point operations (FLOPs) of a neural network.
My problem is the following. I'm using a sigmoid function. My question is how to eval the FLOPs of the exponential function. I'm using Tensorflow which relies on NumPy for the exp function.
I tried to dig into the Numpy code but didn't find the implementation ... I saw some subjects here talking about fast implementation of exponential but it doesn't really help.
My guess is that it would use a Taylor implementation or Chebychev.
Do you have any clue about this? And if so an estimation of the amount of FLOPs. I tried to find some references as well on Google but nothing really standardized ...
Thank you a lot for your answers.
I looked into it for a bit and what i found is that numpy indeed uses the C implementation as seen here.
Tensorflow though doesnt use nmpy implementation, instead it uses the scalar_logistics_opfunction from the C++ library called Eigen. The source for that can be found here.
Is there a python module that integrates simple chromatogram/trace analysis algorithms? I am looking for baseline correction, peak detection and peak integration functionality for simple time-courses (with data stored in numpy arrays).
I spent quite some time searching now and there doesn't seem to be any which really surprises me.
I'm not sure what analysis you are conducting but have you looked at PyLSS ?
It can (and I quote from the documentation):
PyLSS is able to compute:
LSS parameters (log kw and S)
Build and plot chromatograms from experimental/predicted retention times
Regarding peak detection and peak integration, I have used the functionality of the ptp() method in the Numpy module for this and I find that it is pretty powerful. Would this satisfy your requirement?
I am trying to find a equivalent of "NMaximize" optimization command in Mathematica in Python. I tried googling but did not help much.
The mathematica docs describe the methods usable within NMaximize as: Possible settings for the Method option include "NelderMead", "DifferentialEvolution", "SimulatedAnnealing", and "RandomSearch"..
Have a look at scipy's optimize which also supports:
NelderMead
DifferentialEvolution
and much more...
It is very important to find the correct tool for your optimization problem! This is at least dependent on:
Discrete variables?
Smooth optimization function?
Linear, Conic, Non-convex optimization problem?
and again: much more...
Compared to Mathematica's approach, you will have to choose the method a-priori within scipy (at some extent).
Following from this question, is there a way to use any method other than MLE (maximum-likelihood estimation) for fitting a continuous distribution in scipy? I think that my data may be resulting in the MLE method diverging, so I want to try using the method of moments instead, but I can't find out how to do it in scipy. Specifically, I'm expecting to find something like
scipy.stats.genextreme.fit(data, method=method_of_moments)
Does anyone know if this is possible, and if so how to do it?
Few things to mention:
1) scipy does not have support for GMM. There is some support for GMM via statsmodels (http://statsmodels.sourceforge.net/stable/gmm.html), you can also access many R routines via Rpy2 (and R is bound to have every flavour of GMM ever invented): http://rpy.sourceforge.net/rpy2/doc-2.1/html/index.html
2) Regarding stability of convergence, if this is the issue, then probably your problem is not with the objective being maximised (eg. likelihood, as opposed to a generalised moment) but with the optimiser. Gradient optimisers can be really fussy (or rather, the problems we give them are not really suited for gradient optimisation, leading to poor convergence).
If statsmodels and Rpy do not give you the routine you need, it is perhaps a good idea to write out your moment computation out verbose, and see how you can maximise it yourself - perhaps a custom-made little tool would work well for you?
Is there a Python library out there that solves for the Nash equilibrium of two-person zero-games? I know the solution can be written down in terms of linear constraints and, in theory, scipy should be able to optimize it. However, for two-person zero-games the solution is exact and unique, but some of the solvers fail to converge for certain problems.
Rather than listing any of the libraries on Linear programing on the Python website, I would like to know what library would be most effective in terms of ease of use and speed.
Raymond Hettinger wrote a recipe for solving zero-sum payoff matrices. It should serve your purposes alright.
As for a more general library for solving game theory, there's nothing specifically designed for that. But, like you said, scipy can tackle optimization problems like this. You might be able to do something with GarlicSim, which claims to be for "any kind of simulation: Physics, game theory..." but I've never used it before so I can't recommend it.
There is Gambit, which is a little difficult to set up, but has a python API.
I've just started putting together some game theory python code: http://drvinceknight.github.com/Gamepy/
There's code which:
solves matching games,
calculates shapley values in cooperative games,
runs agent based simulations to identify emergent behaviour in normal form games,
(clumsily - my python foo is still growing) uses the lrs library (written in C: http://cgm.cs.mcgill.ca/~avis/C/lrs.html) to calculate the solutions to normal form games (this is I believe what you want).
The code is all available on github and that site (the first link at the beginning of this answer) explains how the code works and gives user examples.
You might also want to check out 'Gambit' which I've never used.