Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm looking to do k-means clustering on an Google App Engine app (built for Google Glass). The App Engine only accepts pure Python libraries.
I have tried using SciPy's kmeans and kmeans2 edited to use py_vq instead of the C implementation of vq, but they take too long (~100 seconds) on ~500,000 x,y,z points and I get a DeadlineExceededError in my App Engine Logs.
I have already reduced my sample size by 90% to save on the memory limit, so I would like to shy away from reducing my sample size any more.
Are there any pure Python k-means libraries that are fast enough for Google App Engine?
If you get a DeadlineExceededError, consider using a backend. Backends have no deadline and a configurable memory limit (not very high though, 1GB max).
See: https://developers.google.com/appengine/docs/python/backends/
Well, the C implementation will certainly be faster than a pure python implementation...
For numerical stuff, pure python can be really expensive and slow. That is why the python libraries such as numpy and scipy have a lot of stuff implemented in faster languages in the backend.
You can also try to limit the number of iterations. The k-means result after 10 iterations will likely be as good as the final result in practise. It's the first iterations where things change a lot.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I'm trying put my optimization problem into Pyomo, but it is strongly dependent upon standard linear algebra operations - qr, inverse, transpose, product. Actually, this is Kalman filter problem; recursive linear algebra for long time series. I failed to find pyomo functions to implement it like I could in tensor flow. Is it possible?
Connected questions:
Am I right that numpy target function is practically not usable in pyomo?
Is there a better free optimization solution for the purpose? (scipy cannot approach efficiency of Matlab by far, tensor flow is extremely slow for particular problem, though I do not see why, algorithmic differentiation in Matlab was reasonably fast though not fast enough)
Many thanks,
Vladimir
Pyomo is mainly a package for optimization. i.e. specifying data -> building problem -> sending to the solver -> wait for solver's results -> retrieving solution. Even if it can handle matrix-like data, it cannot manipulate it with matrix operations. This should be done using a good external library, before you send your data to Pyomo. Once you have all your matrixes ready to be used as data in your optimization model, then you can use Pyomo for optimization.
That being said, you should look into finding a library that fits your needs to build your data, since your data values must be static, once you provide it as an input to your model.
Also, keep in mind that Pyomo, like any optimization tools, is deterministic. It is not meant to do data analysis or data description, but to provide a way to find one optimal solution of a mathematical problem. In your case, Pyomo is not meant to do the Kalman filter problem, but to give you the solution of minimizing the mean square error.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
We can perform noise reduction using Open-source Software like Audacity, which is commonly used for the purpose. Please click the below link for reference.
denoising with audacity image
Is there a python library that can perform a similar function?
If you want to reduce noise the audacity way, to my understanding, you should program your algorithm using scipy filters provided by scipy library.
Besides that pyaudio is one dedicated library for audio analysis and here is a kickstart tutorial.
If you are not restricted only to Python, you can check out on Essentia. This is by far an exhaustive library for music and audio analysis.
Nutshell: While python libraries provide functionalities, it is you who should code your noise reduction algorithm (tailored to your needs). May be you can follow the audacity's approach.
You can refer this question for better, technical/implementation, clarity: Noise reduction on wave file
Good luck! Try to be precise and post questions focusing on implementation pertaining to programming languages rather than generic things.
As a general guideline:
Understand the behavior of your noise and then you can choose your noise removal strategy accordingly.May be you need a simple low pass filter or high-pass filter.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I am trying to determine the following step characteristics for a step response in Python:
RiseTime
SettlingTime
SettlingMin
SettlingMax
Overshoot
Undershoot
Peak
PeakTime
Matlab offers me the function stepinfo, but I am unable to find a suitable alternative in Python. I did try to roll my own using numpy and scipy, but I haven't had much luck yet, my knowledge of signal processing is lacking.
Most information that I can find on the internet look rather complicated but I do like to learn more about this. If any one could recommend me a good book or other source to learn more from I would appreciate it! Thank you!
This is the step response that I currently have:
This discussion suggests a sort of implementation:
def step_info(t,yout):
print "OS: %f%s"%((yout.max()/yout[-1]-1)*100,'%')
print "Tr: %fs"%(t[next(i for i in range(0,len(yout)-1) if yout[i]>yout[-1]*.90)]-t[0])
print "Ts: %fs"%(t[next(len(yout)-i for i in range(2,len(yout)-1) if abs(yout[-i]/yout[-1])>1.02)]-t[0])
Then you need to use numpy functions in the Signal Processing section to get the other information that you want.
Could you not just implement the formulas? (Assuming that this is a second order system / has two dominant poles and can be approximated as second order)
For rise and settling time there are a few different approximations, so the internet is your friend.
You could also figure out the damped frequency (from the maxima and minima of your plot data), and use that to figure out the natural frequency:
There are a handful of formulas that relate these various quantities, depending on what you know.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm working on a network inspection tool which allows users to match regex over network flows. Its working fine and now I would like to add graphing functionality to it. This is to provide the user a dot graph that represents the internal state machine of the regex.
I found interactive regex debugger and visualizer quite useful and want to have mostly similar Python-based functionality to generate the graph. I've a tool that accepts a custom regex language and converts the expression to such a graph. Since its not compatible with Python re expressions, I can not use it.
Any pointers to how I can convert Python regex to FA state machines?
If you allow the full spectrum of Python regular expressions, this can't be done. The reason is that some expressions supported by re are not regular at all and cannot be represented by a finite automaton. One simple example is r"(a+)b\1", which matches some number of a's, a b, and then the same number of a's as before (see Regular expression implementation details). There is no DFA that admits this expression.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
What Python libraries are recommended to complement with scikit learn (a machine learning library)?
I have .wav files that I would like to represent as feature vectors, so that I could perform audio recognition.
Is scikit.audiolab a good candidate?
It would be highly appreciated if a sample code or a reference is given, which reads a .wav file to a feature vector :).
Thanks in advance!
If I'm not mistaken, scikit.audiolab is merely for reading/writing audio files but I think in addition you'll want to look at the signal processing libraries in scipy to actually build your feature vectors.
http://docs.scipy.org/doc/scipy/reference/signal.html