I am not quite sure if I should ask this question here.
I want to make an art project.
I want to use voice as an input, and an image as output.
The image changes accordingly to the sound.
How can I realise this? Because I need realtime or a delay under 50 ms.
At first I thougt it would be better to use a micro controller.
But I want to calculate huge images, maybe I microcontroller can't do this.
For example I want to calculate 10.000 moving objects.
Could I realise this with windows/linux/mircocontroller?
It would be very good if I could use Python.
Or do you thing processing is a better choice?
Do you need more details?
Have you thought about using a graphical dataflow environment like Pure Data (Pd) or Max? Max is a commercial product, but Pd is free.
Even if you don't end up using Pd for your final project, it makes an excellent rapid prototyping tool. Whilst the graphics processing capabilities of Pd are limited, there are extensions such as Gridflow and Gem, which may help you. Certainly with Pd you can analyse incoming sound using the [fiddle~] object, which will give you the overall pitch and frequency/amplitude of individual partials and [env~], which will give you RMS amplitude. You could then very easily map changes in sound (pitch, amplitude, timbre) to various properties of an image such as colour, shapes, number of elements and so on in Gem or Gridflow.
10k moving objects sounds like a heck of a lot even on a modern desktop GPU! Calculating all of those positions on-the-fly is going to consume a lot of resources. I think even with a dedicated C++ graphics library like openFrameworks, this might be a struggle. You might want to consider an optimisation strategy like pre-rendering aspects of the image, and using the real-time audio control to determine which pre-rendered components are displayed at any given time. This might give the illusion of control over 10k objects, when in reality much of it is pre-rendered.
Good luck!
The above answer is a good one, PD is very flexible, but if you want something more code oriented and better suited to mixing with MCUs, processing might be better.
Another good way to do it would be to use Csound with the Csound Python API. Csound has a steep learning curve, but tons tons of audio analysis functionality and it's very good for running with low latency. You could definitely analyse an input signal in real time and then send control values out to a graphic environment, scripting both with Python.
Related
I noticed a lack of good soundfont-compatible synthesizers written in Python. So, a month or so ago, I started some work on my own (for reference, it's here). Making this was also a challenge that I set for myself.
I keep coming up against the same problem again and again and again, summarized by this:
To play sound, a stream of data with a more-or-less constant rate of flow must be sent to the audio device
To synthesize sound in real time based on user input, little-to-no buffering can be used
Thus, there is a cap on the amount of time one 'buffer generation loop' can take
Python, as a language, simply cannot run fast enough to do synthesize sound within this time limit
The problem is not my code, or at least, I've tried to optimize it to extreme levels - using local variables in time-sensitive parts of the code, avoiding using dots to access variables in loops, using itertools for iteration, using pre-compiled macros like max, changing thread switching parameters, doing as few calculations as possible, making approximations, this list goes on.
Using Pypy helps, but even that starts to struggle after not too long.
It's worth noting that (at best) my synth at the moment can play about 25 notes simultaneously. But this isn't enough. Fluidsynth, a synth written in C, has a cap on the number of notes per instrument at 128 notes. It also supports multiple instruments at a time.
Is my assertion that Python simply cannot be used to write a synthesizer correct? Or am I missing something very important?
I am a new user of Python and an amateur programmer in general - I am hoping to be able to filter a signal using just the numpy library. It will be programmed onto a BeagleBone Black and the OS is Angstrom Linux, so the furthest numpy library it will update to is 1.4 and due to either rumored data limitations (I am not actually sure how to check) or just the version of numpy being too early, scipy will not work on the board.
So the first solution is to get a new operating system but I would not know where to start; I am more comfortable in the realm of putting equations into a program.
I was hoping to use the filtfilt function but maybe it would be best to start with lfilter. This site seemed helpful for implementing it but it is a bit beyond me:
http://docs.scipy.org/doc/scipy-0.13.0/reference/generated/scipy.signal.lfilter.html
I am capable of getting the filter coefficients in MATLAB then transferring them to the BeagleBone. The x is just the array that is my signal which I can upload.
The second section is a bit of a jump - so is there a way to perform a z-transform in just numpy, not scipy? Also, based on all of the secrecy of the filter algorithm in MATLAB, I do not have faith in working that out, but is there some sort of mathematical algorithm description, or better yet code, describing how I may accomplish this?
Thanks for your patience in reading through this and the response. Please do not use complicated language in the response!
-Rob
For the filter design functions, you can copy the code from sicpy.signal.filter_design.py, they are almost pure python code.
But to do lfilter for IIR filters, you need a for loop for every sample in the data array. Since for loop in Python is slow, I think you need to implement it in C, and call it throught ctypes. Do you have a c compile in the target machine?
If you can design your filter as a FIR filter, then you can use numpy.convolve(b, x).
I'm writing an application to simulate train sounds. I got very short (0.2s) audio samples for every speed of the train and I need to be able to loop up to 20 of them (one for every train) without gaps at the same time.
Gapless changing of audio samples (train speed) is also a Must-Have.
I've been searching for possible python-audio-solutions, including
PyAudio
PyMedia
pyaudiere
but I'm not sure which one suits best my use-case, so I do really appreciate any propositions and experiences!
PS: I did already try out gstreamer but since the 1.0 release is not there yet and I cant figure out how to get gapless playback to work with pygi, i thought there might be a better choice. I also tried pygame, but it seems like it's limited to 8 audio channels??
I am using PyAudio for a lot of things and are quite happy with it. If it can do this, I do not know, but I think it can.
One solution is to feed sound buffer manually and control / set the needed latency. I have done this and it works quite well. If you have the latency high enough it will work.
An other solution, similar to this, is to manage the latency your self. You can queue up and or mix your small sound files manually to e.g. sizes of 0.5 -1 sec. This will greatly reduce the requirement to the "realtimeness" and alow you to do some pretty cool transitions between "speeds"
I do not know what sort of latency you can cope with, but if we are talking about train speeds, I guess they do not change instantaneous - hence latency of 500ms to several seconds is most likely acceptable.
I'm working on a modeling/reconstruction algorithm for point cloud data. So far I've been developing in Python, and been relatively happy with VPython for my visualization needs.
One problem I have is that VPython becomes quite slow when rendering a great many objects (at least on my non-3d accelerated Linux laptop), making visual inspection of complicated models quite difficult.
I've been trying to use an external tool for visualization, but the problem is that I'm a bit lost in the sea of possible file formats and available tools. I've been trying MeshLab for instance, which works great for displaying point cloud data in simple ascii formats, but I couldn't decide in which compatible format to export my other types of geometry, to superimpose on the point cloud layer.
Here are the requirements for my whole pipeline:
The point cloud data may contain millions of points, stored as simple xyz ascii coords
The modeling primitives are primarily lines and cylinders (i.e. no polygons), numbered in the thousands
The visualization tool should ideally be cross-platform (it must run at least on Linux)
There should be a Python module for easy data import/export of the chosen file format (or the format is simple enough to write a simple converter, if not)
I've been googling a lot about this so I have tentative answers for all of these, but none that is 100% satisfying in my context. Any help or advice would be greatly appreciated.. many thanks in advance!
I finally settled for Geomview: the viewer itself is powerful enough, and the many OOGL file formats that it implements answer my needs. I use the .off format for point cloud data, and .skel for my other modeling primitives. These file formats are also human-readable, which makes writing import/export functions easy.
How about Panda3D? It's cross-platform, and it should be able to handle rendering millions of points as long as you have a decent graphics card.
I do a lot of statistical work and use Python as my main language. Some of the data sets I work with though can take 20GB of memory, which makes operating on them using in-memory functions in numpy, scipy, and PyIMSL nearly impossible. The statistical analysis language SAS has a big advantage here in that it can operate on data from hard disk as opposed to strictly in-memory processing. But, I want to avoid having to write a lot of code in SAS (for a variety of reasons) and am therefore trying to determine what options I have with Python (besides buying more hardware and memory).
I should clarify that approaches like map-reduce will not help in much of my work because I need to operate on complete sets of data (e.g. computing quantiles or fitting a logistic regression model).
Recently I started playing with h5py and think it is the best option I have found for allowing Python to act like SAS and operate on data from disk (via hdf5 files), while still being able to leverage numpy/scipy/matplotlib, etc. I would like to hear if anyone has experience using Python and h5py in a similar setting and what they have found. Has anyone been able to use Python in "big data" settings heretofore dominated by SAS?
EDIT: Buying more hardware/memory certainly can help, but from an IT perspective it is hard for me to sell Python to an organization that needs to analyze huge data sets when Python (or R, or MATLAB etc) need to hold data in memory. SAS continues to have a strong selling point here because while disk-based analytics may be slower, you can confidently deal with huge data sets. So, I am hoping that Stackoverflow-ers can help me figure out how to reduce the perceived risk around using Python as a mainstay big-data analytics language.
We use Python in conjunction with h5py, numpy/scipy and boost::python to do data analysis. Our typical datasets have sizes of up to a few hundred GBs.
HDF5 advantages:
data can be inspected conveniently using the h5view application, h5py/ipython and the h5* commandline tools
APIs are available for different platforms and languages
structure data using groups
annotating data using attributes
worry-free built-in data compression
io on single datasets is fast
HDF5 pitfalls:
Performance breaks down, if a h5 file contains too many datasets/groups (> 1000), because traversing them is very slow. On the other side, io is fast for a few big datasets.
Advanced data queries (SQL like) are clumsy to implement and slow (consider SQLite in that case)
HDF5 is not thread-safe in all cases: one has to ensure, that the library was compiled with the correct options
changing h5 datasets (resize, delete etc.) blows up the file size (in the best case) or is impossible (in the worst case) (the whole h5 file has to be copied to flatten it again)
I don't use Python for stats and tend to deal with relatively small datasets, but it might be worth a moment to check out the CRAN Task View for high-performance computing in R, especially the "Large memory and out-of-memory data" section.
Three reasons:
you can mine the source code of any of those packages for ideas that might help you generally
you might find the package names useful in searching for Python equivalents; a lot of R users are Python users, too
under some circumstances, it might prove convenient to just link to R for a particular analysis using one of the above-linked packages and then draw the results back into Python
Again, I emphasize that this is all way out of my league, and it's certainly possible that you might already know all of this. But perhaps this will prove useful to you or someone working on the same problems.