I've been getting really confused with FFT in python. What I'm trying to do is plot the FFT of the note number 61 (or middle C#). Here is the code that I tried to use which I found here using this wav file. After running that code, I got this output after zooming in a bit.
I think that this is completely wrong due to the fact that after looking online, the note number 61 has a frequency of 277.2hz. This means that there should be a peak around that value right? But to me, it seems like that the values are completely off. This is the code that I'm running right now to get the plot.
import matplotlib.pyplot as plt
from scipy.fftpack import fft
from scipy.io import wavfile # get the api
fs, data = wavfile.read("MAPS_ISOL_NO_P_S0_M61_AkPnBsdf.wav") # load the data
a = data.T[0] # this is a two channel soundtrack, I get the first track
b=[(ele/2**8.)*2-1 for ele in a] # this is 8-bit track, b is now normalized on [-1,1)
c = fft(b) # create a list of complex number
d = len(c)/2 # you only need half of the fft list
plt.plot(abs(c[:(d-1)]),'r')
plt.xlabel('Frequency')
plt.ylabel('Magnitude')
plt.show()
I'm also not sure if I have the axis labeled correctly for the x and y axis as I believe each entry in the array is a bin of size Fs / N where Fs is the sample rate and N is the size of the FFT? I'm just really confused and overwhelmed after looking online for weeks about all this. Thanks for any help!
Related
I'm try to produce a tone that transitions linearly between a list of arbitrary frequencies with respect to time. I'm using scipy.signal to create waves that transition between pairs of frequencies, then concatenating them. When the frequencies are round numbers and relatively far from each other this works. When the numbers are close together or aren't as nice I get pops between each transition.
What is causing these pops, why only in the first case and not the other two, and what can I do to fix it? (Or if there's a better/easier way to do what I'm trying to do, what is it?)
Any thoughts would be very much appreciated.
from scipy.signal import sweep_poly
import numpy
import sounddevice
sample_rate = 44100.0
time = 1.0
amplitude = 10000
sounddevice.default.samplerate = sample_rate
freq_list = [100, 200, 100, 200]
#freq_list = [100.05,200.21,100.02,200.65,100.16]
#freq_list = [100,101,102,103]
#get samples for each segment
samples = numpy.arange(sample_rate * time) / sample_rate
#make all the different segments
wave_list = []
for i in range(len(freq_list)-1):
wave_list.append(amplitude * sweep_poly(samples, [float(freq_list[i+1])-float(freq_list[i]),float(freq_list[i])]))
#join them together
wave = numpy.concatenate(wave_list)
#convert it to wav format (16 bits)
wav_wave = numpy.array(wave, dtype=numpy.int16)
sounddevice.play(wav_wave, blocking=True)
My comment was correct. The problem was that when I made a new wave it always started at its peak, which didn't necessarily line up with the old wave, resulting in discontinuities:
I fixed this by setting the phase offset parameter of sweep_poly to (180/math.pi)*math.acos(prev_point/amplitude) where prev_point was the last point in the previous sine wave.
Unfortunately since sine isn't one-to-one sometimes I got waves where the values matched, but the slopes didn't:
My fix for this was to check if the signs of the slopes matched, and if they didn't, slowly increase the offset until they did, then continue slowly increasing the offset until the discontinuity was small (<10). I'm sure this isn't the nicest or most mathematically satisfying way to solve this, but it works well enough for me. Now I have beautiful (pretty close to) continuously differentiable waves.
I'm new to Python and signal processing, and I'm having a problem with FFT.
I'm supposed to analyze a set of data and find the modulation frequencies from it. I wrote a basic FFT script to do this, and the output looked kinda weird. It does show the peaks like a normal FFT graph. However, for each line it has a horizontal line that connects the two ends, instead of the ends spreading out.
I would like to ask what might be the problem here.
This is my script:
from numpy.fft import fft, fftfreq
import matplotlib.pyplot as plt
Nsample = len(data)
n=96 #for zero-padded
window = np.hanning(Nsample/2) #Hann window
data1 = data[(data['condition']=='a')]
data2 = data[(data['condition']=='b')]
#Apply Hann window then do FFT
data1_Hann = data1['val']*window
data1_FFT = fft(data1_Hann, n)
freq1 = fftfreq(n, d=0.026)
data2_Hann = data2['val']*window
data2_FFT = fft(data2_Hann, n)
freq2 = fftfreq(n, d=0.026)
plt.plot(freq1, np.abs(data1_FFT), freq2, np.abs(data2_FFT))
plt.xlabel("Frequency (Hz)")
plt.ylabel("Amplitude")
data is a DataFrame containing values of 2 different conditions shuffled together, that's why I separate them and the Hanning window is applied for half the initial sample number. For each condition there are only 12 values, so I do zero padding so as to make the peaks appear clearly and the graph look smoother (I did try with smaller number for zero padding, but the line still remain).
Answer: The graph look like that because of the order of the fft calculation output: it starts with 0 Hz (more details presented here: https://numpy.org/doc/stable/reference/generated/numpy.fft.fftfreq.html)
I fixed this by using numpy.fft.fftshift() function (more details presented here: https://numpy.org/doc/stable/reference/generated/numpy.fft.fftshift.html).
Solution has been suggested by #gavinb and #yanziselman.
i have a dataframe with 120 000 rows and i want to plot two features of it over the whole time of 120 000 seconds. It is the Y of my Regression and my prediction of the Random Forest-
I am doing it with this code:
laenge_xcheck = list(range(0,len(X_gute_vorhersage),1))
X_gute_vorhersage['Zeit_artificial'] = laenge_xcheck
from numpy import *
import math
import matplotlib.pyplot as plt
plt.figure(figsize=(910,50))
plt.plot(X_gute_vorhersage.Zeit_artificial, X_gute_vorhersage.Y_test, 'r') # plotting t, a separately
plt.plot(X_gute_vorhersage.Zeit_artificial, X_gute_vorhersage.predicitons_rf, 'b') # plotting t, b separately
#plt.savefig('test1.png')
plt.show()
my goal is to get a picture where i can zoom in with a high resolution, higher than 910,50 - because it is not possible to see a lot with this. I would like to have a really long x axis and a short y and then be able to zoom in. I get it already in this shape:
Is there any possibility to get this in a resolution where i can zoom in and see it in a good resolution. It doesn't matter if the picture gets 200MB. Is there any way to go beyond the size of 910,50 without getting this error message:
ValueError: Image size of 72000x3600 pixels is too large. It must be less than 2^16 in each direction.
Thank you,
R
I've got a little problem managing FFT data. I was looking for many examples of how to do FFT, but I couldn't get what I want from any of them. I have a random wave file with 44kHz sample rate and I want to get magnitude of N harmonics each X ms, let's say 100ms should be enough. I tried this code:
import scipy.io.wavfile as wavfile
import numpy as np
import pylab as pl
rate, data = wavfile.read("sound.wav")
t = np.arange(len(data[:,0]))*1.0/rate
p = 20*np.log10(np.abs(np.fft.rfft(data[:2048, 0])))
f = np.linspace(0, rate/2.0, len(p))
pl.plot(f, p)
pl.xlabel("Frequency(Hz)")
pl.ylabel("Power(dB)")
pl.show()
This was last example I used, I found it somewhere on stackoverflow. The problem is, this gets magnitude which I want, gets frequency, but no time at all. FFT analysis is 3D as far as I know and this is "merged" result of all harmonics. I get this:
X-axis = Frequency, Y-axis = Magnitude, Z-axis = Time (invisible)
From my understanding of the code, t is time - and it seems like that, but is not needed in the code - We'll maybe need it though. p is array of powers (or magnitude), but it seems like some average of all magnitudes of each frequency f, which is array of frequencies. I don't want average/merged value, I want magnitude for N harmonics each X milliseconds.
Long story short, we can get: 1 magnitude of all frequencies.
We want: All magnitudes of N freqeuencies including time when certain magnitude is present.
Result should look like this array: [time,frequency,amplitude]
So in the end if we want 3 harmonics, it would look like:
[0,100,2.85489] #100Hz harmonic has 2.85489 amplitude on 0ms
[0,200,1.15695] #200Hz ...
[0,300,3.12215]
[100,100,1.22248] #100Hz harmonic has 1.22248 amplitude on 100ms
[100,200,1.58758]
[100,300,2.57578]
[200,100,5.16574]
[200,200,3.15267]
[200,300,0.89987]
Visualization is not needed, result should be just arrays (or hashes/dictionaries) as listed above.
Further to #Paul R's answer, scipy.signal.spectrogram is a spectrogram function in scipy's signal processing module.
The example at the above link is as follows:
from scipy import signal
import matplotlib.pyplot as plt
# Generate a test signal, a 2 Vrms sine wave whose frequency linearly
# changes with time from 1kHz to 2kHz, corrupted by 0.001 V**2/Hz of
# white noise sampled at 10 kHz.
fs = 10e3
N = 1e5
amp = 2 * np.sqrt(2)
noise_power = 0.001 * fs / 2
time = np.arange(N) / fs
freq = np.linspace(1e3, 2e3, N)
x = amp * np.sin(2*np.pi*freq*time)
x += np.random.normal(scale=np.sqrt(noise_power), size=time.shape)
#Compute and plot the spectrogram.
f, t, Sxx = signal.spectrogram(x, fs)
plt.pcolormesh(t, f, Sxx)
plt.ylabel('Frequency [Hz]')
plt.xlabel('Time [sec]')
plt.show()
It looks like you're trying to implement a spectrogram, which is a sequence of power spectrum estimates, typically implemented with a succession of (usually overlapping) FFTs. Since you only have one FFT (spectrum) then you have no time dimension yet. Put your FFT code in a loop, and process one block of samples (e.g. 1024) per iteration, with a 50% overlap between successive blocks. The sequence of generated spectra will then be a 3D array of time v frequency v magnitude.
I'm not a Python person, but I can give you some pseudo code which should be enough to get you coding:
N = length of data input
N_FFT = no of samples per block (== FFT size, e.g. 1024)
i = 0 ;; i = index of spectrum within 3D output array
for block_start = 0 to N - block_start
block_end = block_start + N_FFT
get samples from block_start .. block_end
apply window function to block (e.g. Hamming)
apply FFT to windowed block
calculate magnitude spectrum (20 * log10( re*re + im*im ))
store spectrum in output array at index i
block_start += N_FFT / 2 ;; NB: 50% overlap
i++
end
Edit: Oh, so it seems this returns values, but they don't fit to the audio file at all. Even though they can be used as magnitude on spectrogram, they won't work for example in those classic audio visualizers which you can see in many music players. I also tried matplotlib's pylab for the spectrogram, but the result is same.
import os
import wave
import pylab
import math
from numpy import amax
from numpy import amin
def get_wav_info(wav_file,mi,mx):
wav = wave.open(wav_file, 'r')
frames = wav.readframes(-1)
sound_info = pylab.fromstring(frames, 'Int16')
frame_rate = wav.getframerate()
wav.close()
spectrum, freqs, t, im = pylab.specgram(sound_info, NFFT=1024, Fs=frame_rate)
n = 0
while n < 20:
for index,power in enumerate(spectrum[n]):
print("%s,%s,%s" % (n,int(round(t[index]*1000)),math.ceil(power*100)/100))
n += 1
get_wav_info("wave.wav",1,20)
Any tips how to obtain dB that's usable in visualization?
Basically, we apparently have all we need from the code above, just how to make it return normal values? Ignore mi and mx as these are just adjusting values in array to fit into mi..mx interval - that would be for visualization usage. If I am correct, spectrum in this code returns array of arrays which contains amplitudes for each frequency from freqs array, which are present on time according to t array, but how does the value work - is it really amplitude if it returns these weird values and if it is, how to convert it to dBs for example.
tl;dr I need output for visualizer like music players have, but it shouldn't work realtime, I want just the data, but values don't fit the wav file.
Edit2: I noticed there's one more issue. For 90 seconds wav, t array contains times till 175.x, which seems very weird considering the frame_rate is correct with the wav file. So now we have 2 problems: spectrum doesn't seem to return correct values (maybe it will fit if we get correct time) and t seems to return exactly double time of the wav.
Fixed: Case completely solved.
import os
import pylab
import math
from numpy import amax
from numpy import amin
from scipy.io import wavfile
frame_rate, snd = wavfile.read(wav_file)
sound_info = snd[:,0]
spectrum, freqs, t, im = pylab.specgram(sound_info,NFFT=1024,Fs=frame_rate,noverlap=5,mode='magnitude')
Specgram needed a little adjustment and I loaded only one channel with scipy.io library (instead of wave library). Also without mode set to magnitude, it returns 10log10 instead of 20log10, which is reason why it didn't return correct values.
Audio processing is pretty new for me. And currently using Python Numpy for processing wave files. After calculating FFT matrix I am getting noisy power values for non-existent frequencies. I am interested in visualizing the data and accuracy is not a high priority. Is there a safe way to calculate the clipping value to remove these values, or should I use all FFT matrices for each sample set to come up with an average number ?
regards
Edit:
from numpy import *
import wave
import pymedia.audio.sound as sound
import time, struct
from pylab import ion, plot, draw, show
fp = wave.open("500-200f.wav", "rb")
sample_rate = fp.getframerate()
total_num_samps = fp.getnframes()
fft_length = 2048.
num_fft = (total_num_samps / fft_length ) - 2
temp = zeros((num_fft,fft_length), float)
for i in range(num_fft):
tempb = fp.readframes(fft_length);
data = struct.unpack("%dH"%(fft_length), tempb)
temp[i,:] = array(data, short)
pts = fft_length/2+1
data = (abs(fft.rfft(temp, fft_length)) / (pts))[:pts]
x_axis = arange(pts)*sample_rate*.5/pts
spec_range = pts
plot(x_axis, data[0])
show()
Here is the plot in non-logarithmic scale, for synthetic wave file containing 500hz(fading out) + 200hz sine wave created using Goldwave.
Simulated waveforms shouldn't show FFTs like your figure, so something is very wrong, and probably not with the FFT, but with the input waveform. The main problem in your plot is not the ripples, but the harmonics around 1000 Hz, and the subharmonic at 500 Hz. A simulated waveform shouldn't show any of this (for example, see my plot below).
First, you probably want to just try plotting out the raw waveform, and this will likely point to an obvious problem. Also, it seems odd to have a wave unpack to unsigned shorts, i.e. "H", and especially after this to not have a large zero-frequency component.
I was able to get a pretty close duplicate to your FFT by applying clipping to the waveform, as was suggested by both the subharmonic and higher harmonics (and Trevor). You could be introducing clipping either in the simulation or the unpacking. Either way, I bypassed this by creating the waveforms in numpy to start with.
Here's what the proper FFT should look like (i.e. basically perfect, except for the broadening of the peaks due to the windowing)
Here's one from a waveform that's been clipped (and is very similar to your FFT, from the subharmonic to the precise pattern of the three higher harmonics around 1000 Hz)
Here's the code I used to generate these
from numpy import *
from pylab import ion, plot, draw, show, xlabel, ylabel, figure
sample_rate = 20000.
times = arange(0, 10., 1./sample_rate)
wfm0 = sin(2*pi*200.*times)
wfm1 = sin(2*pi*500.*times) *(10.-times)/10.
wfm = wfm0+wfm1
# int test
#wfm *= 2**8
#wfm = wfm.astype(int16)
#wfm = wfm.astype(float)
# abs test
#wfm = abs(wfm)
# clip test
#wfm = clip(wfm, -1.2, 1.2)
fft_length = 5*2048.
total_num_samps = len(times)
num_fft = (total_num_samps / fft_length ) - 2
temp = zeros((num_fft,fft_length), float)
for i in range(num_fft):
temp[i,:] = wfm[i*fft_length:(i+1)*fft_length]
pts = fft_length/2+1
data = (abs(fft.rfft(temp, fft_length)) / (pts))[:pts]
x_axis = arange(pts)*sample_rate*.5/pts
spec_range = pts
plot(x_axis, data[2], linewidth=3)
xlabel("freq (Hz)")
ylabel('abs(FFT)')
show()
FFT's because they are windowed and sampled cause aliasing and sampling in the frequency domain as well. Filtering in the time domain is just multiplication in the frequency domain so you may want to just apply a filter which is just multiplying each frequency by a value for the function for the filter you are using. For example multiply by 1 in the passband and by zero every were else. The unexpected values are probably caused by aliasing where higher frequencies are being folded down to the ones you are seeing. The original signal needs to be band limited to half your sampling rate or you will get aliasing. Of more concern is aliasing that is distorting the area of interest because for this band of frequencies you want to know that the frequency is from the expected one.
The other thing to keep in mind is that when you grab a piece of data from a wave file you are mathmatically multiplying it by a square wave. This causes a sinx/x to be convolved with the frequency response to minimize this you can multiply the original windowed signal with something like a Hanning window.
It's worth mentioning for a 1D FFT that the first element (index [0]) contains the DC (zero-frequency) term, the elements [1:N/2] contain the positive frequencies and the elements [N/2+1:N-1] contain the negative frequencies. Since you didn't provide a code sample or additional information about the output of your FFT, I can't rule out the possibility that the "noisy power values at non-existent frequencies" aren't just the negative frequencies of your spectrum.
EDIT: Here is an example of a radix-2 FFT implemented in pure Python with a simple test routine that finds the FFT of a rectangular pulse, [1.,1.,1.,1.,0.,0.,0.,0.]. You can run the example on codepad and see that the FFT of that sequence is
[0j, Negative frequencies
(1+0.414213562373j), ^
0j, |
(1+2.41421356237j), |
(4+0j), <= DC term
(1-2.41421356237j), |
0j, v
(1-0.414213562373j)] Positive frequencies
Note that the code prints out the Fourier coefficients in order of ascending frequency, i.e. from the highest negative frequency up to DC, and then up to the highest positive frequency.
I don't know enough from your question to actually answer anything specific.
But here are a couple of things to try from my own experience writing FFTs:
Make sure you are following Nyquist rule
If you are viewing the linear output of the FFT... you will have trouble seeing your own signal and think everything is broken. Make sure you are looking at the dB of your FFT magnitude. (i.e. "plot(10*log10(abs(fft(x))))" )
Create a unitTest for your FFT() function by feeding generated data like a pure tone. Then feed the same generated data to Matlab's FFT(). Do a absolute value diff between the two output data series and make sure the max absolute value difference is something like 10^-6 (i.e. the only difference is caused by small floating point errors)
Make sure you are windowing your data
If all of those three things work, then your fft is fine. And your input data is probably the issue.
Check the input data to see if there is clipping http://www.users.globalnet.co.uk/~bunce/clip.gif
Time doamin clipping shows up as mirror images of the signal in the frequency domain at specific regular intervals with less amplitude.