I used fft.fft(data) and plotted that result I was expecting to the frequency that I gave in data.
I was expecting to see 50 hz but I got something strange.
import numpy as np
import math as m
import matplotlib.pyplot as plt
data=[]
for x in range(1000):
data.append(m.sin(2*m.pi*50*0.001*x))
plt.plot(np.fft.fft(data)/len(data))
plt.show()
What should I do to see 50 Hz as result?
Thank you very much
You need to specify the x axis in your plot.
First, create the data:
import numpy as np
import matplotlib.pyplot as plt
t = np.linspace(0, 1, 1000)
data = np.sin(2*np.pi*50*t)
Now, get the frequencies:
f = np.fft.fftfreq(len(data), t[1]-t[0]) # length of data, and dt
And plot the magnitude of the fft vs frequencies:
data_fft = np.abs(np.fft.fft(data)) / len(data)
plt.plot(f, data_fft)
This is really a question for the DSP stack exchange (https://dsp.stackexchange.com/).
You are doing two things that are causing the odd result:
You are performing a complex to complex FFT on real data, so you will have your signal mirrored about the Nyquist frequency (Hermitian symmetry).
You are dividing and plotting the complex output, not the Fourier amplitudes or powers.(Matplotlib doesn't "get" complex numbers, so this comes out looking like garbage.)
try this instead:
plt.plot(abs(np.fft.rfft(data))/(len(data)/2))
Related
I am trying to implement a butterworthfilter with python in jupyter Notebook. I wrote this code by a tutorial.
The data are from a CSV-File, it calls Samples.csv
The data in Samples.csv are like
998,4778415
1009,209592
1006,619094
1001,785406
993,9426543
990,1408991
992,736118
995,8127334
1002,381664
1006,094429
1000,634799
999,3287747
1002,318812
999,3287747
1004,427698
1008,516733
1007,964781
1002,680906
1000,14449
994,257009
The column calls Euclidian Norm. The range of the data are from 0 to 1679.286158 and theyre are 1838 rows.
This is the code in Jupyter:
from scipy.signal import filtfilt
from scipy import stats
import csv
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import scipy
def plot():
data=pd.read_csv('Samples.csv',sep=";", decimal=",")
sensor_data=data[['Euclidian Norm']]
sensor_data=np.array(sensor_data)
time=np.linspace(0,1679.286158,1838)
plt.plot(time,sensor_data)
plt.show()
filtered_signal=bandPassFilter(sensor_data)
plt.plot(time,sensor_data)
plt.show()
def bandPassFilter(signal):
fs = 4000.0
lowcut=20.0
highcut=50.0
nyq=0.5*fs
low=lowcut/nyq
high=highcut/nyq
order =2
b,a=scipy.signal.butter(order,[low,high],'bandpass',analog=False)
y=scipy.signal.filtfilt(b,a,signal,axis=0)
return(y)
plot()
My problem is that nothing changes in my data. It doesnt filtered my data. The graph of the filtered data is the same like the source data. Does anyone know what could be wrong?
The first graph is the source data and the second graph is the filtered graph. It looks very similar. Its like the same graph
I can't comment yet.
You're never using filtered_signal and plot with the same arguments twice.
Here`s one of my implementations with added interpolation, very similar to yours:
def butterFit(data, freq, order=2):
ar = scipy.signal.butter(order, freq) # Gets params for filttilt
return spfilter.filtfilt(ar[0], ar[1], data)
def plotFilteredSplines(timeframe, data, amount_points):
# Generate evenly spread indices for the data points.
indices = np.arange(0, len(data), amount_points)
cutoff_freq = 2 / (2/10 * len(timeframe))
# Reshape the data with butter :)
data = butterFit(data, cutoff_freq)
# Plot Fitlered data
plt.plot(timeframe, data, '-.')
interpol_x = np.linspace(timeframe[0], timeframe[-1], 100)
# Get the cubic spline approx function
interpolation = sp.interpolate.interp1d(timeframe, data, kind='cubic')
# Plot the interpolation over the extended time frame.
plt.plot(interpol_x, interpolation(interpol_x), '-r')
i have a single column file(contain only one column) and a matrix file(contain 10 columns) of data which are noisy data and i want to plot the noise spectrum of both file using python.
sample data for single column file is attached here
-1.064599999999999921e-02
-1.146800000000000076e-02
-1.011899999999999952e-02
-7.400200000000000007e-03
-4.306500000000000432e-03
-1.644800000000000081e-03
1.936600000000000127e-04
1.239199999999999980e-03
1.759200000000000043e-03
2.019799999999999981e-03
2.148699999999999916e-03
2.153099999999999806e-03
2.008799999999999822e-03
1.700899999999999981e-03
1.181500000000000042e-03
3.194000000000000116e-04
-1.072000000000000036e-03
-3.133799999999999954e-03
and sample data for matrix file is attached here
-2.596100000000000057e-03 -1.856000000000000011e-03 -1.821400000000000102e-02 5.023599999999999594e-03 -1.064599999999999921e-02 -1.906300000000000008e-02 -6.370799999999999380e-05 5.814800000000000177e-03 -5.391800000000000412e-03 -1.311000000000000013e-02
1.636700000000000047e-03 -8.651600000000000176e-04 -2.490799999999999959e-02 1.645399999999999988e-02 -1.146800000000000076e-02 -4.609199999999999929e-03 6.475800000000000306e-03 1.265800000000000085e-02 1.855799999999999898e-03 -5.387499999999999928e-03
4.516499999999999682e-03 1.438899999999999901e-03 -2.911599999999999952e-02 2.590800000000000047e-02 -1.011899999999999952e-02 2.378800000000000012e-02 1.080200000000000084e-02 1.994299999999999892e-02 8.882299999999999224e-03 2.866500000000000124e-03
5.604699999999999786e-03 4.557799999999999872e-03 -2.870800000000000088e-02 2.832300000000000095e-02 -7.400200000000000007e-03 2.882099999999999940e-02 1.145799999999999944e-02 2.488800000000000040e-02 1.367299999999999939e-02 8.998799999999999508e-03
4.797400000000000275e-03 7.657399999999999970e-03 -2.582800000000000026e-02 2.288000000000000103e-02 -4.306500000000000432e-03 8.315499999999999975e-03 7.967600000000000030e-03 2.487999999999999934e-02 1.516600000000000066e-02 1.177899999999999954e-02
2.314300000000000038e-03 9.749700000000000033e-03 -2.252099999999999935e-02 1.762000000000000025e-02 -1.644800000000000081e-03 -1.257800000000000064e-02 1.220600000000000070e-03 1.866299999999999903e-02 1.377199999999999952e-02 1.163999999999999931e-02
-1.290700000000000094e-03 9.894599999999999923e-03 -1.928900000000000059e-02 1.360300000000000051e-02 1.936600000000000127e-04 -2.438999999999999849e-02 -6.739199999999999878e-03 6.961199999999999853e-03 1.086299999999999939e-02 1.015199999999999957e-02
-5.137400000000000300e-03 7.453800000000000009e-03 -1.615099999999999869e-02 1.018799999999999914e-02 1.239199999999999980e-03 -1.585699999999999957e-02 -1.349500000000000005e-02 -7.773600000000000301e-03 7.680499999999999827e-03 9.148399999999999241e-03
-8.159500000000000086e-03 2.403600000000000094e-03 -1.270400000000000001e-02 5.359000000000000048e-03 1.759200000000000043e-03 -9.746799999999999908e-03 -1.730999999999999900e-02 -2.229599999999999985e-02 4.641100000000000433e-03 9.871700000000000613e-03
-9.419600000000000195e-03 -4.305599999999999705e-03 -8.259700000000000028e-03 -3.140800000000000015e-03 2.019799999999999981e-03 -5.883300000000000161e-03 -1.772100000000000064e-02 -2.695099999999999926e-02 1.592399999999999892e-03 1.255299999999999992e-02
-8.469000000000000833e-03 -1.101399999999999949e-02 -2.205400000000000155e-03 -1.641199999999999951e-02 2.148699999999999916e-03 -3.635199999999999890e-03 -1.558000000000000010e-02 -1.839000000000000010e-02 -1.408900000000000039e-03 1.642899999999999916e-02
-5.529599999999999967e-03 -1.553999999999999999e-02 5.413199999999999956e-03 -4.248000000000000040e-03 2.153099999999999806e-03 -2.403199999999999868e-03 -1.255099999999999966e-02 -8.339100000000000332e-03 -3.665700000000000035e-03 2.009499999999999828e-02
i tried with https://www.earthinversion.com/techniques/visualizing-power-spectral-density-demo-obspy/ but for my ascii data set i could not do it.I hope experts may help me .Thanks in advance.
Maybe this can give you a start. Give your matrix of data in the file "x.data", this plots the raw data as 10 curves, then runs an FFT on each column and displays the FFT. The FFT isn't very interesting with only 12 points, but it will spark ideas.
There's still the problem of "how do you define noise"? The signals you presented do not seem to be very noisy. Unless you know what kind of signal you're expecting,an FFT might not do much good.
import numpy as np
import matplotlib.pyplot as plt
import scipy.fftpack
data = np.loadtxt("x.data")
for i in range(data.shape[1]):
plt.plot( data[:,i] )
plt.show()
for i in range(data.shape[1]):
f = scipy.fftpack.fft(data[:,i])
plt.plot(np.abs(f))
plt.show()
Use numpy.loadtxt() to convert the data to a numpy array. Then you can apply the method described in the link you provided in order to obtain the spectra. E.g.:
import numpy as np
data = np.loadtxt("file.txt")
The you plot the spectrum for that data. E.g.:
import matplotlib.pyplot as plt import scipy.fftpack
yf = scipy.fftpack.fft(data)
fig, ax = plt.subplots()
ax.plot(np.abs(yf))
plt.show()
I have been thinking about it for a long time, but I don't find out what the problem is. Hope you can help me, Thank you.
F(s) Gaussian function
F(s)=1/(√2π s) e^(-(w-μ)^2/(2s^2 ))
Code:
import numpy as np
from matplotlib import pyplot as plt
from math import pi
from scipy.fft import fft
def F_S(w, mu, sig):
return (np.exp(-np.power(w-mu, 2)/(2 * np.power(sig, 2))))/(np.power(2*pi, 0.5)*sig)
w=np.linspace(-5,5,100)
plt.plot(w, np.real(np.fft.fft(F_S(w, 0, 1))))
plt.show()
Result:
As was mentioned before you want the absolute value, not the real part.
A minimal example, showing the the re/im , abs/phase spectra.
import numpy as np
import matplotlib.pyplot as p
%matplotlib inline
n=1001 # add 1 to keep the interval a round number when using linspace
t = np.linspace(-5, 5, n ) # presumed to be time
dt=t[1]-t[0] # time resolution
print(f'sampling every {dt:.3f} sec , so at {1/dt:.1f} Sa/sec, max. freq will be {1/2/dt:.1f} Hz')
y = np.exp(-(t**2)/0.01) # signal in time
fr= np.fft.fftshift(np.fft.fftfreq(n, dt)) # shift helps with sorting the frequencies for better plotting
ft=np.fft.fftshift(np.fft.fft(y)) # fftshift only necessary for plotting in sequence
p.figure(figsize=(20,12))
p.subplot(231)
p.plot(t,y,'.-')
p.xlabel('time (secs)')
p.title('signal in time')
p.subplot(232)
p.plot(fr,np.abs(ft), '.-',lw=0.3)
p.xlabel('freq (Hz)')
p.title('spectrum, abs');
p.subplot(233)
p.plot(fr,np.real(ft), '.-',lw=0.3)
p.xlabel('freq (Hz)')
p.title('spectrum, real');
p.subplot(235)
p.plot(fr,np.angle(ft), '.-', lw=0.3)
p.xlabel('freq (Hz)')
p.title('spectrum, phase');
p.subplot(236)
p.plot(fr,np.imag(ft), '.-',lw=0.3)
p.xlabel('freq (Hz)')
p.title('spectrum, imag');
you have to change from time scale to frequency scale
When you make a FFT you will get the simetric tranformation, i.e, mirror of the positive to negative curve. Usually, you only will look at the positive side.
Also, you should take care with sample rate, as FFT is designed to transform time domain input to frequency domain, the time, or sample rate, of input info matters. So add timestep in np.fft.fftfreq(n, d=timestep) for your sample rate.
If you simple want to make a fft of normal dist signal, here is another question with it and some good explanations on why are you geting this behavior:
Fourier transform of a Gaussian is not a Gaussian, but thats wrong! - Python
There are two mistakes in your code:
Don't take the real part, take the absoulte value when plotting.
From the docs:
If A = fft(a, n), then A[0] contains the zero-frequency term (the mean
of the signal), which is always purely real for real inputs. Then
A[1:n/2] contains the positive-frequency terms, and A[n/2+1:] contains
the negative-frequency terms, in order of decreasingly negative
frequency.
You can rearrange the elements with np.fft.fftshift.
The working code:
import numpy as np
from matplotlib import pyplot as plt
from math import pi
from scipy.fftpack import fft, fftshift
def F_S(w, mu, sig):
return (np.exp(-np.power(w-mu, 2)/(2 * np.power(sig, 2))))/(np.power(2*pi, 0.5)*sig)
w=np.linspace(-5,5,100)
plt.plot(w, fftshift(np.abs(np.fft.fft(F_S(w, 0, 1)))))
plt.show()
Also, you might want to consider scaling the x axis too.
I was trying to fit beta prime distribution to my data using python. As there's scipy.stats.betaprime.fit, I tried this:
import numpy as np
import math
import scipy.stats as sts
import matplotlib.pyplot as plt
N = 5000
nb_bin = 100
a = 12; b = 106; scale = 36; loc = -a/(b-1)*scale
y = sts.betaprime.rvs(a,b,loc,scale,N)
a_hat,b_hat,loc_hat,scale_hat = sts.betaprime.fit(y)
print('Estimated parameters: \n a=%.2f, b=%.2f, loc=%.2f, scale=%.2f'%(a_hat,b_hat,loc_hat,scale_hat))
plt.figure()
count, bins, ignored = plt.hist(y, nb_bin, normed=True)
pdf_ini = sts.betaprime.pdf(bins,a,b,loc,scale)
pdf_est = sts.betaprime.pdf(bins,a_hat,b_hat,loc_hat,scale_hat)
plt.plot(bins,pdf_ini,'g',linewidth=2.0,label='ini');plt.grid()
plt.plot(bins,pdf_est,'y',linewidth=2.0,label='est');plt.legend();plt.show()
It shows me the result that:
Estimated parameters:
a=9935.34, b=10846.64, loc=-90.63, scale=98.93
which is quite different from the original one and the figure from the PDF:
If I give the real value of loc and scale as the input of fit function, the estimation result would be better. Has anyone worked on this part already or got a better solution?
I use Tektronix oscilloscope to perform some signal acquisition. I get 10.000 measurement points (few signal periods) and I have to do a frequency analysis on that set of data. My signal is 8MHz sine wave. When I use either SciPy or NumPy I get the same result - frequencies are spreaded too wide. The distance between two values is 500kHz and the highest frequency is 2.5GHz (absurd). When I want to measure frequency bandwidth around 8MHz I can only get exact values of 7.5, 8.0 and 8.5 MHz. I tried to change sample spacing determined by (x[1]-x[0]) and I got nothing better.
def CalculateFFT(t_val,p_val):
x = t_val #Two parameters: [x,y] values
y = lambda x: p_val
com_signal = y(x) # Combined signal
FFT_val = abs(scipy.fft(com_signal))
freq_val = scipy.fftpack.fftfreq(len(com_signal), x[1]-x[0])
spec_val = 20*scipy.log10(FFT_val)
return freq_val, spec_val
It is worth reading in more depth how DFFTs work but you should always have the following formulae in mind. For a time series with n points and maximum time Tmax, the time resolution is given by dt = Tmax / n
A DFFT will produce n points with
Fmax = 1 / dt
dF = 1 / Tmax
You seem to suggest the maximum frequency is sufficient (so the time resolution is okay) but the frequency resolution isn't good enough: you need to collect more data, at the same time resolution.
If (1) the sampling time is too short, (2) you require higher estimation frequency accuracy, and, (3) you know that your signal is a sine wave, then you can fit the signal to a sine wave. Like in How do I fit a sine curve to my data with pylab and numpy?,
with the exception that the frequency needs to be added.
Here is an example figure with a frequency of around 8 MHz:
Below is the example code:
""" Modified from https://stackoverflow.com/a/16716964/6036470 """
from numpy import sin, linspace, pi,average;
from pylab import plot, show, title, xlabel, ylabel, subplot, scatter
from scipy import fft, arange, ifft
import scipy
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import leastsq
ff = 8e6; # frequency of the signal
Fs = ff*128; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = arange(0,((1/ff)/128)*(128)*5,Ts) # time vector
A = 2.5;
ff_0 = 8.1456e6
y = A*np.sin(2*np.pi*ff_0*t+15.38654*pi/180) + np.random.randn(len(t))/5
guess_b = 0
guess_a = y.std()*2**0.5;
guess_c = 10*pi/180
guess_d = ff*0.98*2*pi
fig = plt.figure(facecolor="white")
plt.plot(t,y,'.', label='Signal Fred. %0.4f Hz'%(ff_0/1e6))
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.grid(alpha=0.5);
optimize_func = lambda x: (x[0]*np.sin(x[2]*t+x[1]) - y);
est_a, est_c, est_d = leastsq(optimize_func, [guess_a, guess_c, guess_d])[0]
data_fit = est_a*np.sin(est_d*t+est_c) ;
plt.plot(t,data_fit,label='Fitted Est. Freq. %0.4f Hz'%(est_d/(2*pi)/1e6))
plt.legend()
plt.tight_layout();
plt.show();
fig.save("sinfit.png")