We Keep Coding

Gaussian fit for Python with two different function

I used the Gaussian fit with 3 gauss to adjust but datai but I utility data that sometimes my curve contains only two Gaussians in it not find the parameter remnants to use and but great an error is what there is a method that but allows to change with curve fit function use if two or three gaussians .
for my function main, i have this code :
FitGWPS = mainCurveFitGWPS(global_ws, period, All_Max_GWPS, DoupleDip)
and my code for fit is :
import numpy as np
from scipy.optimize import curve_fit
#Functions-----------------------------------------
#Gaussian function
def _1gaus(X,C,X_mean,sigma):
return C*np.exp(-(X-X_mean)**2/(2*sigma**2))
def _3gaus(x, amp1,cen1,sigma1, amp2,cen2,sigma2, amp3,cen3,sigma3):
return amp1*np.exp(-(x-cen1)**2/(2*sigma1**2)) +\
amp2*np.exp(-(x-cen2)**2/(2*sigma2**2)) + amp3*np.exp(-(x-
cen3)**2/(2*sigma3**2))
def ParamFit (Gws, P, Max, popt_Firstgauss):
#Calculating the Lorentzian PDF values given Gaussian parameters and random variableX
width=0
Amp = []
cen = []
wid = []
for j in range(len(Max-1)):
Amp.append(0.8 * (Gws[Max[j]])) # Amplitude
cen.append(P[Max[j]]) # Frequency
if j == 0 : wid.append(0.3 + width * 2.) # Width
else : wid.append(0.3 + popt_Firstgauss[2] * 2.)
return Amp,wid,cen
def mainCurveFitGWPS(global_ws_in, period_in, All_Max_GWPS, DoupleDip):
#Calculating the Gaussian PDF values given Gaussian parameters and random variable X
# For the first fit we calculate with function of the max values
mean = sum(period_in*(global_ws_in))/sum((global_ws_in ))
sigma = np.sqrt(sum((global_ws_in)*(period_in-mean)**2)/sum((global_ws_in)))
Cst = 1 / ( 2* np.pi * sigma)
width=0
Amp = 0.8 * (global_ws_in[All_Max_GWPS[0]]) # Amplitude
cen = period_in[All_Max_GWPS[0]] # Frequency
wid = 0.3 + width * 2. #Width
Amp = []
cen = []
wid = []
for j in range(len(All_Max_GWPS-1)):
Amp.append(0.8 * (global_ws_in[All_Max_GWPS[j]])) # Amplitude
cen.append(period_in[All_Max_GWPS[j]]) # Frequency
if j == 0 : wid.append(0.3 + width * 2.)
else : wid.append(0.3 + popt_gauss[2] * 2.)
#do the fit!
popt_gauss, pcov_gauss = curve_fit(_1gaus, period_in, global_ws_in, p0 = [Cst,
mean, sigma])
FitGauss = _1gaus(period_in, *popt_gauss)
#I use the center, amplitude, and sigma values which I used to create the fake
#data
popt_3gauss, pcov_3gauss = curve_fit(_3gaus, period_in, global_ws_in, p0=[Amp[0],
cen[0], wid[0],Amp[1], cen[1], wid[1],Amp[2], cen[2], wid[2]], maxfev =5000)
Fit3Gauss = _3gaus(period_in, *popt_3gauss)
return Fit3Gauss
for example picture :
and

Correct amplitude of the python fft (for a Skew normal distribution)

The Situation
I am currently writing a program that will later on be used to analyze a signal that is somewhat of a asymmetric Gaussian. I am interested in how many frequencies I need to reproduce the signal somewhat exact and especially the amplitudes of those frequencies.
Before I input the real data I'm testing the program with a default (asymmetric) Gaussian, as can be seen in the code below.
My Problem
To ensure I that get the amplitudes right, I am rebuilding the original signal using the whole frequency spectrum, but there are some difficulties. I get to reproduce the signal somewhat well multiplying amp with 0.16, which I got by looking at the fraction rebuild/original. Of course, this is really unsatisfying and can't be the correct solution.
To be precise the difference is not dependant on the time length and seems to be a Gaussian too, following the form of the original, increasing in asymmetry according to the Skewnorm function itself. The amplitude of the difference function is correlated linear to 'height'.
My Question
I am writing this post because I am out of ideas for getting the amplitude right. Maybe anyone has had the same / a similar problem and can share their solution for this / give a hint.
Further information
Before focusing on a (asymmetric) Gaussian I analyzed periodic signals and rectangular pulses, which sadly were very unstable to variations in the time length of the input signal. In this context, I experimented with window functions, which seemed to speed up the process and increase the stability, the reason being that I had to integrate the peaks. Working with the Gaussian I got told to take each peak, received via the bare fft and ditch the integration approach, therefore my incertitude considering the amplitude described above. Maybe anyone got an opinion on the approach chosen by me and if necessary can deliver an improvement.
Code
from numpy.fft import fft, fftfreq
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import skewnorm
np.random.seed(1234)
def data():
height = 1
data = height * skewnorm.pdf(t, a=0, loc=t[int(N/2)])
# noise_power = 1E-6
# noise = np.random.normal(scale=np.sqrt(noise_power), size=t.shape)
# data += noise
return data
def fft_own(data):
freq = fftfreq(N, dt)
data_fft = fft(data) * np.pi
amp = 2/N * np.abs(data_fft) # * factor (depending on t1)
# amp = 2/T * np.abs(data_fft)**2
phase = np.angle(data_fft)
peaks, = np.where(amp >= 0) # use whole spectrum for rebuild
return freq, amp, phase, peaks
def rebuild(fft_own):
freq, amp, phase, peaks = fft_own
df = freq[1] - freq[0]
data_rebuild = 0
for i in peaks:
amplitude = amp[i] * df
# amplitude = amp[i] * 0.1
# amplitude = np.sqrt(amp[i] * df)
data_rebuild += amplitude * np.exp(0+1j * (2*np.pi * freq[i] * t
+ phase[i]))
f, ax = plt.subplots(1, 1)
# mask = (t >= 0) & (t <= t1-1)
ax.plot(t, data_init, label="initial signal")
ax.plot(t, np.real(data_rebuild), label="rebuild")
# ax.plot(t[mask], (data_init - np.real(data_rebuild))[mask], label="diff")
ax.set_xlim(0, t1-1)
ax.legend()
t0 = 0
t1 = 10 # diff(t0, t1) ∝ df
# T = t1- t0
N = 4096
t = np.linspace(t0, t1, int(N))
dt = (t1 - t0) / N
data_init = data()
fft_init = fft_own(data_init)
rebuild_init = rebuild(fft_init)

You should get a perfect reconstruction if you divide amp by N, and remove all your other factors.
Currently you do:
data_fft = fft(data) * np.pi # Multiply by pi
amp = 2/N * np.abs(data_fft) # Multiply by 2/N
amplitude = amp[i] * df # Multiply by df = 1/(dt*N) = 1/10
This means that you currently multiply by a total of pi * 2 / 10, or 0.628, that you shouldn't (only the 1/N factor in there is correct).
Correct code:
def fft_own(data):
freq = fftfreq(N, dt)
data_fft = fft(data)
amp = np.abs(data_fft) / N
phase = np.angle(data_fft)
peaks, = np.where(amp >= 0) # use whole spectrum for rebuild
return freq, amp, phase, peaks
def rebuild(fft_own):
freq, amp, phase, peaks = fft_own
data_rebuild = 0
for i in peaks:
data_rebuild += amp[i] * np.exp(0+1j * (2*np.pi * freq[i] * t
+ phase[i]))
Your program can be significantly simplified by using ifft. Simply set to 0 those frequencies in data_fft that you don't want to include in the reconstruction, and apply ifft to it:
data_fft = fft(data)
data_fft[np.abs(data_fft) < threshold] = 0
rebuild = ifft(data_fft).real
Note that the Fourier transform of a Gaussian is a Gaussian, so you won't be picking out individual peaks, you are picking a compact range of frequencies that will always include 0. This is an ideal low-pass filter.

How do I get the power at a particular frequency of a sound file?

I'm working on my end of the degree thesis in which I have to measure the Sound Pressure Level of underwater recordings (wav files) at a particular frequency (2000Hz). So I came up with this code:
'''
def get_value(filename, f0, NFFT=8192, plot = False):
#Load audio
data, sampling_frequency = soundfile.read(filename)
# remove stereo
if len(data.shape)> 1:
data = data[:, 0]
# remove extra length
if len(data)>sampling_frequency:
data = data[0:sampling_frequency]
# remove DC
data = data - data.mean()
# power without filtering
total_power = 10*np.log10(np.mean(data**2))
# fft
NFFT = 4096 # number of samples in the FFT
window = np.array(1) #np.hamming(len(data))
fftdata = np.fft.fft(data / NFFT, n = NFFT)
SPL = 20 * np.log10(np.abs(fftdata)) # Sound Pressure Level [dB]
freq = np.linspace(0, sampling_frequency, NFFT) # frequency axis [Hz]
# take value at desired frequency
power_at_frequency = SPL[np.argmin(np.abs(freq-f0))]
print(power_at_frequency)
'''
However, I checked the value with audacity and is completely different.
Thanks beforehand.

If you are interested in only one frequency you don't have to compute the FFT you can simply use
totalEnergy = np.sum((data - np.mean(data)) ** 2)
freqEnergy = np.abs(np.sum(data * np.exp(2j * np.pi * np.arange(len(data)) * target_freq / sampling_freq)))
And if you are using FFT and the window size is not a multiple of the wave period the frequency will leak to other frequencies. To avoid this your
import numpy as np;
import matplotlib.pyplot as plt
sampling_frequency = 48000;
target_frequency = 2000.0;
ns = 1000000;
data = np.sin(2*np.pi * np.arange(ns) * target_frequency / sampling_frequency);
# power
print('a sine wave have power 0.5 ~', np.mean(data**2), 'that will be split in two ')
## Properly scaled frequency
plt.figure(figsize=(12, 5))
plt.subplot(121);
z = np.abs(np.fft.fft(data[:8192])**2) / 8192**2
print('tuned with 8192 samples', max(z), ' some power leaked in other frequencies')
plt.semilogy(np.fft.fftfreq(len(z)) * sampling_frequency, z)
plt.ylabel('power')
plt.title('some power leaked')
plt.subplot(122);
# 6000 samples = 1/8 second is multiple of 1/2000 second
z = np.abs(np.fft.fft(data[:6000])**2) / 6000**2
print('tuned with 6000 samples', max(z))
plt.semilogy(np.fft.fftfreq(len(z)) * sampling_frequency, z)
plt.xlabel('frequency')
plt.title('all power in exact two symmetric bins')
## FFT of size not multiple of 2000
print(np.sum(np.abs(np.fft.fft(data[:8192]))**2) / 8192)

How do I use np.fft.fft() to correctly identify "peak energy" and also get the associated winding frequencies?

I conceptually understand Fourier transforms. I wrote a naive algorithm to compute the transform, decompose a wave and plot it's individual components. I know it's not 'fast', and it also doesn't reconstruct the right amplitude. It was just meant to code the math behind the machinery, and it gives me this nice output:
Questions
How do I do something similar with np.fft
How do I recover whatever winding frequencies numpy chose under the hood?
How do I recover the amplitude of component waves that I find using the transform?
I've tried a few things. However, when I use p = np.fft.fft(signal) on the same exact wave as the above, I get really wacky plots, like this one:
f1 = 3
f2 = 5
start = 0
stop = 1
sample_rate = 0.005
x = np.arange(start, stop, sample_rate)
y = np.cos(f1 * 2 * np.pi * x) + np.sin(f2 * 2 * np.pi *x)
p = np.fft.fft(y)
plt.plot(np.real(p))
Or if I try to use np.fft.freq() to get the right frequencies for the horizontal axis:
p = np.fft.fft(y)
f = np.fft.fftfreq(y.shape[-1], d=sampling_rate)
plt.plot(f, np.real(p))
And as a recent addition, my attempt to implement #wwii's suggestions resulted in an improvement, but the frequency powers are still off in the transform:
f1 = 3
f2 = 5
start = 0
stop = 4.5
sample_rate = 0.01
x = np.arange(start, stop, sample_rate)
y = np.cos(f1 * 2 * np.pi * x) + np.sin(f2 * 2 * np.pi *x)
p = np.fft.fft(y)
freqs= np.fft.fftfreq(y.shape[-1], d=sampling_rate)
q = np.abs(p)
q = q[freqs > 0]
f = freqs[freqs > 0]
peaks, _ = find_peaks(q)
peaks
plt.plot(f, q)
plt.plot(freqs[peaks], q[peaks], 'ro')
plt.show()
So again, my question is, how do I use np.fft.fft and np.fft.fftfreqs to get the same information as my naive method does? And secondly, how do I recover amplitude information from the fft (amplitude of the component waves that add up to the composite).
I've read the documentation, but it is far from helpful.
For context here is my my naive method:
def wind(timescale, data, w_freq):
"""
wrap time-series data around complex plain at given winding frequency
"""
return data * np.exp(2 * np.pi * w_freq * timescale * 1.j)
def transform(x, y, freqs):
"""
Returns center of mass of each winding frequency
"""
ft = []
for f in freqs:
mapped = wind(x, y, f)
re, im = np.real(mapped).mean(), np.imag(mapped).mean()
mag = np.sqrt(re ** 2 + im ** 2)
ft.append(mag)
return np.array(ft)
def get_waves(parts, time):
"""
Generate sine waves based on frequency parts.
"""
num_waves = len(parts)
steps = len(time)
waves = np.zeros((num_waves, steps))
for i in range(num_waves):
waves[i] = np.sin(parts[i] * 2 * np.pi * time)
return waves
def decompose(time, data, freqs, threshold=None):
"""
Decompose and return the individual components of a composite wave form.
Plot each component wave.
"""
powers = transform(time, data, freqs)
peaks, _ = find_peaks(powers, threshold=threshold)
plt.plot(freqs, powers, 'b.--', label='Center of Mass')
plt.plot(freqs[peaks], powers[peaks], 'ro', label='Peaks')
plt.xlabel('Frequency')
plt.legend(), plt.grid()
plt.show()
return get_waves(freqs[peaks], time)
And the signal set-up I used to generate the plots:
# sample data plot: sin with frequencey of 3 hz.
f1 = 3
f2 = 5
start = 0
stop = 1
sample_rate = 0.005
x = np.arange(start, stop, sample_rate)
y = np.cos(f1 * 2 * np.pi * x) + np.sin(f2 * 2 * np.pi *x)
plt.plot(x, y, '.')
plt.xlabel('time')
plt.ylabel('amplitude')
plt.show()
freqs = np.arange(0, 20, .5)
waves = decompose(x, y, freqs, threshold=0.12)
for w in waves:
plt.plot(x, w)
plt.show()

f1 = 3
f2 = 5
start = 0
stop = 1
sample_rate = 0.005
x = np.arange(start, stop, sample_rate)
y = np.cos(f1 * 2 * np.pi * x) + np.sin(f2 * 2 * np.pi *x)
p = np.fft.fft(y)
freqs = np.fft.fftfreq(y.shape[0],sample_rate)
The fft returns the complex values so you need the sqrt of the sum of the squares like you did for mag in transform.
>>> p[:2]
array([-1.42663659e-14+0.00000000e+00j, -1.77635684e-15+1.38777878e-17j])
q = np.absolute(p)
>>> q[:2]
array([1.77641105e-15, 2.70861628e-14])
fft and fftfreqs give you both sides of the transform reflected around zero hz. You can see the negative frequencies at the end.
>>> freqs[-10:]
array([-10., -9., -8., -7., -6., -5., -4., -3., -2., -1.])
You only care about the positive frequencies so you can filter for them and plot.
q = q[freqs > 0]
freqs = freqs[freqs > 0]
plt.bar(freqs,q)
plt.show()
plt.close()
If there is a dc component and you want to see it your filter would be freqs >= 0.
Your example has 200 data points so you get 100 (n/2) positive frequencies and the graph ranges from zero to one hundred Hz with peaks at three and five.
numpy.fft.rfft only computes the positive frequencies. Using numpy.fft.rfftfreq to get the frequencies.
For me, this was/is an awesome resource - The Scientist and Engineer's Guide to
Digital Signal Processing - it's on my desk at work.

Simple DFT Coefficients => Amplitude/Frequencies => Plot

Im trying on DFT and FFT in Python with numpy and pyplot.
My Sample Vector is
x = np.array([1,2,4,3]
The DFT coefficients for that vector are
K = [10+0j, -3+1j, 0+0j, -3-1j]
so basically we have 10, -3+i, 0 and -3-1i as DFT coefficients.
My problem now is to get a combination of sin and cos to fit all 4 points.
Let's assume we have a sample Rate of 1hz.
This is my code :
from matplotlib import pyplot as plt
import numpy as np
x = np.array([1,2,4,3])
fft = np.fft.fft(x)
space = np.linspace(0,4,50)
values = np.array([1,2,3,4])
cos0 = fft[0].real * np.cos(0 * space)
cos1 = fft[1].real * np.cos(1/4 * np.pi * space)
sin1 = fft[1].imag * np.sin(1/4 * np.pi * space)
res = cos0 + cos1 + sin1
plt.scatter(values, x, label="original")
plt.plot(space, cos0, label="cos0")
plt.plot(space, cos1, label="cos1")
plt.plot(space, sin1, label="sin1")
plt.plot(space, res, label="combined")
plt.legend()
As result i get the plot:
(source: heeser-it.de)
Why isnt the final curve hitting any point?
I would appreciate your help. Thanks!
EDIT:
N = 1000
dataPoints = np.linspace(0, np.pi, N)
function = np.sin(dataPoints)
fft = np.fft.fft(function)
F = np.zeros((N,))
for i in range(0, N):
F[i] = (2 * np.pi * i) / N
F_sin = np.zeros((N,N))
F_cos = np.zeros((N,N))
res = 0
for i in range(0, N):
F_sin[i] = fft[i].imag / 500 * np.sin(dataPoints * F[i])
F_cos[i] = fft[i].real / 500* np.cos(dataPoints * F[i])
res = res + F_sin[i] + F_cos[i]
plt.plot(dataPoints, function)
plt.plot(dataPoints, res)
my plot looks like:
(source: heeser-it.de)
where do i fail?

Your testing vector x looks bit like a sawtooth because it rises linearly and then starts to decrease but with that few datapoints it's hard to tell what signal it is. This has an infinite FFT series, which means it has lot of higher harmonic frequency components in it. So to describe it with DTF coefficients and get close to original points, you would have to use
higher sample rate, to get information about higher frequencies (you should learn about nyquist theorem)
more data points (samples), so you can extract more precise information about frequencies in your signal) This means you have to have more items in your array 'x'.
Also you could try to fit some simpler signal. What about you try to fit a sine signal for start? Generate 1000 data points of low frequency sine (1Hz or one cycle per 1000 samples) and then run DTF on it to check if your code works.

There are a few mistakes:
The xs you assigned to the original values are off by one
The frequency you assigned to fft[1] is incorrect
The coefficients are incorrectly scaled
This one works:
from matplotlib import pyplot as plt
import numpy as np
x = np.array([1,2,4,3])
fft = np.fft.fft(x)
space = np.linspace(0,4,50)
values = np.array([0,1,2,3])
cos0 = fft[0].real * np.cos(0 * space)/4
cos1 = fft[1].real * np.cos(1/2 * np.pi * space)/2
sin1 = -fft[1].imag * np.sin(1/2 * np.pi * space)/2
res = cos0 + cos1 + sin1
plt.scatter(values, x, label="original")
plt.plot(space, cos0, label="cos0")
plt.plot(space, cos1, label="cos1")
plt.plot(space, sin1, label="sin1")
plt.plot(space, res, label="combined")
plt.legend()
plt.show()