I'm trying to obtain only positive autocorrelation values from a timeseries waveform using scipy.signal.correlate() which should look like the following:
But I am ending up getting the following - which has both positive and negative values and also a trend present:
Can anyone please tell how to get only positive & de-trended Autocorrelation values?
The dataset for which I'm finding the autocorrelation, is generated using the following code (which you could use as it is for your reference):
import json
import sys, os
import numpy as np
import pandas as pd
import glob
import pickle
from statsmodels.tsa.stattools import adfuller, acf, pacf
from scipy.signal import find_peaks, square
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import matplotlib.pyplot as plt
#GENERATION OF A FUNCTION WITH DUAL SEASONALITY & NOISE
def white_noise(mu, sigma, num_pts):
""" Function to generate Gaussian Normal Noise
Args:
sigma: std value
num_pts: no of points
mu: mean value
Returns:
generated Gaussian Normal Noise
"""
noise = np.random.normal(mu, sigma, num_pts)
return noise
def signal_line_plot(input_signal: pd.Series, title: str = "", y_label: str = "Signal"):
""" Function to plot a time series signal
Args:
input_signal: time series signal that you want to plot
title: title on plot
y_label: label of the signal being plotted
Returns:
signal plot
"""
plt.plot(input_signal)
plt.title(title)
plt.ylabel(y_label)
plt.show()
t_week = np.linspace(1,480, 480)
t_weekend=np.linspace(1,192,192)
T=96 #Time Period
x_weekday = 10*square(2*np.pi*t_week/T, duty=0.7)+10 + white_noise(0, 1,480)
x_weekend = 2*square(2*np.pi*t_weekend/T, duty=0.7)+2 + white_noise(0,1,192)
x_daily_weekly = np.concatenate((x_weekday, x_weekend))
x_daily_weekly_long = np.concatenate((x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly,x_daily_weekly))
signal_line_plot(x_daily_weekly_long)
signal_line_plot(x_daily_weekly_long[0:1000])
#x_daily_weekly_long is the final waveform on which I'm carrying out Autocorrelation
I'm performing Autocorrelation as follows (whose resulting output is as I've shown above, which is what I'm not satisfied with):
#DETERMINING AUTOCORRELATION AND LAG VALUES:
import scipy.signal as signal
autocorr = signal.correlate(x_daily_weekly_long, x_daily_weekly_long, mode = "same")
lags = signal.correlation_lags(len(x_daily_weekly_long), len(x_daily_weekly_long), mode = "same")
#VISUALIZATION:
f = plt.figure()
f.set_figwidth(40)
f.set_figheight(10)
plt.plot(lags, autocorr)
Could anyone please help?
Related
Data clip I'm using
I'm trying to bandpass the attached EEG signal, then apply a hilbert transform and take the absolute of the hilbert to get the instantaneous power (e.g., here). The bandpassed signal looks fine (first plot), and the hilbert of the raw signal looks fine (second plot), but the hilbert of the bandpassed signal does not show up (last plot). The resulting array is: [nan+nanj nan+nanj nan+nanj ... nan+nanj nan+nanj nan+nanj].
Reproducible error with:
import numpy as np
from neurodsp.filt import filter_signal
from scipy import signal
import matplotlib.pyplot as plt
Fs = 1024
LBP, HBP = 1, 100
Chan1 = np.loadtxt('SampleData')
Chan1_BP = filter_signal(Chan1, Fs, 'bandpass', (LBP,HBP))
analytical_signal = signal.hilbert(Chan1)
amplitude_envelope = np.abs(analytical_signal)
#Show bandpassed signal works:
fig0 = plt.figure(figsize=(10, 8))
plt.plot(Chan1)
plt.plot(Chan1_BP)
fig1 = plt.figure(figsize=(10, 8))
plt.plot(Chan1)
plt.plot(amplitude_envelope)
# Now with bandpassed signal
analytical_signal = signal.hilbert(Chan1_BP)
amplitude_envelope = np.abs(analytical_signal)
fig2 = plt.figure(figsize=(10, 8))
plt.plot(Chan1_BP)
plt.plot(amplitude_envelope)
Take a closer look at the values in Chan1_BP. You'll see that the values at the beginning and end of the array are nan. The nans were generated by neurodsp.filt.filter_signal. The default filter used by filter_signal is a FIR filter, and the default behavior is to pad the output with nans for values that cannot be computed with the full length of the FIR filter.
You can change that behavior by passing remove_edges=False, e.g.
Chan1_BP = filter_signal(Chan1, Fs, 'bandpass', (LBP,HBP), remove_edges=False)
With that change, the plots should look like you expected.
I have a sound signal of 5 secs length and it is from the sound of a propeller. I need to find rpm of the propeller by finding frequency of the envelopes.
import wave
import numpy as np
import matplotlib.pyplot as plt
raw = wave.open('/content/drive/MyDrive/Demon.wav','r')
signal = raw.readframes(-1)
signal = np.frombuffer(signal , dtype="int16")
frate = raw.getframerate()
time = np.linspace(0,len(signal) / frate,num = len(signal))
plt.figure(1)
plt.title("Sound Wave")
plt.xlabel("Time")
plt.plot(time, signal)
plt.show()
Here is the link to the sound file itself: https://sndup.net/5v3j
And since it is a 5 second-length signal and has 80.000 samples, I want to see it in details by looking 1 second part of the signal.
partial_signal = signal [1 : 16000]
partial_time = time[1 : 16000]
plt.plot(partial_time,partial_signal)
plt.show()
Output of the plot is shown below.
Edit: Looks like image will not show up here is the link to the image:
https://imgur.com/P5lnSM1
Now I need to find frequency of the envelopes thus rpm of the propeller by using only python.
You can do that quite easily with a fast Fourier transform (FFT) applied on the signal amplitude. Here is an example:
import wave
import numpy as np
import matplotlib.pyplot as plt
from scipy.fft import rfft, rfftfreq
from scipy.ndimage import gaussian_filter
raw = wave.open('Demon.wav','r')
signal = raw.readframes(-1)
signal = np.frombuffer(signal , dtype="int16")
frate = raw.getframerate()
time = np.linspace(0,len(signal) / frate,num = len(signal))
# Compute the amplitude of the sound signal
signalAmplitude = signal.astype(np.float64)**2
# Filter the signal to remove very short-timed amplitude modulations (<= 1 ms)
signalAmplitude = gaussian_filter(signalAmplitude, sigma=frate/1000)
# Compute the frequency amplitude of the FFT signal
tmpFreq = np.abs(rfft(signalAmplitude))
# Get the associated practical frequency for this signal
hzFreq = rfftfreq(signal.shape[0], d=1/frate)
finalFrequency = hzFreq[1+tmpFreq[1:].argmax()]
print(finalFrequency)
# Show sound frequency diagram
plt.xticks(np.arange(21))
plt.xlim([1, 20]) # Show only interesting low frequencies
plt.plot(hzFreq, tmpFreq)
plt.show()
The frequency diagram is the following:
The final detected frequency is 3.0 Hz which is very consistent with what we can hear.
I am trying to integrate the area of multiple peaks in Python. I am measuring different amplitudes of signals from an oscilloscope and managed to plot them as a function of time as shown here
.
I am trying now to use trapezoidal area function for each peak in order to plot area distribution as a function of count. I have written this code but there is no results.
import numpy as np
import matplotlib.pyplot as plt
from readTrc import Trc
from numpy import trapz
trc = Trc()
fName = "./Am.trc"
datX, datY, d = trc.open(fName)
y = np.reshape(datY, (10000,4002))
for i in range(10000):
plt.plot(y[i,:]) # Here to get all amplitudes as a function of time
area = trapz(datX, datY)
What am I doing wrong here and how can I accomplish this?
I am using the below codes so as to generate a Pulse Amplitude
Modulation signal by using the Boolean operation between sine wave and
Pulse Width Modulation(PWM) signal.I am using the vectorisation method
so as to get zero values where the PWM signal is low(zero or false) and
sine wave where the PWM values as high (True or one). Please refer the
below screen shot for the required output.In addition to this how do
you automate the PAM wave generation as I am facing problem with
spacing of x values?
import numpy as np
import matplotlib.pyplot as plt
from pylab import *
percent=50.0
TimePeriod=10.0 #Frozen Value Do not change
Cycles=10 #Frozen Value Do not change
dt=0.01 #Frozen Value Do not change
t=np.arange(0,Cycles*TimePeriod,dt);
pwm= t%TimePeriod<TimePeriod*percent/100
x=np.linspace(-10,10,10000) #Frozen Value Do not change
y=(np.sin(x))
y[(pwm =='False')] = 0 #Vectorisation for zero values
y[(pwm =='True')] = (y-pwm) # #Vectorisation for sine wave
plt.plot(t,y)
plt.ylim([-3,3])
plt.grid()
plt.show()
When removing the line y[(pwm =='True')] = (y-pwm) (which I don't understand) and not comparing to strings, you would get the following, which looks pretty much like the desired plot.
import numpy as np
import matplotlib.pyplot as plt
percent=40.0
TimePeriod=10.0
Cycles=30
dt=0.01
t=np.arange(0,Cycles*TimePeriod,dt);
pwm= (t%TimePeriod) < (TimePeriod*percent/100)
x=np.linspace(-10,10,len(pwm))
y=(np.sin(x))
y[pwm == 0] = 0
plt.plot(t,y)
plt.ylim([-3,3])
plt.grid()
plt.show()
I use Tektronix oscilloscope to perform some signal acquisition. I get 10.000 measurement points (few signal periods) and I have to do a frequency analysis on that set of data. My signal is 8MHz sine wave. When I use either SciPy or NumPy I get the same result - frequencies are spreaded too wide. The distance between two values is 500kHz and the highest frequency is 2.5GHz (absurd). When I want to measure frequency bandwidth around 8MHz I can only get exact values of 7.5, 8.0 and 8.5 MHz. I tried to change sample spacing determined by (x[1]-x[0]) and I got nothing better.
def CalculateFFT(t_val,p_val):
x = t_val #Two parameters: [x,y] values
y = lambda x: p_val
com_signal = y(x) # Combined signal
FFT_val = abs(scipy.fft(com_signal))
freq_val = scipy.fftpack.fftfreq(len(com_signal), x[1]-x[0])
spec_val = 20*scipy.log10(FFT_val)
return freq_val, spec_val
It is worth reading in more depth how DFFTs work but you should always have the following formulae in mind. For a time series with n points and maximum time Tmax, the time resolution is given by dt = Tmax / n
A DFFT will produce n points with
Fmax = 1 / dt
dF = 1 / Tmax
You seem to suggest the maximum frequency is sufficient (so the time resolution is okay) but the frequency resolution isn't good enough: you need to collect more data, at the same time resolution.
If (1) the sampling time is too short, (2) you require higher estimation frequency accuracy, and, (3) you know that your signal is a sine wave, then you can fit the signal to a sine wave. Like in How do I fit a sine curve to my data with pylab and numpy?,
with the exception that the frequency needs to be added.
Here is an example figure with a frequency of around 8 MHz:
Below is the example code:
""" Modified from https://stackoverflow.com/a/16716964/6036470 """
from numpy import sin, linspace, pi,average;
from pylab import plot, show, title, xlabel, ylabel, subplot, scatter
from scipy import fft, arange, ifft
import scipy
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import leastsq
ff = 8e6; # frequency of the signal
Fs = ff*128; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = arange(0,((1/ff)/128)*(128)*5,Ts) # time vector
A = 2.5;
ff_0 = 8.1456e6
y = A*np.sin(2*np.pi*ff_0*t+15.38654*pi/180) + np.random.randn(len(t))/5
guess_b = 0
guess_a = y.std()*2**0.5;
guess_c = 10*pi/180
guess_d = ff*0.98*2*pi
fig = plt.figure(facecolor="white")
plt.plot(t,y,'.', label='Signal Fred. %0.4f Hz'%(ff_0/1e6))
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.grid(alpha=0.5);
optimize_func = lambda x: (x[0]*np.sin(x[2]*t+x[1]) - y);
est_a, est_c, est_d = leastsq(optimize_func, [guess_a, guess_c, guess_d])[0]
data_fit = est_a*np.sin(est_d*t+est_c) ;
plt.plot(t,data_fit,label='Fitted Est. Freq. %0.4f Hz'%(est_d/(2*pi)/1e6))
plt.legend()
plt.tight_layout();
plt.show();
fig.save("sinfit.png")