We Keep Coding

Time series dBFS plot output modification - current output plot not as expected (matplotlib)

I'm trying to plot the Amplitude (dBFS) vs. Time (s) plot of an audio (.wav) file using matplotlib. I managed to do that with the following code:
def convert_to_decibel(sample):
ref = 32768 # Using a signed 16-bit PCM format wav file. So, 2^16 is the max. value.
if sample!=0:
return 20 * np.log10(abs(sample) / ref)
else:
return 20 * np.log10(0.000001)
from scipy.io.wavfile import read as readWav
from scipy.fftpack import fft
import matplotlib.pyplot as gplot1
import matplotlib.pyplot as gplot2
import numpy as np
import struct
import gc
wavfile1 = '/home/user01/audio/speech.wav'
wavsamplerate1, wavdata1 = readWav(wavfile1)
wavdlen1 = wavdata1.size
wavdtype1 = wavdata1.dtype
gplot1.rcParams['figure.figsize'] = [15, 5]
pltaxis1 = gplot1.gca()
gplot1.axhline(y=0, c="black")
gplot1.xticks(np.arange(0, 10, 0.5))
gplot1.yticks(np.arange(-200, 200, 5))
gplot1.grid(linestyle = '--')
wavdata3 = np.array([convert_to_decibel(i) for i in wavdata1], dtype=np.int16)
yvals3 = wavdata3
t3 = wavdata3.size / wavsamplerate1
xvals3 = np.linspace(0, t3, wavdata3.size)
pltaxis1.set_xlim([0, t3 + 2])
pltaxis1.set_title('Amplitude (dBFS) vs Time(s)')
pltaxis1.plot(xvals3, yvals3, '-')
which gives the following output:
I had also plotted the Power Spectral Density (PSD, in dBm) using the code below:
from scipy.signal import welch as psd # Computes PSD using Welch's method.
fpsd, wPSD = psd(wavdata1, wavsamplerate1, nperseg=1024)
gplot2.rcParams['figure.figsize'] = [15, 5]
pltpsdm = gplot2.gca()
gplot2.axhline(y=0, c="black")
pltpsdm.plot(fpsd, 20*np.log10(wPSD))
gplot2.xticks(np.arange(0, 4000, 400))
gplot2.yticks(np.arange(-150, 160, 10))
pltpsdm.set_xlim([0, 4000])
pltpsdm.set_ylim([-150, 150])
gplot2.grid(linestyle = '--')
which gives the output as:
The second output above, using the Welch's method plots a more presentable output. The dBFS plot though informative is not very presentable IMO. Is this because of:
the difference in the domains (time in case of 1st output vs frequency in the 2nd output)?
the way plot function is implemented in pyplot?
Also, is there a way I can plot my dBFS output as a peak-to-peak style of plot just like in my PSD (dBm) plot rather than a dense stem plot?
Would be much helpful and would appreciate any pointers, answers or suggestions from experts here as I'm just a beginner with matplotlib and plots in python in general.

TLNR
This has nothing to do with pyplot.
The frequency domain is different from the time domain, but that's not why you didn't get what you want.
The calculation of dbFS in your code is wrong.
You should frame your data, calculate RMSs or peaks in every frame, and then convert that value to dbFS instead of applying this transformation to every sample point.
When we talk about the amplitude, we are talking about a periodic signal. And when we read in a series of data from a sound file, we read in a series of sample points of a signal(may be or be not periodic). The value of every sample point represents a, say, voltage value, or sound pressure value sampled at a specific time.
We assume that, within a very short time interval, maybe 10ms for example, the signal is stationary. Every such interval is called a frame.
Some specific function is applied to each frame usually, to reduce the sudden change at the edge of this frame, and these functions are called window functions. If you did nothing to every frame, you added rectangle windows to them.
An example: when the sampling frequency of your sound is 44100Hz, in a 10ms-long frame, there are 44100*0.01=441 sample points. That's what the nperseg argument means in your psd function but it has nothing to do with dbFS.
Given the knowledge above, now we can talk about the amplitude.
There are two methods a get the value of amplitude in every frame:
The most straightforward one is to get the maximum(peak) values in every frame.
Another one is to calculate the RMS(Root Mean Sqaure) of every frame.
After that, the peak values or RMS values can be converted to dbFS values.
Let's start coding:
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
# Determine full scall(maximum possible amplitude) by bit depth
bit_depth = 16
full_scale = 2 ** bit_depth
# dbFS function
to_dbFS = lambda x: 20 * np.log10(x / full_scale)
# Read in the wave file
fname = "01.wav"
fs,data = wavfile.read(fname)
# Determine frame length(number of sample points in a frame) and total frame numbers by window length(how long is a frame in seconds)
window_length = 0.01
signal_length = data.shape[0]
frame_length = int(window_length * fs)
nframes = signal_length // frame_length
# Get frames by broadcast. No overlaps are used.
idx = frame_length * np.arange(nframes)[:,None] + np.arange(frame_length)
frames = data[idx].astype("int64") # Convert to in 64 to avoid integer overflow
# Get RMS and peaks
rms = ((frames**2).sum(axis=1)/frame_length)**.5
peaks = np.abs(frames).max(axis=1)
# Convert them to dbfs
dbfs_rms = to_dbFS(rms)
dbfs_peak = to_dbFS(peaks)
# Let's start to plot
# Get time arrays of every sample point and ever frame
frame_time = np.arange(nframes) * window_length
data_time = np.linspace(0,signal_length/fs,signal_length)
# Plot
f,ax = plt.subplots()
ax.plot(data_time,data,color="k",alpha=.3)
# Plot the dbfs values on a twin x Axes since the y limits are not comparable between data values and dbfs
tax = ax.twinx()
tax.plot(frame_time,dbfs_rms,label="RMS")
tax.plot(frame_time,dbfs_peak,label="Peak")
tax.legend()
f.tight_layout()
# Save serval details
f.savefig("whole.png",dpi=300)
ax.set_xlim(1,2)
f.savefig("1-2sec.png",dpi=300)
ax.set_xlim(1.295,1.325)
f.savefig("1.2-1.3sec.png",dpi=300)
The whole time span looks like(the unit of the right axis is dbFS):
And the voiced part looks like:
You can see that the dbFS values become greater while the amplitudes become greater at the vowel start point:

Change the melody of human speech using FFT and polynomial interpolation

I'm trying to do the following:
Extract the melody of me asking a question (word "Hey?" recorded to
wav) so I get a melody pattern that I can apply to any other
recorded/synthesized speech (basically how F0 changes in time).
Use polynomial interpolation (Lagrange?) so I get a function that describes the melody (approximately of course).
Apply the function to another recorded voice sample. (eg. word "Hey." so it's transformed to a question "Hey?", or transform the end of a sentence to sound like a question [eg. "Is it ok." => "Is it ok?"]). Voila, that's it.
What I have done? Where am I?
Firstly, I have dived into the math that stands behind the fft and signal processing (basics). I want to do it programatically so I decided to use python.
I performed the fft on the entire "Hey?" voice sample and got data in frequency domain (please don't mind y-axis units, I haven't normalized them)
So far so good. Then I decided to divide my signal into chunks so I get more clear frequency information - peaks and so on - this is a blind shot, me trying to grasp the idea of manipulating the frequency and analyzing the audio data. It gets me nowhere however, not in a direction I want, at least.
Now, if I took those peaks, got an interpolated function from them, and applied the function on another voice sample (a part of a voice sample, that is also ffted of course) and performed inversed fft I wouldn't get what I wanted, right?
I would only change the magnitude so it wouldn't affect the melody itself (I think so).
Then I used spec and pyin methods from librosa to extract the real F0-in-time - the melody of asking question "Hey?". And as we would expect, we can clearly see an increase in frequency value:
And a non-question statement looks like this - let's say it's moreless constant.
The same applies to a longer speech sample:
Now, I assume that I have blocks to build my algorithm/process but I still don't know how to assemble them beacause there are some blanks in my understanding of what's going on under the hood.
I consider that I need to find a way to map the F0-in-time curve from the spectrogram to the "pure" FFT data, get an interpolated function from it and then apply the function on another voice sample.
Is there any elegant (inelegant would be ok too) way to do this? I need to be pointed in a right direction beceause I can feel I'm close but I'm basically stuck.
The code that works behind the above charts is taken just from the librosa docs and other stackoverflow questions, it's just a draft/POC so please don't comment on style, if you could :)
fft in chunks:
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
import os
file = os.path.join("dir", "hej_n_nat.wav")
fs, signal = wavfile.read(file)
CHUNK = 1024
afft = np.abs(np.fft.fft(signal[0:CHUNK]))
freqs = np.linspace(0, fs, CHUNK)[0:int(fs / 2)]
spectrogram_chunk = freqs / np.amax(freqs * 1.0)
# Plot spectral analysis
plt.plot(freqs[0:250], afft[0:250])
plt.show()
spectrogram:
import librosa.display
import numpy as np
import matplotlib.pyplot as plt
import os
file = os.path.join("/path/to/dir", "hej_n_nat.wav")
y, sr = librosa.load(file, sr=44100)
f0, voiced_flag, voiced_probs = librosa.pyin(y, fmin=librosa.note_to_hz('C2'), fmax=librosa.note_to_hz('C7'))
times = librosa.times_like(f0)
D = librosa.amplitude_to_db(np.abs(librosa.stft(y)), ref=np.max)
fig, ax = plt.subplots()
img = librosa.display.specshow(D, x_axis='time', y_axis='log', ax=ax)
ax.set(title='pYIN fundamental frequency estimation')
fig.colorbar(img, ax=ax, format="%+2.f dB")
ax.plot(times, f0, label='f0', color='cyan', linewidth=2)
ax.legend(loc='upper right')
plt.show()
Hints, questions and comments much appreciated.

The problem was that I didn't know how to modify the fundamental frequency (F0). By modifying it I mean modify F0 and its harmonics, as well.
The spectrograms in question show frequencies at certain points in time with power (dB) of certain frequency point.
Since I know which time bin holds which frequency from the melody (green line below) ...
....I need to compute a function that represents that green line so I can apply it to other speech samples.
So I need to use some interpolation method which takes as parameters the sample F0 function points.
One need to remember that degree of the polynomial should equal to the number of points. The example doesn't have that unfortunately, but the effect is somehow ok as for the prototype.
def _get_bin_nr(val, bins):
the_bin_no = np.nan
for b in range(0, bins.size - 1):
if bins[b] <= val < bins[b + 1]:
the_bin_no = b
elif val > bins[bins.size - 1]:
the_bin_no = bins.size - 1
return the_bin_no
def calculate_pattern_poly_coeff(file_name):
y_source, sr_source = librosa.load(os.path.join(ROOT_DIR, file_name), sr=sr)
f0_source, voiced_flag, voiced_probs = librosa.pyin(y_source, fmin=librosa.note_to_hz('C2'),
fmax=librosa.note_to_hz('C7'), pad_mode='constant',
center=True, frame_length=4096, hop_length=512, sr=sr_source)
all_freq_bins = librosa.core.fft_frequencies(sr=sr, n_fft=n_fft)
f0_freq_bins = list(filter(lambda x: np.isfinite(x), map(lambda val: _get_bin_nr(val, all_freq_bins), f0_source)))
return np.polynomial.polynomial.polyfit(np.arange(0, len(f0_freq_bins), 1), f0_freq_bins, 3)
def calculate_pattern_poly_func(coefficients):
return np.poly1d(coefficients)
Method calculate_pattern_poly_coeff calculates polynomial coefficients.
Using pythons poly1d lib I can compute function which can modify the speech. How to do that?
I just need to move up or down all values vertically at certain point in time.
for instance I want to move all frequencies at time bin 0,75 seconds up 3 times -> it means that frequency will be increased and the melody at that point will sound higher.
Code:
def transform(sentence_audio_sample, mode=None, show_spectrograms=False, frames_from_end_to_transform=12):
# cutting out silence
y_trimmed, idx = librosa.effects.trim(sentence_audio_sample, top_db=60, frame_length=256, hop_length=64)
stft_original = librosa.stft(y_trimmed, hop_length=hop_length, pad_mode='constant', center=True)
stft_original_roll = stft_original.copy()
rolled = stft_original_roll.copy()
source_frames_count = np.shape(stft_original_roll)[1]
sentence_ending_first_frame = source_frames_count - frames_from_end_to_transform
sentence_len = np.shape(stft_original_roll)[1]
for i in range(sentence_ending_first_frame + 1, sentence_len):
if mode == 'question':
by = int(_question_pattern(i) / 500)
elif mode == 'exclamation':
by = int(_exclamation_pattern(i) / 500)
else:
by = 0
rolled = _roll_column(rolled, i, by)
transformed_data = librosa.istft(rolled, hop_length=hop_length, center=True)
def _roll_column(two_d_array, column, shift):
two_d_array[:, column] = np.roll(two_d_array[:, column], shift)
return two_d_array
In this case I am simply rolling up or down frequencies referencing certain time bin.
This needs to be polished as it doesn't take into consideration an actual state of the transformed sample. It just rolls it up/down according to the factor calculated using the polynomial function computer earlier.
You can check full code of my project at github, "audio" package contains pattern calculator and audio transform algorithm described above.
Feel free to ask if something's unclear :)

How to get and plot a signal envelope

I would like to know if there is a function envelope in Python to have the same result as this
I have already tried an envelope function in Python but there is this result and it doesn't correspond with what I want.

Though you don't mention exactly what function you use, it seems like you are using two different kinds of envelopes.
The way you call envelope in matlab, the relevant description is:
[yupper,ylower] = envelope(x) returns the upper and lower envelopes of
the input sequence, x, as the magnitude of its analytic signal. The
analytic signal of x is found using the discrete Fourier transform as
implemented in hilbert. The function initially removes the mean of x
and adds it back after computing the envelopes. If x is a matrix, then
envelope operates independently over each column of x.
Based on this, I suppose you would be looking for a way to get the Hilber transform in python. An example of this can be found here:
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import hilbert, chirp
duration = 1.0
fs = 400.0
samples = int(fs*duration)
t = np.arange(samples) / fs
signal = chirp(t, 20.0, t[-1], 100.0)
signal *= (1.0 + 0.5 * np.sin(2.0*np.pi*3.0*t) )
analytic_signal = hilbert(signal)
amplitude_envelope = np.abs(analytic_signal)
instantaneous_phase = np.unwrap(np.angle(analytic_signal))
instantaneous_frequency = np.diff(instantaneous_phase) / (2.0*np.pi) * fs
fig = plt.figure()
ax0 = fig.add_subplot(211)
ax0.plot(t, signal, label='signal')
ax0.plot(t, amplitude_envelope, label='envelope')
ax0.set_xlabel("time in seconds")
ax0.legend()
ax1 = fig.add_subplot(212)
ax1.plot(t[1:], instantaneous_frequency)
ax1.set_xlabel("time in seconds")
ax1.set_ylim(0.0, 120.0)
Resulting in:

Sometimes I would use obspy.signal.filter.envelope(data_array); But you can only get the upper line in your given example.
Obspy is a very useful package dealing with seismogram.

Phase shift of the signal after using FIR filter firwin in Python

So after my two last questions I come to my actual problem. Maybe somebody finds the error in my theoretical procedure or I did something wrong in programming.
I am implementing a bandpass filter in Python using scipy.signal (using the firwin function). My original signal consists of two frequencies (w_1=600Hz, w_2=800Hz). There might be a lot more frequencies that's why I need a bandpass filter.
In this case I want to filter the frequency band around 600 Hz, so I took 600 +/- 20Hz as cutoff frequencies. When I implemented the filter and reproduce the signal in the time domain using lfilter the right frequency is filtered.
To get rid of the phase shift I plotted the frequency response by using scipy.signal.freqz with the return h of firwin as numerator and 1 as predefined denumerator.
As described in the documentation of freqz I plotted the phase (== angle in the doc) as well and was able to look at the frequency response plot to get the phase shift for the frequency 600 Hz of the filtered signal.
So the phase delay t_p is
t_p=-(Tetha(w))/(w)
Unfortunately when I add this phase delay to the time data of my filtered signal, it has not got the same phase as the original 600 Hz signal.
I added the code. It is weird, before eliminating some part of the code to keep the minimum, the filtered signal started at the correct amplitude - now it is even worse.
################################################################################
#
# Filtering test
#
################################################################################
#
from math import *
import numpy as np
from scipy import signal
from scipy.signal import firwin, lfilter, lti
from scipy.signal import freqz
import matplotlib.pyplot as plt
import matplotlib.colors as colors
################################################################################
# Nb of frequencies in the original signal
nfrq = 2
F = [60,80]
################################################################################
# Sampling:
nitper = 16
nper = 50.
fmin = np.min(F)
fmax = np.max(F)
T0 = 1./fmin
dt = 1./fmax/nitper
#sampling frequency
fs = 1./dt
nyq_rate= fs/2
nitpermin = nitper*fmax/fmin
Nit = int(nper*nitpermin+1)
tps = np.linspace(0.,nper*T0,Nit)
dtf = fs/Nit
################################################################################
# Build analytic signal
# s = completeSignal(F,Nit,tps)
scomplete = np.zeros((Nit))
omg1 = 2.*pi*F[0]
omg2 = 2.*pi*F[1]
scomplete=scomplete+np.sin(omg1*tps)+np.sin(omg2*tps)
#ssingle = singleSignals(nfrq,F,Nit,tps)
ssingle=np.zeros((nfrq,Nit))
ssingle[0,:]=ssingle[0,:]+np.sin(omg1*tps)
ssingle[1,:]=ssingle[0,:]+np.sin(omg2*tps)
################################################################################
## Construction of the desired bandpass filter
lowcut = (60-2) # desired cutoff frequencies
highcut = (60+2)
ntaps = 451 # the higher and closer the signal frequencies, the more taps for the filter are required
taps_hamming = firwin(ntaps,[lowcut/nyq_rate, highcut/nyq_rate], pass_zero=False)
# Use lfilter to get the filtered signal
filtered_signal = lfilter(taps_hamming, 1, scomplete)
# The phase delay of the filtered signal
delay = ((ntaps-1)/2)/fs
plt.figure(1, figsize=(12, 9))
# Plot the signals
plt.plot(tps, scomplete,label="Original signal with %s freq" % nfrq)
plt.plot(tps-delay, filtered_signal,label="Filtered signal %s freq " % F[0])
plt.plot(tps, ssingle[0,:],label="original signal %s Hz" % F[0])
plt.grid(True)
plt.legend()
plt.xlim(0,1)
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
# Plot the frequency responses of the filter.
plt.figure(2, figsize=(12, 9))
plt.clf()
# First plot the desired ideal response as a green(ish) rectangle.
rect = plt.Rectangle((lowcut, 0), highcut - lowcut, 5.0,facecolor="#60ff60", alpha=0.2,label="ideal filter")
plt.gca().add_patch(rect)
# actual filter
w, h = freqz(taps_hamming, 1, worN=1000)
plt.plot((fs * 0.5 / np.pi) * w, abs(h), label="designed rectangular window filter")
plt.xlim(0,2*F[1])
plt.ylim(0, 1)
plt.grid(True)
plt.legend()
plt.xlabel('Frequency (Hz)')
plt.ylabel('Gain')
plt.title('Frequency response of FIR filter, %d taps' % ntaps)
plt.show()'

The delay of your FIR filter is simply 0.5*(n - 1)/fs, where n is the number of filter coefficients (i.e. "taps") and fs is the sample rate. Your implementation of this delay is fine.
The problem is that your array of time values tps is not correct. Take a look
at 1.0/(tps[1] - tps[0]); you'll see that it does not equal fs.
Change this:
tps = np.linspace(0.,nper*T0,Nit)
to, for example, this:
T = Nit / fs
tps = np.linspace(0., T, Nit, endpoint=False)
and your plots of the original and filtered 60 Hz signals will line up beautifully.
For another example, see http://wiki.scipy.org/Cookbook/FIRFilter.
In the script there, the delay is calculated on line 86. Below this, the delay is used to plot the original signal aligned with the filtered signal.
Note: The cookbook example uses scipy.signal.lfilter to apply the filter. A more efficient approach is to use numpy.convolve.

Seems like you may have had this answered already, but I believe that this is what the filtfilt function is used for. Basically, it does both a forward sweep and a backward sweep through your data, thus reversing the phase shift introduced by the initial filtering. Might be worth looking into.

Frequency resolution issue using FFT in numpy

I use Tektronix oscilloscope to perform some signal acquisition. I get 10.000 measurement points (few signal periods) and I have to do a frequency analysis on that set of data. My signal is 8MHz sine wave. When I use either SciPy or NumPy I get the same result - frequencies are spreaded too wide. The distance between two values is 500kHz and the highest frequency is 2.5GHz (absurd). When I want to measure frequency bandwidth around 8MHz I can only get exact values of 7.5, 8.0 and 8.5 MHz. I tried to change sample spacing determined by (x[1]-x[0]) and I got nothing better.
def CalculateFFT(t_val,p_val):
x = t_val #Two parameters: [x,y] values
y = lambda x: p_val
com_signal = y(x) # Combined signal
FFT_val = abs(scipy.fft(com_signal))
freq_val = scipy.fftpack.fftfreq(len(com_signal), x[1]-x[0])
spec_val = 20*scipy.log10(FFT_val)
return freq_val, spec_val

It is worth reading in more depth how DFFTs work but you should always have the following formulae in mind. For a time series with n points and maximum time Tmax, the time resolution is given by dt = Tmax / n
A DFFT will produce n points with
Fmax = 1 / dt
dF = 1 / Tmax
You seem to suggest the maximum frequency is sufficient (so the time resolution is okay) but the frequency resolution isn't good enough: you need to collect more data, at the same time resolution.

If (1) the sampling time is too short, (2) you require higher estimation frequency accuracy, and, (3) you know that your signal is a sine wave, then you can fit the signal to a sine wave. Like in How do I fit a sine curve to my data with pylab and numpy?,
with the exception that the frequency needs to be added.
Here is an example figure with a frequency of around 8 MHz:
Below is the example code:
""" Modified from https://stackoverflow.com/a/16716964/6036470 """
from numpy import sin, linspace, pi,average;
from pylab import plot, show, title, xlabel, ylabel, subplot, scatter
from scipy import fft, arange, ifft
import scipy
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import leastsq
ff = 8e6; # frequency of the signal
Fs = ff*128; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = arange(0,((1/ff)/128)*(128)*5,Ts) # time vector
A = 2.5;
ff_0 = 8.1456e6
y = A*np.sin(2*np.pi*ff_0*t+15.38654*pi/180) + np.random.randn(len(t))/5
guess_b = 0
guess_a = y.std()*2**0.5;
guess_c = 10*pi/180
guess_d = ff*0.98*2*pi
fig = plt.figure(facecolor="white")
plt.plot(t,y,'.', label='Signal Fred. %0.4f Hz'%(ff_0/1e6))
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.grid(alpha=0.5);
optimize_func = lambda x: (x[0]*np.sin(x[2]*t+x[1]) - y);
est_a, est_c, est_d = leastsq(optimize_func, [guess_a, guess_c, guess_d])[0]
data_fit = est_a*np.sin(est_d*t+est_c) ;
plt.plot(t,data_fit,label='Fitted Est. Freq. %0.4f Hz'%(est_d/(2*pi)/1e6))
plt.legend()
plt.tight_layout();
plt.show();
fig.save("sinfit.png")