I have 1024 samples and I want to cut them into 32 chunks of 32 and run FFT on each one of them and plot it via a Frequency-Amplitude spectrum, I have most of the code working just the part of applying FFT to each chunk is not working, however I am able to apply FFT to the whole samples array.
I tried doing something like that:
realFFT = [for chunk in chunks(amplitude,32): np.fft.fft(chunk)]
but that is wrong syntax
I also tried going over the chunks array by casting it to a List and then saving it all to another list but that didn't work either.
Here is my code:
# Python example - Fourier transform using numpy.fft method
import numpy as np
import matplotlib.pyplot as plotter
from os import times
from PIL import Image
import numpy as np
from numpy.lib.type_check import real
def chunks(lst, n):
"""Yield successive n-sized chunks from lst."""
for i in range(0, len(lst), n):
yield lst[i:i + n]
# How many time points are needed i,e., Sampling Frequency
samplingFrequency = 100
# At what intervals time points are sampled
samplingInterval = 1 / samplingFrequency
# Begin time period of the signals
beginTime = 0
# End time period of the signals
endTime = 10.24
# Frequency of the signals
signal1Frequency = 4
signal2Frequency = 7
# Time points
time = np.arange(beginTime, endTime, samplingInterval)
# Create two sine waves
amplitude1 = 0.7* np.sin(2*np.pi*signal1Frequency*time)
amplitude2 = np.sin(2*np.pi*signal2Frequency*time)
# Create subplot
figure, axis = plotter.subplots(2, 1)
plotter.subplots_adjust(hspace=2)
# Time domain representation for sine wave 1
amplitude = amplitude1
axis[0].set_title('Sine wave with a frequency of 4 Hz')
axis[0].plot(time, amplitude)
axis[0].set_xlabel('Time')
axis[0].set_ylabel('Amplitude')
# Frequency domain representation
realFFT = [for chunk in chunks(amplitude,32): np.fft.fft(chunk)]
#fourierTransform = np.fft.fft(amplitude) # Normalize amplitude
fourierTransform = realFFT[range(int(len(amplitude)/2))] # Exclude sampling frequency
tpCount = len(amplitude)
values = np.arange(int(tpCount/2))
timePeriod = tpCount/samplingFrequency
frequencies = values/timePeriod
# Frequency domain representation
axis[1].set_title('Fourier transform depicting the frequency components')
#dBm = 30 + (20 * np.log10(abs(fourierTransform)))
axis[1].plot(frequencies, abs(fourierTransform))
axis[1].set_xlabel('Frequency')
axis[1].set_ylabel('Amplitude')
plotter.show()
You have the syntax slightly wrong
realFFT = [np.fft.fft(chunk) for chunk in chunks(amplitude, 32)]
The syntax of list comprehension is a little bit different that a for loop.
Related
I'm trying to plot the Amplitude (dBFS) vs. Time (s) plot of an audio (.wav) file using matplotlib. I managed to do that with the following code:
def convert_to_decibel(sample):
ref = 32768 # Using a signed 16-bit PCM format wav file. So, 2^16 is the max. value.
if sample!=0:
return 20 * np.log10(abs(sample) / ref)
else:
return 20 * np.log10(0.000001)
from scipy.io.wavfile import read as readWav
from scipy.fftpack import fft
import matplotlib.pyplot as gplot1
import matplotlib.pyplot as gplot2
import numpy as np
import struct
import gc
wavfile1 = '/home/user01/audio/speech.wav'
wavsamplerate1, wavdata1 = readWav(wavfile1)
wavdlen1 = wavdata1.size
wavdtype1 = wavdata1.dtype
gplot1.rcParams['figure.figsize'] = [15, 5]
pltaxis1 = gplot1.gca()
gplot1.axhline(y=0, c="black")
gplot1.xticks(np.arange(0, 10, 0.5))
gplot1.yticks(np.arange(-200, 200, 5))
gplot1.grid(linestyle = '--')
wavdata3 = np.array([convert_to_decibel(i) for i in wavdata1], dtype=np.int16)
yvals3 = wavdata3
t3 = wavdata3.size / wavsamplerate1
xvals3 = np.linspace(0, t3, wavdata3.size)
pltaxis1.set_xlim([0, t3 + 2])
pltaxis1.set_title('Amplitude (dBFS) vs Time(s)')
pltaxis1.plot(xvals3, yvals3, '-')
which gives the following output:
I had also plotted the Power Spectral Density (PSD, in dBm) using the code below:
from scipy.signal import welch as psd # Computes PSD using Welch's method.
fpsd, wPSD = psd(wavdata1, wavsamplerate1, nperseg=1024)
gplot2.rcParams['figure.figsize'] = [15, 5]
pltpsdm = gplot2.gca()
gplot2.axhline(y=0, c="black")
pltpsdm.plot(fpsd, 20*np.log10(wPSD))
gplot2.xticks(np.arange(0, 4000, 400))
gplot2.yticks(np.arange(-150, 160, 10))
pltpsdm.set_xlim([0, 4000])
pltpsdm.set_ylim([-150, 150])
gplot2.grid(linestyle = '--')
which gives the output as:
The second output above, using the Welch's method plots a more presentable output. The dBFS plot though informative is not very presentable IMO. Is this because of:
the difference in the domains (time in case of 1st output vs frequency in the 2nd output)?
the way plot function is implemented in pyplot?
Also, is there a way I can plot my dBFS output as a peak-to-peak style of plot just like in my PSD (dBm) plot rather than a dense stem plot?
Would be much helpful and would appreciate any pointers, answers or suggestions from experts here as I'm just a beginner with matplotlib and plots in python in general.
TLNR
This has nothing to do with pyplot.
The frequency domain is different from the time domain, but that's not why you didn't get what you want.
The calculation of dbFS in your code is wrong.
You should frame your data, calculate RMSs or peaks in every frame, and then convert that value to dbFS instead of applying this transformation to every sample point.
When we talk about the amplitude, we are talking about a periodic signal. And when we read in a series of data from a sound file, we read in a series of sample points of a signal(may be or be not periodic). The value of every sample point represents a, say, voltage value, or sound pressure value sampled at a specific time.
We assume that, within a very short time interval, maybe 10ms for example, the signal is stationary. Every such interval is called a frame.
Some specific function is applied to each frame usually, to reduce the sudden change at the edge of this frame, and these functions are called window functions. If you did nothing to every frame, you added rectangle windows to them.
An example: when the sampling frequency of your sound is 44100Hz, in a 10ms-long frame, there are 44100*0.01=441 sample points. That's what the nperseg argument means in your psd function but it has nothing to do with dbFS.
Given the knowledge above, now we can talk about the amplitude.
There are two methods a get the value of amplitude in every frame:
The most straightforward one is to get the maximum(peak) values in every frame.
Another one is to calculate the RMS(Root Mean Sqaure) of every frame.
After that, the peak values or RMS values can be converted to dbFS values.
Let's start coding:
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
# Determine full scall(maximum possible amplitude) by bit depth
bit_depth = 16
full_scale = 2 ** bit_depth
# dbFS function
to_dbFS = lambda x: 20 * np.log10(x / full_scale)
# Read in the wave file
fname = "01.wav"
fs,data = wavfile.read(fname)
# Determine frame length(number of sample points in a frame) and total frame numbers by window length(how long is a frame in seconds)
window_length = 0.01
signal_length = data.shape[0]
frame_length = int(window_length * fs)
nframes = signal_length // frame_length
# Get frames by broadcast. No overlaps are used.
idx = frame_length * np.arange(nframes)[:,None] + np.arange(frame_length)
frames = data[idx].astype("int64") # Convert to in 64 to avoid integer overflow
# Get RMS and peaks
rms = ((frames**2).sum(axis=1)/frame_length)**.5
peaks = np.abs(frames).max(axis=1)
# Convert them to dbfs
dbfs_rms = to_dbFS(rms)
dbfs_peak = to_dbFS(peaks)
# Let's start to plot
# Get time arrays of every sample point and ever frame
frame_time = np.arange(nframes) * window_length
data_time = np.linspace(0,signal_length/fs,signal_length)
# Plot
f,ax = plt.subplots()
ax.plot(data_time,data,color="k",alpha=.3)
# Plot the dbfs values on a twin x Axes since the y limits are not comparable between data values and dbfs
tax = ax.twinx()
tax.plot(frame_time,dbfs_rms,label="RMS")
tax.plot(frame_time,dbfs_peak,label="Peak")
tax.legend()
f.tight_layout()
# Save serval details
f.savefig("whole.png",dpi=300)
ax.set_xlim(1,2)
f.savefig("1-2sec.png",dpi=300)
ax.set_xlim(1.295,1.325)
f.savefig("1.2-1.3sec.png",dpi=300)
The whole time span looks like(the unit of the right axis is dbFS):
And the voiced part looks like:
You can see that the dbFS values become greater while the amplitudes become greater at the vowel start point:
I followed a tutorial on how to get sound data with pyaudio. It shows the amplitude of sound on certain frequencies.
import pyaudio
import struct
import numpy as np
from scipy.fftpack import fft
CHUNK = 2**10
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
p = pyaudio.PyAudio()
stream=p.open(format=FORMAT,channels=CHANNELS,rate=RATE,input=True,
output=True, frames_per_buffer=CHUNK)
data = struct.unpack(str(CHUNK*CHANNELS) + 'h', stream.read(CHUNK))
fft_data = fft(data)
fft_data = np.abs(fft_data[:CHUNK]) * 2 / (256 * CHUNK)
print(fft_data)
Now I want to group these together so that I can make a simple bar graph showing the amplitude of certain ranges of frequencies. When I run this code it returns the amplitude of 1024 frequencies, so I tried using this function to group them into 32 groups.
def split_freq(freq): # splits given sound frequencies into groups of frequencies to feed into turtle
freq_ranges = []
for i in range(len(freq)-1): # split the frequencies into 32 groups
if i % abs((len(freq)//32)) == 0: # create new array every time i is a multiple of the number of frequencies divided by 32
if len(freq_ranges) > 0:
freq_ranges[len(freq_ranges)-2] = freq_ranges[len(freq_ranges)-2] / (len(freq)//32)
freq_ranges.append(0)
freq_ranges[len(freq_ranges)-1] = freq_ranges[len(freq_ranges)-1] + freq[i]
return [i * 400 for i in freq_ranges]
My frequency grouping method seems to work, but when I feed in a certain tone the bar graph doesn't change in any meaningful way. Does anybody know what the problem is?
First of all, you should use rfft instead of fft since the data you are transforming only contains real values. The rfft should provide a more meaningful representation of the data.
The way you are splitting the data has no real physical meaning as far as I can see. I would recommend grouping by frequency bands; you can use a list comprehension to do this by grouping values in bins and then apply a function like sum or average:
n = fourier_data.size // 32 # 32 frequency bands
bands = [sum(fft_data[i:(i + n)]) for i in range(0, fft_data.size, n)]
or:
from statistics import mean
n = fourier_data.size // 32 # 32 frequency bands
bands = [mean(fft_data[i:(i + n)]) for i in range(0, fft_data.size, n)]
I'm a musician and I'm making a script that takes a wave file and snaps each of its frequencies from the fourier transforms to the nearest musical harmonic. Thanks to help from another question I posted here, that part works, but what I need to do now is make it so throughout the sound wave it tunes one frequency at a time so that at the beginning of the output it sounds the same as the input, and by the end it sounds like an instrument.
If I just wanted them to fade into each other then that would be easy, I could just crossfade in audacity or take a weighted average of the original fourier transform and the output one, but what I want to do instead is tune one frequency at a time. This means that at 50% of the way through the output, 50% of the frequencies have been snapped to the nearest harmonic and the other 50% are untouched. How could I accomplish this without computing every single sample of the output individually?
Also, I was thinking of reducing the MAX_HARMONIC over time as well but it would have a similar problem.
This is the sample I'm testing with (rename it to missile.wav):
https://my.mixtape.moe/iltlos.wav
Here is the script so far:
import struct
import wave
import numpy as np
# import data from wave
wav_file = wave.open("missile.wav", 'r')
num_samples = wav_file.getnframes()
sampling_rate = wav_file.getframerate() / 2
data = wav_file.readframes(num_samples)
wav_file.close()
data = struct.unpack('{n}h'.format(n=num_samples), data)
data = np.array(data)
# fast fourier transform makes an array of the frequencies of sine waves that comprise the sound
data_fft = np.fft.rfft(data)
# the higher MAX_HARMONIC is, the more it sounds like the original,
# the lower it is, the more it sounds like an instrument
MAX_HARMONIC = 2
# generate list of ratios that can be used for tuning (not octave reduced)
valid_ratios = []
for i in range(1, MAX_HARMONIC + 1):
for j in range(1, MAX_HARMONIC + 1):
if i % 2 != 0 and j % 2 != 0:
valid_ratios.append(i/float(j))
valid_ratios.append(j/float(i))
# remove dupes
valid_ratios = list(set(valid_ratios))
# find all the frequencies with the valid ratios
valid_frequencies = []
multiple = 2
while(multiple < num_samples / 2):
multiple *= 2
for ratio in valid_ratios:
frequency = ratio * multiple
if frequency < num_samples / 2:
valid_frequencies.append(frequency)
# remove dupes and sort and turn into a numpy array
valid_frequencies = np.sort(np.array(list(set(valid_frequencies))))
# bin the data_fft into the nearest valid frequency
valid_frequencies = valid_frequencies.astype(np.int64)
boundaries = np.concatenate([[0], np.round(np.sqrt(0.25 + valid_frequencies[:-1] * valid_frequencies[1:])).astype(np.int64)])
select = np.abs(data_fft) > 1
filtered_data_fft = np.zeros_like(data_fft)
filtered_data_fft[valid_frequencies] = np.add.reduceat(np.where(select, data_fft, 0), boundaries)
# do the inverse fourier transform to get a sound wave back
recovered_signal = np.fft.irfft(filtered_data_fft)
# write sound wave to wave file
comptype="NONE"
compname="not compressed"
nchannels=1
sampwidth=2
wav_file=wave.open("missile_output.wav", 'w')
wav_file.setparams((nchannels, sampwidth, int(sampling_rate), num_samples, comptype, compname))
for s in recovered_signal:
wav_file.writeframes(struct.pack('h', s))
wav_file.close()
I am using the random number routines in python in the following code in order to create a noise signal.
res = 10
# Add noise to each X bin accross the signal
X = np.arange(-600,600,res)
for i in range(10000):
noise = [random.uniform(-2,2) for i in xrange(len(X))]
# custom module to save output of X and noise to .fits file
wp.save_fits('test10000', X, noise)
plt.plot(V, I)
plt.show()
In this example I am generate 10,000 'noise.fits' files, that I then wish to co-add together in order to show the expected 1/sqrt(N) dependence of the stacked noise root-mean-square (rms) as a function of the number of objects co-added.
My problem is that the rms follows this dependancy up until ~1000 objects, at which point it deviates upwards, suggesting that the random number generator.
Is there a routine or way to structure the code which will avoid or minimise this repetition? (Ideally with the number as a float in between a max and min value >1 and <-1)?
Here is the output of the co-adding code as well as the code pasted at the bottom for reference.
If I use the module random.random() the result is worse.
Here is my code which adds the noise signal files together, averaging over the number of objects.
import os
import numpy as np
from astropy.io import fits
import matplotlib.pyplot as plt
import glob
rms_arr =[]
#vel_w_arr = []
filelist = glob.glob('/Users/thbrown/Documents/HI_stacking/mockcat/testing/test10000/M*.fits')
filelist.sort()
for i in (filelist[:]):
print(i)
#open an existing FITS file
hdulist = fits.open(str(i))
# assuming the first extension is the table we assign data to record array
tbdata = hdulist[1].data
#index = np.arange(len(filelist))
# Access the signal column
noise = tbdata.field(1)
# access the vel column
X = tbdata.field(0)
if i == filelist[0]:
stack = np.zeros(len(noise))
tot_rms = 0
#print len(stack)
# sum signal in loop
stack = (stack + noise)
rms = np.std(stack)
rms_arr = np.append(rms_arr, rms)
numgal = np.arange(1, np.size(filelist)+1)
avg_rms = rms_arr / numgal
I use Tektronix oscilloscope to perform some signal acquisition. I get 10.000 measurement points (few signal periods) and I have to do a frequency analysis on that set of data. My signal is 8MHz sine wave. When I use either SciPy or NumPy I get the same result - frequencies are spreaded too wide. The distance between two values is 500kHz and the highest frequency is 2.5GHz (absurd). When I want to measure frequency bandwidth around 8MHz I can only get exact values of 7.5, 8.0 and 8.5 MHz. I tried to change sample spacing determined by (x[1]-x[0]) and I got nothing better.
def CalculateFFT(t_val,p_val):
x = t_val #Two parameters: [x,y] values
y = lambda x: p_val
com_signal = y(x) # Combined signal
FFT_val = abs(scipy.fft(com_signal))
freq_val = scipy.fftpack.fftfreq(len(com_signal), x[1]-x[0])
spec_val = 20*scipy.log10(FFT_val)
return freq_val, spec_val
It is worth reading in more depth how DFFTs work but you should always have the following formulae in mind. For a time series with n points and maximum time Tmax, the time resolution is given by dt = Tmax / n
A DFFT will produce n points with
Fmax = 1 / dt
dF = 1 / Tmax
You seem to suggest the maximum frequency is sufficient (so the time resolution is okay) but the frequency resolution isn't good enough: you need to collect more data, at the same time resolution.
If (1) the sampling time is too short, (2) you require higher estimation frequency accuracy, and, (3) you know that your signal is a sine wave, then you can fit the signal to a sine wave. Like in How do I fit a sine curve to my data with pylab and numpy?,
with the exception that the frequency needs to be added.
Here is an example figure with a frequency of around 8 MHz:
Below is the example code:
""" Modified from https://stackoverflow.com/a/16716964/6036470 """
from numpy import sin, linspace, pi,average;
from pylab import plot, show, title, xlabel, ylabel, subplot, scatter
from scipy import fft, arange, ifft
import scipy
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import leastsq
ff = 8e6; # frequency of the signal
Fs = ff*128; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = arange(0,((1/ff)/128)*(128)*5,Ts) # time vector
A = 2.5;
ff_0 = 8.1456e6
y = A*np.sin(2*np.pi*ff_0*t+15.38654*pi/180) + np.random.randn(len(t))/5
guess_b = 0
guess_a = y.std()*2**0.5;
guess_c = 10*pi/180
guess_d = ff*0.98*2*pi
fig = plt.figure(facecolor="white")
plt.plot(t,y,'.', label='Signal Fred. %0.4f Hz'%(ff_0/1e6))
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.grid(alpha=0.5);
optimize_func = lambda x: (x[0]*np.sin(x[2]*t+x[1]) - y);
est_a, est_c, est_d = leastsq(optimize_func, [guess_a, guess_c, guess_d])[0]
data_fit = est_a*np.sin(est_d*t+est_c) ;
plt.plot(t,data_fit,label='Fitted Est. Freq. %0.4f Hz'%(est_d/(2*pi)/1e6))
plt.legend()
plt.tight_layout();
plt.show();
fig.save("sinfit.png")