We Keep Coding

Convert str to float python

Why does this throw an error : ValueError: could not convert string to float:
frequencies.append(float(l[start+1:stop1].strip()))
losses.append(float(l[stop1+5:stop2].strip()))
Doesn't the float() command parse values into the float type? Where am I wrong here? Both frequencies and losses are lists
This is the code:
def Capture():
impedance = 0
losses = []
frequencies = []
Xtalk = []
start = 0
stop1 = 0
stop2 =0
for filename in glob.glob(os.path.join(user_input, '*.txt')):
with open(filename, 'r') as f:
for l in f.readlines():
if l.startswith(' Impedance'):
v = l[12:-7]
impedance = float(v)
if l.startswith(' Xtalk'):
Xtalk.append(l[7:].strip())
if l.startswith(' Loss per inch'):
start = l.find('#')
stop1 = l.find('GHz', start)
stop2 = l.find('dB', start)
frequencies.append(float(l[start+1:stop1].strip()))
losses.append(float(l[stop1+5:stop2].strip()))
print(impedance, frequencies, losses, Xtalk)
It basically takes values from a text file and prints them onto the console
And the text files look like this:
Impedance = 71.28 ohms
Begin Post processing
Frequency multiplier = 1Hz
number of ports = 12
Start Frequency = 0
End Frequency = 40000000000
Number of Frequency points = 4001
Touchstone Output file = C:\Users\Aravind_Sampathkumar\Desktop\IMLC\BO\Output_TW_3.5-TS_3-core_h_2.xml_5000mil.s12p
Output format = Real - Imaginary
Loss per inch # 2.500000e+00 GHz = -0.569 dB
Loss per inch # 5 GHz = -0.997 dB
Xtalk #1 (Conductor 1 2):
Step response Next= -0.56 mV
Step response Fext peak # 5 inches= 0.11 mV
Xtalk #2 (Conductor 5 6):
Step response Next= -0.56 mV
Step response Fext peak # 5 inches= 0.11 mV
Finished post processing

First make sure which is the format of variable.
string with comma can't be converted to float with "float()" parser
a = "1,2345"
float(a)
Traceback (most recent call last):
File "<input>", line 1, in <module>
ValueError: could not convert string to float: '1,2345'
a = "1.2345"
float(a)
1.2345

TypeError: a bytes-like object is required, not 'str', laspy

I am new to programming and wanted to convert Las file into grid file using laspy. It keeps giving error
"TypeError: a bytes-like object is required, not 'str'".
I know fmt gives a string, so I tried fmt = '%1.2f'.encode() to change in to binary, but got the same error.
from laspy.file import File
import numpy as np
source = "/655-7878.las"
target = "/lidar.asc"
cell = 1.0
NODATA = 0
las = File(source, mode = "r")
#xyz min and max
min = las.header.min
max = las.header.max
#Get the x axis distance
xdist = max[0] - min[0]
#Get the y axis distance
ydist = max[1] - min[1]
#Number of columns for our grid
cols = int((xdist)/cell)
#Number of rows for our grid
rows = int((ydist)/cell)
cols += 1
rows += 1
#Track how many elevation
#values we aggregate
count = np.zeros((rows, cols)).astype(np.float32)
#Aggregate elevation values
zsum = np.zeros((rows, cols)).astype(np.float32)
#Y resolution is negative
ycell = -1 * cell
#Project x,y values to grid
projx =(las.x -min[0]) / cell
projy = (las.y - min[1])/ ycell
#Cas to integers and clip for use as index
ix = projx.astype(np.int32)
iy = projy.astype(np.int32)
#Loop through x,y,z arrays, add to grid shape and aggregate values for averaging
for x,y,z in np.nditer([ix, iy, las.z]):
count[y, x] +=1
zsum[y, x]+=z
# Change 0 values to 1 to avoid numpy warnings and NaN values in array
nonzero = np.where(count>0, count, 1)
#Average our z values
zavg = zsum/nonzero
#Interpolate 0 values in array to avoid any holes in the grid
mean = np.ones((rows, cols)) * np.mean(zavg)
left = np.roll(zavg, -1,1)
lavg = np.where(left>0, left, mean)
right = np.roll(zavg, 1, 1)
ravg = np.where(right>0, right, mean)
interpolate = (lavg + ravg)/2
fill = np.where(zavg>0, zavg, interpolate)
#Create ASCII DEM header
header = "ncols %s\n" % fill.shape[1]
header += "nrows %s\n" % fill.shape[0]
header += "xllcorner %s\n" % min[0]
header += "yllcorner %s\n" % min[1]
header += "cellsize %s\n" % cell
header += "NODATA_value %s\n" % NODATA
#Open the output file, add the header, save the array
with open(target, "wb") as f:
f.write(header)
# The fmt string ensures we output floats
#That have at least one number but only two decimal places
np.savetxt(f, fill, fmt = '%1.2f')`
Can someone please help me to sort it out.

f.write(bytes(header, 'UTF-8'))

if you are using python3 when you open a file with 'b' you can't write strings to the file, only raw binary data . if you have a string you want to write to the file you should either open it in text mode (without 'b') or convert it to a bytearray()
so writing to file would look like this:
with open(target, "wb") as f:
f.write(bytearray(header,'utf-8'))

Extracting frequencies from a wav file python

I am familiar with python but new to numpy, so please pardon me if I am wrong.
I am trying to read a .wav file having multiple frequencies (separated by silence). So far I've been able to read the values and find the various parts in the file where there is a sound. Then, I am trying to find the Discrete Cosine Transform and calculate the frequencies from it (ref: how to extract frequency associated with fft values in python)
However, I'm getting an error:
index 46392 is out of bounds for axis 0 with size 25
Here's my code:
import wave
import struct
import numpy as np
def isSilence(windowPosition):
sumVal = sum( [ x*x for x in sound[windowPosition:windowPosition+windowSize+1] ] )
avg = sumVal/(windowSize)
if avg <= 0.0001:
return True
else:
return False
#read from wav file
sound_file = wave.open('test.wav', 'r')
file_length = sound_file.getnframes()
data = sound_file.readframes(file_length)
sound_file.close()
#data = struct.unpack("<h", data)
data = struct.unpack('{n}h'.format(n=file_length), data)
sound = np.array(data)
#sound is now a list of values
#detect silence and notes
i=0
windowSize = 2205
windowPosition = 0
listOfLists = []
listOfLists.append([])
maxVal = len(sound) - windowSize
while True:
if windowPosition >= maxVal:
break
if not isSilence(windowPosition):
while not isSilence(windowPosition):
listOfLists[i].append(sound[windowPosition:windowPosition+ windowSize+1])
windowPosition += windowSize
listOfLists.append([]) #empty list
i += 1
windowPosition += windowSize
frequencies = []
#Calculating the frequency of each detected note by using DFT
for signal in listOfLists:
if not signal:
break
w = np.fft.fft(signal)
freqs = np.fft.fftfreq(len(w))
l = len(signal)
#imax = index of first peak in w
imax = np.argmax(np.abs(w))
fs = freqs[imax]
freq = imax*fs/l
frequencies.append(freq)
print frequencies
Edit: Here is the traceback:
Traceback (most recent call last):
File "final.py", line 61, in <module>
fs = freqs[imax]
IndexError: index 46392 is out of bounds for axis 0 with size 21

The problem was that I assumed listOfLists was actually a list of lists, but actually it was a list of list of lists. The line:
listOfLists[i].append(sound[windowPosition:windowPosition+ windowSize+1])
was appending a list everytime, but I assumed it was appending the elements to existing list.
For instance, if listOfLists was:
[ [1,2,3] ]
Then, listOfLists[0].append([4,5,6]) would give:
[ [ [1,2,3],[4,5,6] ] ]
But I was expecting:
[ [1,2,3,4,5,6] ]
Replacing the problematic line with the code below worked for me:
for v in sound[windowPosition:windowPosition+windowSize+1]:
listOfLists[i].append(v)

"Invalid destination position for blit" in pygame when loading configuration from file

I am trying unsuccessfully to load a room from a file during runtime. What confuses me the most is that this line of code:
ObjGlobal.instances.append(oPlayer.oPlayer(x, y))
successfully creates an object when executed in the main function, but when put not when in the file loading function:
File "E:\Fun Stuff\Python Stuff\Python
projects\SimpleEngine\Main.py", line 56, in main
ObjGlobal.drawObjects(displaySurface) File "E:\Fun Stuff\Python Stuff\Python projects\SimpleEngine\Global.py", line 57, in drawObjects
surface.blit(self.instances[i].sprite, (self.instances[i].x, self.instances[i].y)) TypeError: invalid destination position for blit
That error occurs later on when I try and call one of the objects' variables or functions, of course. Here is the function loading the room:
def loadRoom(ObjGlobal, fname):
# Get the list of stuff
openFile = open(fname, "r")
data = openFile.read().split(",")
openFile.close()
# Loop through the list to assign said stuff
for i in range(len(data) / 3):
# Create the object at the position
x = data[i * 3 + 1]
y = data[i * 3 + 2]
# Temporary object string
tempObject = data[i * 3]
# Create object
if (tempObject == "oPlayer"):
ObjGlobal.instances.append(oPlayer.oPlayer(x, y))
elif (tempObject == "Wall"):
ObjGlobal.instances.append(CommonObjects.Wall(x, y))
else: # Error found
print ("Error: No object with name '%s'" % (tempObject))
My file is in the correct format. Note when I call it in main I replace x and y with 32, 32.

When reading in data from a file, it is in string format by default. You should convert it into integer format before constructing an object with it:
x = int(data[i * 3 + 1])
y = int(data[i * 3 + 2])

how to convert bytes to np.array

I'm read a buffer of bytes from data recorded through my computer's microphone (2 channels) using pyaudio example taken from site.
import pyaudio
import wave
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 2
RATE = 44100
RECORD_SECONDS = 5
WAVE_OUTPUT_FILENAME = "output.wav"
p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
channels=CHANNELS,
rate=RATE,
input=True,
frames_per_buffer=CHUNK)
print("* recording")
frames = []
for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):
data = stream.read(CHUNK)
frames.append(data)
print("* done recording")
print frames
frames looks like this:
['\x00\xfd\xff\xff.....\xfc\xff\xff', '\xff\xfc\xff\xff......\xfc\xff\xff', ... ]
or if I change CHUNK = 1:
['\x00\xfd\xff\xff', '\xff\xfc\xff\xff', '\x00\xfd\xcc\xcf']
though of course much longer. I suspect that the bytes are interleaved for each channel so I think I need to break them out in pairs of two.
What I'd like is an array like this:
np.array([
[123, 43],
[3, 433],
[43, 66]
])
where the first column is the values from the first channel, and the second from the second channel. how do I go about interpreting these encoded values (with CHUNK set to a reasonable value like 1024)?
UPDATE:
I'm quite confused. I used the below to change the format list of strings into a single string of space-separated hex values, but there appears to be an odd number of them...which wouldn't happen if there are two values, one for each channel (would be even number):
fms = ''.join(frames)
fms_string = ''.join( [ "%02X " % ord( x ) for x in fms ] ).strip()
fms_list = fms_string.split(" ")
print len(fms_list) # this prints an ODD number...
UPDATE 2:
I tried a simpler route and tried this:
import array
fstring = ''.join(frames)
wave_nums = array.array('h', fstring) # this correctly returns list of ints!
print len(wave_nums)
I tried this for different recording times and got the following (confusing results):
RECORD_SECONDS = 2 ---> len(wave_nums) is 132300 (132300 / 44100 = 3 seconds of frames)
RECORD_SECONDS = 4 ---> len(wave_nums) is 308700 (308700 / 44100 = 7 seconds of frames)
RECORD_SECONDS = 5 ---> len(wave_nums) is 396900 (396900 / 44100 = 9 seconds of frames)
which implies that I'm getting a number of frames consistent with 2*(number of seconds recording) - 1 seconds...how is this possible?

Based on a quick glance of the portaudio source it looks like the channels are in fact interleaved
You can use a join to flatten the list, calculate the left and right values (you set them to be 16 bits long), and then zip the list with itself.
joined = ''.join(frames).encode('latin-1')
left = map(lambda m, l: (m << 8) + l, joined[0::4], joined[1::4])
right = map(lambda m, l: (m << 8) + l, joined[2::4], joined[3::4])
zipped = zip(left, right)
On python 2.x, the encode latin1 trick doesn't work, so you'll need to do
joined = ''.join(frames)
joined = map(ord, joined)
left = map(lambda m, l: (m << 8) + l, joined[0::4], joined[1::4])
right = map(lambda m, l: (m << 8) + l, joined[2::4], joined[3::4])
zipped = zip(left, right)
This has something to do with python 2.x's preference for ascii strings vs unicode.
Update:
w.r.t the odd number of bytes, read might have tried to read too many bytes ahead and failed silently, only returning whatever it had at the moment. You should always receive a multiple of CHUNK bytes from read under normal conditions, so unless your join function has an error, something is wrong on their end. Try it with mine and see what happens.

The simplest answer appears to be this:
import array
f = ''.join(frames)
nums = array.array('h', f)
left = nums[1::2]
right = nums[0::2]
#Dylan's answer is also good but a bit more verbose and also the values are unsigned, where wav values are signed.
Also changing CHUNK to a value of 1225 is best since 44100 is a multiple of 1225, and no frames are lost as a result of rounding error.