I have a function to read output stored on Arduino. The output looks like this:
['stopped','15.00','25.44','40.5','nan','off','off','on','50.00','yes','on','45']
['version1.0']
['3.0','0.1']
The first line includes temperatures at different places, and I need the fourth one. The second line is meaningless, but I need its command to keep the device running. The third line is position and force.
I need to keep sending command to the device in oder to maintain the heaters on, or the device will automatically shut down if no command within 10s. Therefore, I add a command
ser.write(b"y\n")
The function:
def read_output(ser, verbose=True):
position = []
force = []
temp = []
while ser.inWaiting() > 0:
line = ser.readline().strip()
line = ''.join(line.decode()[1:-1])
line = line.split(' ')
ser.write(b"y\n") #meaningless command print "version1.0"
print(line)
if verbose:
print(line)
if len(line) == 2:
position += [float(line[0])]
force += [float(line[1])]
if len(line) == 12:
temp += [float(line[3])]
if int(float(line[3])) > 50
break
time.sleep(0.001) # no busy-wait, reduce CPU
return position, force, temp
I will collect data by p, f, temp = read_output(ser) and save them in csv at the end.
The code sometimes will be interrupted by misplaced output and with an error like this
['version1.0<3.0','0.1']
ValueError: could not convert string to float: 'version1.0<3.0'
It looks like the second and the third line overlap. I guess this could be due to that ['version1.0'] and position&force outputs ['3.0','1.0'] were printout at the same time.
If this happens, I cannot get the output. Can anyone help me skip this error and continue running the code?
Related
I have a code that generates characters from 000000000000 to ffffffffffff which are written to a file.
I'm trying to implement a check to see if the program was closed so that I can read from the file, let's say at 00000000781B, and continue for-loop from the file.
The Variable "attempt" in (for attempt in to_attempt:) has tuple type and always starting from zero.
Is it possible to continue the for-loop from the specified value?
import itertools
f = open("G:/empty/last.txt", "r")
lines = f.readlines()
rand_string = str(lines[0])
f.close()
letters = '0123456789ABCDEF'
print(rand_string)
for length in range(1, 20):
to_attempt = itertools.product(letters, repeat=length)
for attempt in to_attempt:
gen_string = rand_string[length:] + ''.join(attempt)
print(gen_string)
You have to store the value on a file to keep track of what value was last being read from. I'm assuming the main for loop running from 000000000000 to ffffffffffff is the to_attempt one. All you need store the value of the for loop in a file. You can use a new variable to keep track of it.
try:
with open('save.txt','r') as reader:
save = int(reader.read())
except FileNotFoundError:
save = 0
#rest of the code
for i in range(save,len(to_attempt)):
with open('save.txt','r') as writer:
writer.write(i)
#rest of the code
I've finished coding my program and now I'm testing it. I've been testing my code by inputting individual input strings and it has been able to solve with the correct output within a matter of seconds. However, when taking that same input from a file of a number of input strings, my program slows down and I'm not sure if this is due to the way I'm calling the input file or if my program is slow.
However, when taking that same input from a file of a number of input strings, my program slows down and I'm not sure if this is due to the way I'm calling the input file or if my program is slow. To clarify:
In the first one I manually input a string input while b.txt contains a file of string inputs. it was taking considerably long to solve a certain string in the file but when I tested this string individually it was solved in a matter of seconds. What could I be misunderstanding?
for line in sys.argv[1].split("\n"):
if len(line) < 9:
continue
start = time.time()
board = [SOME CODE HERE]
sboard = backtrack(board)
end = time.time()
total = end - start
print(total)
compared to below which takes lines of input from this code segment where
s = open("b.txt", "r")
sl = s.read()
t=0
for line in sl.split("\n"):
t+=1
if len(line) < 9:
continue
board = [SOME CODE HERE]
start = time.time()
sboard = backtrack(board)
end = time.time()
total = end - start
print(str(t)+": "+str(total))
All,
I am trying to write a python script that will go through a crime file and separate the file based on the following items: UPDATES, INCIDENTS, and ARRESTS. The reports that I generally receive either show these sections as I have previously listed or by **UPDATES**, **INCIDENTS**, or **ARRESTS**. I have already started to write the following script to separate the files based on the following format with the **. However, I was wondering if there was a better way to check the files for both formats at the same time? Also, sometimes there is not an UPDATES or ARRESTS section which causes my code to break. I was wondering if there is a check I can do for this instance, and if this is the case, how can I still get the INCIDENTS section without the other two?
with open('CrimeReport20150518.txt', 'r') as f:
content = f.read()
print content.index('**UPDATES**')
print content.index('**INCIDENTS**')
print content.index('**ARRESTS**')
updatesLine = content.index('**UPDATES**')
incidentsLine = content.index('**INCIDENTS**')
arrestsLine = content.index('**ARRESTS**')
#print content[updatesLine:incidentsLine]
updates = content[updatesLine:incidentsLine]
#print updates
incidents = content[incidentsLine:arrestsLine]
#print incidents
arrests = content[arrestsLine:]
print arrests
You are currently using .index() to locate the headings in the text. The documentation states:
Like find(), but raise ValueError when the substring is not found.
That means that you need to catch the exception in order to handle it. For example:
try:
updatesLine = content.index('**UPDATES**')
print "Found updates heading at", updatesLine
except ValueError:
print "Note: no updates"
updatesLine = -1
From here you can determine the correct indexes for slicing the string based on which sections are present.
Alternatively, you could use the .find() method referenced in the documentation for .index().
Return -1 if sub is not found.
Using find you can just test the value it returned.
updatesLine = content.find('**UPDATES**')
# the following is straightforward, but unwieldy
if updatesLine != -1:
if incidentsLine != -1:
updates = content[updatesLine:incidentsLine]
elif arrestsLine != -1:
updates = content[updatesLine:arrestsLine]
else:
updates = content[updatesLine:]
Either way, you'll have to deal with all combinations of which sections are and are not present to determine the correct slice boundaries.
I would prefer to approach this using a state machine. Read the file line by line and add the line to the appropriate list. When a header is found then update the state. Here is an untested demonstration of the principle:
data = {
'updates': [],
'incidents': [],
'arrests': [],
}
state = None
with open('CrimeReport20150518.txt', 'r') as f:
for line in f:
if line == '**UPDATES**':
state = 'updates'
elif line == '**INCIDENTS**':
state = 'incidents'
elif line == '**ARRESTS**':
state = 'arrests'
else:
if state is None:
print "Warn: no header seen; skipping line"
else
data[state].append(line)
print data['arrests'].join('')
Try using content.find() instead of content.index(). Instead of breaking when the string isn't there, it returns -1. Then you can do something like this:
updatesLine = content.find('**UPDATES**')
incidentsLine = content.find('**INCIDENTS**')
arrestsLine = content.find('**ARRESTS**')
if incidentsLine != -1 and arrestsLine != -1:
# Do what you normally do
updatesLine = content.index('**UPDATES**')
incidentsLine = content.index('**INCIDENTS**')
arrestsLine = content.index('**ARRESTS**')
updates = content[updatesLine:incidentsLine]
incidents = content[incidentsLine:arrestsLine]
arrests = content[arrestsLine:]
elif incidentsLine != -1:
# Do whatever you need to do to files that don't have an arrests section here
elif arreststsLine != -1:
# Handle files that don't have an incidents section here
else:
# Handle files that are missing both
Probably you'll need to handle all four possible combinations slightly differently.
Your solution generally looks OK to me as long as the sections always come in the same order and the files don't get too big. You can get real feedback at stack exchange's code review https://codereview.stackexchange.com/
I'm having trouble in writing a script for reading some temperature sensors (DS18B20) on a raspberry pi.
I have a working script, but sometimes the sensors fall out and then the script also stops.
I'm trying to make a more robust version by integrating a try-except statement. The goal is to proceed to the next sensor in the range if one of the sensors doesn't react. If I emulate sensor failure by plugging one of the sensors out, the script stops taking measurements for all the sensors (instead of for the sensor that has been plugged out). And it doesn't give me an error. Any ideas?
This is the part of the script with the try statement:
if time.time() <= timeout:
for index in range (numsensors):
try:
def read_temp_raw(): # gets the temps one by one
f = open(device_file[index], 'r')
lines = f.readlines()
f.close()
return lines
def read_temp(): # checks the received temperature for errors
lines = read_temp_raw()
while lines[0].strip()[-3:] != 'YES':
time.sleep(0.2)
lines = read_temp_raw()
equals_pos = lines[1].find('t=')
if equals_pos != -1:
temp_string = lines[1][equals_pos+2:]
# set proper decimal place for deg C
temp = float(temp_string) / 1000.0
# Round temp to x decimal points --> round(temp,x)
temp = round(temp, 2)
return temp
reading = (read_temp())
temp[index].append(reading)
print device[index],"=", temp[index]
continue
except IOError:
print "Error"
"What has been asked" inventory:
Is using try-except construct making underlying system more
robust?
Why is the code not giving any error on indicated sensor-failure?
A1:
The try-except clause sounds as self-explanatory & life-saving package, however it is not.
One has to fully understand what all sorts of exceptions-types the code is expecting to meet face-to-face and how to handle each of them. Naive or erroneous use of this syntax construct effectively masks the rest of the exceptions from your debugging radar screen, leaving the unhandled cases fail in a dark silence, out of your control and without knowing about them at all. True "Robustness" and "Failure Resilience" is something else than this.
This code sample will leave hidden all real-life collisions, except the only one listed, the IOError, but if that will not happen, all the others, that do happen, are not handled:
if time.time() <= timeout: # START if .time() is still before a T/O
for index in range (numsensors): # ITERATE over all sensors
try: # TRY:
<<<something>>> # <<<something>>>
except IOError: # EXC IOError:
<<<IOError__>>> # Handle EXC.IOError
Guess all your def...(..):-s may belong to a non-repeating section of the code, prior to the if:/for: as you need not modify the code "on-the-fly", do you?
def read_temp_raw(): # DEF: gets the temps one by one
f = open(device_file[index],'r') # SET aFileHANDLE access to IO.DEV ( beware var index "visibility" )
lines = f.readlines() # IO.DEV.READ till <EoF>
f.close() # IO.DEV.CLOSE
return lines # RET all lines
def read_temp(): # DEF: checks the received temperature for errors
lines = read_temp_raw() # GET lines from read_temp_raw()
while lines[0].strip()[-3:]!='YES': # WHILE last-3Bytes of 1st-line!="YES"
time.sleep(0.2) # NOP/Sleep()
lines = read_temp_raw() # GET lines again (beware var index)
equals_pos =lines[1].find('t=') # SET position of 't=' in 2nd-line
if equals_pos != -1: # IF position != -1
temp_string = lines[1][equals_pos+2:]
temp = float(temp_string) \
/ 1000.0 # DIV( 1000) decimal place for deg C
temp = round(temp, 2) # ROUND temp to x decimal points --> round(temp,x)
return temp # RET->
# ----------------------------- # ELSE: re-loop in WHILE
# -------------------------------- # LOOP AGAIN AD INFIMUM
A2: your try-except clause in the posted code is expecting only one kind of exception to be handled by it -- the IOError -- which is instantiated only when actual IO.DEV operation fails for an I/O-related reason, which does not mean a case, that you physically "un-plug" a sensor, while the IO.DEV is still present and can carry it's IO.DEV.READ(s) and thus no exceptions.EnvironmentError.IOError is to be raise-d
That means, the IO.DEV.READ(s) take place and the code results, as per the condition WHILE last-3Bytes of 1st-line dictates, in an endless loop, because the 1st-line "still" does not end with "YES".
Q.E.D.
The Goal focus
Coming back to the issue, you may rather set a safer test for a real-world case, where an erroneous input may appear during your sensor-network scan.
The principle may look like:
f.close() # IO.DEV.CLOSE
if ( len(lines) < 2 ): # IF <<lines>> are nonsense:
return [ "NULL_LENGTH_READING still with CODE EXPECTED ACK-SIG-> YES", \
"SIG ERROR FOR POST-PROCESSOR WITH AN UNREALISTIC VALUE t=-99999999" \
]
return( lines ) # OTHERWISE RET( lines )
I'm using the NumPy python library to run large-scale edits on a .csv file. I'm using this python code:
import numpy as np
def main():
try:
e,a,ad,c,s,z,ca,fn,ln,p,p2,g,ssn,cn,com,dob,doh,em = np.loadtxt('c:\wamp\www\_quac\carryover_data\SI\Employees.csv',delimiter=',',unpack=True,dtype='str')
x=0
dob = dob.split('/')
for digit in dob:
if len(digit) == 1:
digit = str('0'+digit)
dob = str(dob[2]+'-'+dob[0]+'-'+dob[1])
doh = doh.split('/')
for digit in doh:
if len(digit) == 1:
digit = str('0'+digit)
doh = str(doh[2]+'-'+doh[0]+'-'+doh[1])
for eID in e:
saveLine=eID+','+a[x]+','+ad[x]+','+c[x]+','+s[x]+','+z[x]+','+ca[x]+','+fn[x]+','+ln[x]+','+p[x]+','+p2[x]+','+g[x]+','+ssn[x]+','+cn[x]+','+com[x]+','+dob[x]+','+doh[x]+','+em[x]+'\n'
saveFile = open('fixedEmployees.csv','a')
saveFile.write(saveLine)
saveFile.close()
x+=1
except Exception, e:
print str(e)
main()
dob and doh contain a string, e.g. 4/26/2012 and I'm trying to convert these to mysql friendly DATE forms, e.g. 2012-04-26. The error that is printed when I run this script is
cannot set an array element with a sequence
It does not specify a line and so I don't know what this really means. I'm pretty new to python; I've checked other questions with this same error but I can't make sense of their code. Any help is very appreciated.
Try using zfill to reformat the date string so you can have a '0' before your '4'. (zfill pads a string on the left with zeros to fill the width.)
doh = '4/26/2012'
doh = doh.split('/')
for i, s in enumerate(doh):
doh[i] = s.zfill(2)
doh = doh[2]+'-'+doh[0]+'-'+doh[1]
# result: '2012-04-26'
As for the cannot set an array element with a sequence it would be helpful to know
where that is occurring. I'm guessing there is something wrong with structure of the array.
Ok, to solve it I had to do a couple things. After removing the try-except commands, I found out that the error was on line 5, the line with e,a,ad,c,s etc. I couldn't eliminate the problem until I simply copied the 2 columns I wanted to focus on only and made a new program for dealing with those.
Then I had to create a .txt instead of a .csv because Excel auto-formats the dates and literally changes the values before I can even touch them. There is no way around that, I've learned. You can't turn the date-auto-format off. A serious problem with excel. So here's my solution for this NumPy script (it changes the first column and keeps the second the same):
import numpy as np
def main():
dob,doh=np.loadtxt('temp.csv',
delimiter=',',
unpack=True,
dtype='str')
x=0
for eachDate in dob:
if any(c.isalpha() for c in eachDate):
newDate=eachDate
elif (eachDate == ''):
newDate=''
else:
sp = eachDate.split('/')
y=0
ndArray = ['','','']
for eachDig in sp:
if len(eachDig) == 1:
eachDig = str('0'+eachDig)
if y == 0:
ndArray[0] = eachDig
elif y == 1:
ndArray[1] = eachDig
elif y == 2:
ndArray[2] = eachDig
newDate=str(ndArray[2]+'-'+ndArray[0]+'-'+ndArray[1])
y=0
y+=1
print eachDate+'--->'+newDate
"""creates a .txt file with the edited dates"""
saveLine=str(newDate+','+doh[x]+'\n')
saveFile=open('__newTemp.txt','a')
saveFile.write(saveLine)
saveFile.close()
x+=1
main()
I then used Data->Import from text with "TEXT" format option in Excel to get the column into my .csv. I realize this is probably bulky and noobish but it got the job done :3