I use the following piece of code to read the serial port until i get a terminating character.
"""Read until you see a terminating character with a timeout"""
response=[]
byte_read=''
break_yes=0
time_now = time.clock()
while ((not (byte_read=='\r') ) and (break_yes==0)):
byte_read = self.ser.read(1)
if (not(len(byte_read)== 0) and (not (byte_read =='\r'))):
response.append(byte_read)
if ( time.clock() - time_now > 1 ):
if self.DEBUG_FLAG:
print "[animatics Motor class] time out occured. check code"
break_yes=1
if break_yes==0:
return ''.join(response)
else:
return 'FAIL'
This works well but because of the while loop, the cpu resources are taken up.
I think that having a blocking read(1) with a timeout will save some of the cpu.
The flag i am looking for C is "MIN == 0, TIME > 0 (read with timeout)" in termios
i am looking for a similar flag in Python.
I could also use the io.readline to read till i get '\r', but i want to stick to pyserial as much as possible without any other dependency.
Would greatly appreciate advice. Do let me know if i should do it in a completely different way either too.
Thanks,
You should read the documentation of Pyserial: it clearly states that a timeout of 0 as you pass it to the constructor will turn on non-blocking behaviour:
http://pyserial.sourceforge.net/pyserial_api.html#classes
Just get rid of the timeout parameter, and you should be set.
Aight, so I found out a way. Instead of polling with the no timeout, I use the select module in python, which is similar to the one in C.
It returns if any data is available immediately, or waits for the timeout period and exits, which is precisely what i wanted. I took deets comments for cleaning up the code and it looks like so now.
def readOnly(self):
"""Read until you see a terminating character with a timeout"""
response=[]
byte_read=''
while (not (byte_read=='\r')):
reading,_,_ = select.select([self.ser], [], [], 1) #returns immediately if there is data on serial port. waits 1 second to timeout if not.
if reading !=[]: #something is to be read on the file descriptor
byte_read = self.ser.read(1)
if (byte_read !='\r'):
response.append(byte_read)
else: #'\r' received
return ''.join(response)
break
else:
if self.DEBUG_FLAG:
print "[Motor class] time out occured. check code"
return 'FAIL'
break
`
This decreased the cpu usage from 50% to 5% so life is better now.
Thanks,
Related
I have a process that runs data acquisition using PySerial. It's working fine now, but there's a weird thing I had to do to make it work continuously, and I'm not sure this is normal, so I'm asking this question.
What happens: It looks like that the connection drops now and then! Around once every 30-60 minutes, with big error bars (could go for hours and be OK, but sometimes happens often).
My question: Is this standard?
My temporary solution: I wrote a simple "reopen" function that looks like this:
def ReopenDevice(devObject):
try:
devObject.close()
devObject.open()
except Exception as e:
print("Error while trying to connect to device " + devObject.port + ". The error says: " + str(e))
time.sleep(2)
And what I do is that if data pulling fails for 2 minutes, I reopen the device with this function, and it continues working well with no problems.
My program model: It's a GUI program, where the user clicks something like "Start", and that button does some preparations and runs a function through multiprocessing.Process() that starts with:
devObj = serial.Serial()
#... other params
devObj.open()
and that function then runs a while loop that keeps polling data with something like:
bytesToRead = devObj.inWaiting()
if bytesToRead != 0:
buffer = decodeString(devObj.read(bytesToRead))
#process buffer and push it to a list...
The way I know that the problem happened, is that devObj.inWaiting() Keeps returning zero... no matter how much data there's on the device!
Is this behavior expected and should always be considered whether it happens or doesn't happen?
The problem reduced a lot after not calling inWaiting() very frequently. Anyway, I kept the reconnect part to ensure that my program never fails. Thanks for "Kobi K" for suggesting the possible cause of the problem.
I'm running a python script on a raspberry pi that constantly checks on a Yocto button and when it gets pressed it puts data from a different sensor in a database.
a code snippet of what constantly runs is:
#when all set and done run the program
Active = True
while Active:
if ResponseType == "b":
while Active:
try:
if GetButtonPressed(ResponseValue):
DoAllSensors()
time.sleep(5)
else:
time.sleep(0.5)
except KeyboardInterrupt:
Active = False
except Exception, e:
print str(e)
print "exeption raised continueing after 10seconds"
time.sleep(10)
the GetButtonPressed(ResponseValue) looks like the following:
def GetButtonPressed(number):
global buttons
if ModuleCheck():
if buttons[number - 1].get_calibratedValue() < 300:
return True
else:
print "module not online"
return False
def ModuleCheck():
global moduleb
return moduleb.isOnline()
I'm not quite sure about what might be going wrong. But it takes about an hour before the RPI runs out of memory.
The memory increases in size constantly and the button is only pressed once every 15 minutes or so.
That already tells me that the problem must be in the code displayed above.
The problem is that the yocto_api.YAPI object will continue to accumulate _Event objects in its _DataEvents dict (a class-wide attribute) until you call YAPI.YHandleEvents. If you're not using the API's callbacks, it's easy to think (I did, for hours) that you don't need to ever call this. The API docs aren't at all clear on the point:
If your program includes significant loops, you may want to include a call to this function to make sure that the library takes care of the information pushed by the modules on the communication channels. This is not strictly necessary, but it may improve the reactivity of the library for the following commands.
I did some playing around with API-level callbacks before I decided to periodically poll the sensors in my own code, and it's possible that some setting got left enabled in them that is causing these events to accumulate. If that's not the case, I can't imagine why they would say calling YHandleEvents is "not strictly necessary," unless they make ARM devices with unlimited RAM in Switzerland.
Here's the magic static method that thou shalt call periodically, no matter what. I'm doing so once every five seconds and that is taking care of the problem without loading down the system at all. API code that would accumulate unwanted events still smells to me, but it's time to move on.
#noinspection PyUnresolvedReferences
#staticmethod
def HandleEvents(errmsgRef=None):
"""
Maintains the device-to-library communication channel.
If your program includes significant loops, you may want to include
a call to this function to make sure that the library takes care of
the information pushed by the modules on the communication channels.
This is not strictly necessary, but it may improve the reactivity
of the library for the following commands.
This function may signal an error in case there is a communication problem
while contacting a module.
#param errmsg : a string passed by reference to receive any error message.
#return YAPI.SUCCESS when the call succeeds.
On failure, throws an exception or returns a negative error code.
"""
errBuffer = ctypes.create_string_buffer(YAPI.YOCTO_ERRMSG_LEN)
#noinspection PyUnresolvedReferences
res = YAPI._yapiHandleEvents(errBuffer)
if YAPI.YISERR(res):
if errmsgRef is not None:
#noinspection PyAttributeOutsideInit
errmsgRef.value = YByte2String(errBuffer.value)
return res
while len(YAPI._DataEvents) > 0:
YAPI.yapiLockFunctionCallBack(errmsgRef)
if not (len(YAPI._DataEvents)):
YAPI.yapiUnlockFunctionCallBack(errmsgRef)
break
ev = YAPI._DataEvents.pop(0)
YAPI.yapiUnlockFunctionCallBack(errmsgRef)
ev.invokeData()
return YAPI.SUCCESS
I have scripts in both Python and Ruby that run for days at a time and rely on the internet to go to certain domains and collect data. Is there a way to implement a network connectivity check into my script so that I could pause/retry iterations of a loop if there is no connectivity and only restart when there is connectivity?
There may be a more elegant solution, but I'd do this:
require 'open-uri'
def internet_connectivity?
open('http://google.com')
true
rescue => ex
false
end
Well in Python I do something similar with a try except block like the following:
import requests
try:
response = requests.get(URL)
except Exception as e:
print "Something went wrong:"
print e
this is just a sample of what you could do, you can check for error_code or some information on the exception and according to that you can define what to do. I usually put the script to sleep for 10 minutes when something goes wrong on the request.
import time
time.sleep(600)
here's a unix-specific solution:
In [18]: import subprocess
In [19]: subprocess.call(['/bin/ping', '-c1', 'blahblahblah.com'])
Out[19]: 1
In [20]: subprocess.call(['/bin/ping', '-c1', 'google.com'])
Out[20]: 0
ie, ping will return 0 if the ping is successful
Inline way of doing it:
require 'open-uri'
def internet_access?; begin open('http://google.com'); true; rescue => e; false; end; end
puts internet_access?
In Python you can do something like this:
def get_with_retry(url, tries=5, wait=1, backoff=2, ceil=60):
while True:
try:
return requests.get(url)
except requests.exceptions.ConnectionError:
tries -= 1
if not tries:
raise
time.sleep(wait)
wait = min(ceil, wait * backoff)
This tries each request up to tries times, initially delaying wait seconds between attempts, but increasing the delay by a factor of backoff for each attempt up to a maximum of ceil seconds. (The default values mean it will wait 1 second, then 2, then 4, then 8, then fail.) By setting these values, you can set the maximum amount of time you want to wait for the network to come back, before your main program has to worry about it. For infinite retries, use a negative value for tries since that'll never reach 0 by subtracting 1.
At some point you want the program to tell you if it can't get on the network, and you can do that by wrapping the whole program in a try/except that notifies you in some way if ConnectionError occurs.
I have a 'while' loop in my Python app (v2.7) which basically communicates with a Bluetooth GPS device and the loop continues for as long as there is data being received. In essence, the while loop looks like this:-
> data = ""
>
> if data == None:
> print "Connection Failed" else:
> print "Connection Established"
>
> while True:
> data = socket.recv(1024)
Now what I want to do is leave this while loop running continually throughout the life of the program until the bluetooth device is switched off which in turn will end the loop as no data will be being received. Nonetheless, as this while loop continues, I then want to move onto another method which will parse the data.
How can I leave the loop running and move on to the next command? Is threading required? I'm new to the concept of threading and so please be patient with me :)
Thanks
Yeah, because you're essentially going to be doing two things at the same time, you'll have to create a new thread (using threading) or perhaps more simply a new process (using multiprocessing or just os.fork)
The easiest thing to do is to put this part in a function, and use multiprocessing or threading to start the function in a different process. Typically you'll have less weirdness with multiple processes than multiple threads, just FYI.
received = ""
while True:
data = socket.recv(1024)
if not data:
break # ends the while loop
received+=data
do_stuff(received)
I have the following python code that expects data coming from the serial port, and writes it to a file.
import time
import serial
def write_log ( text ):
f = open('logger.log', 'a')
f.write( text )
f.close()
ser = serial.Serial()
ser.port = "/dev/ttyS0"
ser.baudrate = 4800
ser.open()
if ser.isOpen():
while 1:
while ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len ( response ):
write_log( response )
print response
It works to an extent, as after some time, it starts to hang, bringing the CPU all the way up, and not writing anything (or sometimes writing only pieces of text) to the .log file.
The process here is pretty intensive, as my serial port will be writing an 8 bytes string every second, and this python script is supposed to then receive it, and write its contents to the log file.
I'm thinking the problem here is the fact that I'm opening and closing the file too much, and this is somehow making the whole process slow. I'm no python wizz, so any help or advice on improving this code would be greatly appreciated.
Thanks in advance,
You have an infinite loop in your code, and you don't break out from it when there is a problem - or the serial device is no longer open.
Probably use:
while ser.isOpen():
while ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
or even:
while ser.isOpen() && ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
I'm not sure about the sleep, either; you'd do better just waiting in the read for data to become available.
As I think about it, not knowing the methods available in the serial class, the more I think the main loop should be attempting to read from the serial device, hanging happily if there is nothing currently available, and only terminating when the input method indicates there is no more input to come - the device has been closed on you, or has failed in some way.
I think the problem here is that you are polling for updates.
Python's serial.read() function actually blocks the current thread until something becomes readable on the thread, until timeout. What you should do, therefore, is break out another thread to handle serial IO. Have it loop indefinitely, checking a condition (the master thread wants you to stay listening and the serial port is still available). This thread will do something like:
while ser.isOpen() && thisthread_should_keep_doing_this:
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
Then, when you want it to exit, your master thread sets thisthread_should_keep_doing_this=False and when your slave thread has finished reading, it kills itself.
Two things:
Do have the read timeout relatively frequently.
Do not "remote kill" the thread. Pass it a message and have it kill itself. Killing threads remotely creates an awful mess.
See http://pyserial.sourceforge.net/examples.html#miniterm