I have a long running python process running headless on a raspberrypi (controlling a garden) like so:
from time import sleep
def run_garden():
while 1:
/* do work */
sleep(60)
if __name__ == "__main__":
run_garden()
The 60 second sleep period is plenty of time for any changes happening in my garden (humidity, air temp, turn on pump, turn off fan etc), BUT what if i want to manually override these things?
Currently, in my /* do work */ loop, i first call out to another server where I keep config variables, and I can update those config variables via a web console, but it lacks any sort of real time feel, because it relies on the 60 second loop (e.g. you might update the web console, and then wait 45 seconds for the desired effect to take effect)
The raspberryPi running run_garden() is dedicated to the garden and it is basically the only thing taking up resources. So i know i have room to do something, I just dont know what.
Once the loop picks up the fact that a config var has been updated, the loop could then do exponential backoff to keep checking for interaction, rather than wait 60 seconds, but it just doesnt feel like that is a whole lot better.
Is there a better way to basically jump into this long running process?
Listen on a socket in your main loop. Use a timeout (e.g. of 60 seconds, the time until the next garden update should be performed) on your socket read calls so you get back to your normal functionality at least every minute when there are no commands coming in.
If you need garden-tending updates to happen no faster than every minute you need to check the time since the last update, since read calls will complete significantly faster when there are commands coming in.
Python's select module sounds like it might be helpful.
If you've ever used the unix analog (for example in socket programming maybe?), then it'll be familiar.
If not, here is the select section of a C sockets reference I often recommend. And here is what looks like a nice writeup of the module.
Warning: the first reference is specifically about C, not Python, but the concept of the select system call is the same, so the discussion might be helpful.
Basically, it allows you to tell it what events you're interested in (for example, socket data arrival, keyboard event), and it'll block either forever, or until a timeout you specify elapses.
If you're using sockets, then adding the socket and stdin to the list of events you're interested in is easy. If you're just looking for a way to "conditionally sleep" for 60 seconds unless/until a keypress is detected, this would work just as well.
EDIT:
Another way to solve this would be to have your raspberry-pi "register" with the server running the web console. This could involve a little bit extra work, but it would give you the realtime effect you're looking for.
Basically, the raspberry-pi "registers" itself, by alerting the server about itself, and the server stores the address of the device. If using TCP, you could keep a connection open (which might be important if you have firewalls to deal with). If using UDP you could bind the port on the device before registering, allowing the server to respond to the source address of the "announcement".
Once announced, when config. options change on the server, one of two things usually happen:
A) You send a tiny "ping" (in the general sense, not the ICMP host detection protocol) to the device alerting it that config options have changed. At this point the host would immediately request the full config. set, acquiring the update with it.
B) You send the updated config. option (or maybe the entire config. set) back to the device. This decreases the number of messages between the device and server, but would probably take more work as it seems like more a deviation from your current setup.
Why not use an event based loop instead of sleeping for a certain amount of time.
That way your loop will only run when a change is detected, and it will always run when a change is detected (which is the point of your question?).
You can do such a thing by using:
python event objects
Just wait for one or all of your event objects to be triggered and run the loop. You can also wait for X events to be done, etc, depending if you expect one variable to be updated a lot.
Or even a system like:
broadcasting events
Related
I found a basic space invaders pygame on Youtube and I want to modify it in order that, as of right now, the server is doing all the processing and drawing, and the client only sends keyboard input(all run on localhost). The problem is that the game is no longer that responsive after I implemented this mechanism. It appears to be about 1 second delay after I press a key to when the ship is actually moving (when starting the game from pycharm, when it starts from cmd it's much worse).
I don't have any idea why this is happening because there isn't really anything heavy to process and I could really use your help.
I also monitored the Ethernet traffic in wireshark and there seems to be sent about 60-70 packets each second.
Here is the github link with all the necesary things: https://github.com/PaaulFarcas/C-S-Game
I would expect this code in the main loop is the issue:
recv = conn.recv(661)
keys = pickle.loads(recv)
The socket function conn.recv() will block until 661 bytes are received, or there is some socket event (like being closed). So your program is blocking every iteration of the main loop waiting for the data to arrive.
You could try using socket.setblocking( False ) as per the manual.
However I prefer to use the select module (manual link), as I like the better level of control it gives. Basically you can use it to know if any data has arrived on the socket (or if there's an error). This gives you a simple select-read-buffer type logic loop:
procedure receiveSocketData
Use select on the socket, with an immediate timeout.
Did select indicate any data arrived on my socket?
Read the data, appending it to a Rx-buffer
Does the Rx-buffer contain enough for a whole packet?
take the packet-chunk from the head of the Rx-buffer
decode & return it
Else
Keep the Rx-Buffer somewhere safe
return None
Did any errors happen on my socket
clear Rx-Buffer
close socket
return error
I guess using an unknown-sized packet, you could try to un-pickle it, and return OK when successful... this is quite inefficient though. I would use a fixed size packet and the struct module to pack and unpack it in network-byte-order.
I am controlling a test system using PyVisa/GPIB. The system is comprised of two separate testers (A and B) and a laptop. The the laptop passively listens for a GPIB message from tester A, when received the laptop triggers tester B.
I am using the following code to passively listen for events from tester A:
rm = visa.ResourceManager()
con = "GPIB0::3"
tester_A = rm.get_instrument(con, timeout=5000)
while True:
event = None
try:
event = tester_A.read_raw()
except VisaIOError:
logger.warning("Timeout expired.")
if event != None:
# Do something
Is there a better way to listen and respond to events from tester A? Is there a better way to control this system via GPIB?
The approach you describe will work, but as you are experiencing, is not ideal if you are not quite sure when the instrument is going to respond. The solution lies in using the GPIB's service request (SRQ) functionality.
In brief, the GPIB connection also provides various status registers that allow you to quickly check, for example, whether the instrument is on, whether an error has occurred, etc. (pretty picture). Some of the bits in this register can be set so that they turn on or off after particular events, for example when an operation is complete. This means you tell the instrument to execute a series of commands that you suspect will take a while, and to then flip a bit in the status register to indicate it is done.
From within your software you can do a number of things to make use of this:
Keep looping through a while loop until the status bit indicates that the operation is complete - this is very crude and I wouldn't recommend it.
VISA has a viWaitOnEvent function that allows you to wait until the status bit indicatesthat the operation is complete - a good solution if you need all execution to stop until the instrument has taken a measurement.
VISA also allows you to create an event that occurs when the status bit has flipped - This is a particularly nice solution as it allows you to write an event handler to handle the event.
I have script which can be run by any user who is connected to a server. This script writes to a single log file, but there is no restriction on who can use it at one time. So multiple people could attempt to write to the log and data might be lost. Is there a way for one instance of the code to know if other instances of that code are running? Moreover, is it possible to gather this information dynamically? (ie not allow data saving for the second user until the first user has completed hes/her task)
I know I could do this with a text file. So I could write the user name to the file when the start, then delete it when they finish, but this could lead to errors if the either step misses, such as an unexpected script termination. So what other reliable ways are there?
Some information on the system: Python 2.7 is installed on a Windows 7 64-bit server via Anaconda. All connected machines are also Windows 7 64-bit. Thanks in advance
Here is an implementation:
http://www.evanfosmark.com/2009/01/cross-platform-file-locking-support-in-python/
If you are using a lock, be aware that stale locks (that are left by hung or crashed processes) can be a bitch. Have a process that periodically searches for locks that were created longer than X minutes ago and free them.
It just in't clean allowing multiple users to write to a single log and hoping things go ok..
why dont you write a daemon that handles logs? other processes connect to a "logging port" and in the simplest case they only succeed if no one else has connected.
you can just modify the echoserver example given here: (keep a timeout in the server for all connections)
http://docs.python.org/release/2.5.2/lib/socket-example.html
If you want know exactly who logged what, and make sure no one unauthorized gets in, you can use unix sockest to restrict it to only certain uids/gids etc.
here is a very good example
NTEventLogHandler is probably the easiest way for logging to a given Windows machine/server, but it might make more sense to use SyslogHandler if you have a syslog sink on a Unix server.
The catch I can think of with SyslogHandler is that you'll likely need to poke holes through the Windows firewall in order to send packets over the syslog protocol, i.e., 514/TCP ("reliable syslog") and 514/UDP (traditional or "unreliable syslog").
From time to time I suddenly have a need to connect to a device's console via its serial port. The problem is, I never remember what port settings (baud rate, data bits, stop bits, etc...) to use with each particular device, and documentation never seems to be lying around when it's really needed.
I wrote a Python script, which uses a simple brute-force method (i.e. iterates over all possible settings, sends some test input and displays the response for a human to decide if it makes sense ), but:
it takes a long time to complete
does not always work (perhaps port reset/timeout issues)
just does not seem like a proper way to do this :)
So the question is: does anyone know of a procedure to auto-detect what port settings the remote device is using?
Although part 1 is no direct answer to your question:
There are devices, which just have a autodetection (called Auto-bauding) method included, that means: Send a character using your current settings (9k6, 115k2, ..) to the device and chances are high that the device will answer with your (!) settings. I've seen this on HP switches.
Second approach: try to re-order the connection possibilities. E.g. chances are high that the other end uses 9k6 with no hardware handshake, but less that it uses 38k4 with software Xon/Xoff.
If you break down your tries into just a few, the "brute force" method will be much more efficient.
I made an IRC bot which uses a while true loop to receive whatever is said.
To receive I use recv(500), but that stops the loop if there isn't anything to receive, but i need the loop to continue even if there isn't anything to receive.
I need a makeshift timer to continue running.
Example code:
/A lot of stuff/
timer=0
while 1:
timer=timer+1
line=s.recv(500) #If there is nothing to receive, the loop and thus the timer stop.
/A lot of stuff/
So either I need a way to stop it stopping the loop, or I need a better timer.
You can settimeout on the socket so that the call returns promptly (with a suitable exception, so you'll need a try/except around it) if nothing's there -- a timeout of 0.1 seconds actually works better than non-blocking sockets in most conditions.
This is going to prove a bad way to design a network application. I recommend looking into twisted, a networking library with an excellent implementation of the IRC protocol for making a client (like your bot) in twisted.words.protocols.irc.
http://www.habnabit.org/twistedex.html is an example of a very basic IRC bot written using twisted. With very little code, you are able to access a whole, correct, efficient, reconnecting implementation of IRC.
If you are intent on writing this from a socket level yourself, I still recommend studying a networking library like twisted to learn about how to effectively implement network apps. Your current technique will prove less effective than desired.
I usually use irclib which takes care of this sort of detail for you.
If you want to do this with low-level python, consider using the ready_sockets = select.select([s.fileno()], [], [], 0.1) -- this will test the socket s for readability. If your socket's file number is not returned in ready_sockets, then there is no data to read.
Be careful not to use the timout of "0" if you are going to call select repeatedly in a loop that does not otherwise yield the CPU -- that would consume 100% of the CPU as the loop executes. I gave 0.1 seconds timeout as an example; in this case, your timer variable would be counting tenths of a second.
Here's an example:
timer=0
sockets_to_check = [s.fileno()]
while 1:
ready_sockets = select.select(sockets_to_check, [], sockets_to_check, 0.1)
if (len(ready_sockets[2]) > 0):
# Handle socket error or closed connection here -- our socket appeared
# in the 'exceptional sockets' return value so something has happened to
# it.
elif (len(ready_sockets[0]) > 0):
line = s.recv(500)
else:
timer=timer+1 # Note that timer is not incremented if the select did not
# incur a full 0.1 second delay. Although we may have just
# waited for 0.09999 seconds without accounting for that. If
# your timer must be perfect, you will need to implement it
# differently. If it is used only for time-out testing, this
# is fine.
Note that the above code takes advantage of the fact that your input lists contain only one socket. If you were to use this approach with multiple sockets, which select.select does support, the len(ready_sockets[x]) > 0 test would not reveal which socket is ready for reading or has an exception.