I have a callback function but the delegate that issues the callback occasionally takes several seconds to provides updates (because it is waiting for data over a remote connection). For my use case this is troublesome because I need to run a function at a regular time, or quit the program. What is the easiest way to have a timer in python that runs a function, or quits the application if I haven't had an update from the delegate within a certain space of time, like five seconds?
def parseMessage(client, userdata, message): # CALLBACK FUNCTION THAT LISTENS FOR NEW MESSAGES
signal = int(str(message.payload.decode("utf-8")))
writeToSerial(signal)
def exceptionState(): # THIS IS THE FUNCTION I WOULD LIKE TO RUN IF THERE'S NO CALLBACK
print("ERROR, LINK IS DOWN, DISABLING SERVER")
exit()
def mqttSignal():
client.on_message = parseMessage # THIS INVOKES THE CALLBACK FUNCTION
client.loop_forever()
This sounds like a good scenario for setting up a background thread that exits if you don't get an event based on a sentinel value. A simple implementation might look like this:
from datetime import datetime, timedelta
from threading import Thread
from time import sleep
class Watcher:
timeout = timedelta(minutes=5)
def __init__(self):
self.last_signal = datetime.now()
Thread(target=self.exception_state).start()
def parse_message(self):
self.last_signal = datetime.now()
# Other handling code here
def exception_state(self):
while True:
if datetime.now() - self.last_signal > self.timeout:
exit("No signal received.")
sleep(5)
Related
I have been trying to implement a mechanism that request live data (such as reqRealTimeBars) and automatically make the script to exit in case the data connection is broken.
I have been making test with the threading.Timer object (https://docs.python.org/3/library/threading.html#timer-objects), that was showing good promises in that direction:
That script for instance (below) is an infinite loop just like the Eclient.run(), yet it quits after 3 seconds, as expected with the timer
import threading
import time
import sys
import _thread
def exit():
_thread.interrupt_main()
if __name__ == '__main__':
t = threading.Timer(3.0, exit)
t.start()
while True:
time.sleep(3)
However, when I try to apply the same logic to the Eclient proper, it doesn't seem to work.
Here below a simplified version of my code. The logic is that a timer is set with a timeout of 10s (or any duration superior to the 5s timespan between realtimebars) when the app is started. Then each time a new bar is received, the timer is canceled then recreated with the same timeout.
This means that normally the code would never reach the timeout, unless the connection with the tws server is broken.
To test the code below: start he tws, then start this script. It should start printing the data in the console.
Then, manually close the tws. That will break the connection. Normally after the 10s, the timer, that has not been "refreshed" should trigger the exit function and make the program stop, just like the example above.
However, it just stays idle, still waiting for the incoming data. If anybody could have a look, that would be awesome.
I think this technique could be a very nimble and nice way to make any live data collection app robust. It only needs to be coupled with a cronjob that run that script every x minutes, and then some extra logic at the beginning that prevents it from being run a second time in case it is already doing so.
from ibapi.wrapper import EWrapper
from ibapi.client import EClient
from ibapi.contract import Contract
from datetime import timedelta, datetime as dt
import threading
import _thread
def exit():
_thread.interrupt_main()
class App(EWrapper, EClient):
def __init__(self):
EWrapper.__init__(self)
EClient.__init__(self, wrapper=self)
self.timeout = threading.Timer(10.0, exit)
self.timeout.start()
def realtimeBar( self, reqId, datetime, open_, high, low, close, volume, wap, count):
bar = {
'open' : open_,
'high' : high,
'low' : low,
'close' : close,
'volume' : volume
}
print(bar)
self.refresh_timeout(reqId)
def refresh_timeout(self, reqId):
self.timeout.cancel()
self.timeout = threading.Timer(10.0, exit)
self.timeout.start()
def make_contracts(app):
contract = Contract()
contract.__dict__.update({'symbol' : 'CL', 'exchange' : 'NYMEX', 'secType': 'FUT', 'lastTradeDateOrContractMonth' : '202008'})
app.reqRealTimeBars(i, contract, 5, "TRADES", 0, [])
if __name__ == '__main__':
app = App()
app.connect("127.0.0.1", 4002, clientId=0)
make_contracts(app)
app.run()
Note that i have made a trial where i would set the timeout to a value < 5s (for instance 3s) - in this case the script does exit thx to the timer...
This is not really an answer to the question as stated, but the meta-answer to the meta question is: use ib-insync and be happy.
I am trying to implement a heartbeat call that works in the background. How do I create a threaded on interval call of say every 30 seconds, which calls the following function:
self.mqConn.heartbeat_tick()
Also how would I stop this thread?
Many thanks.
Use a thread containing a loop
from threading import Thread
import time
def background_task():
while not background_task.cancelled:
self.mqConn.heartbeat_tick()
time.sleep(30)
background_task.cancelled = False
t = Thread(target=background_task)
t.start()
background_task.cancelled = True
Alternatively, you could subclass timer, to make cancelling easy:
from threading import Timer
class RepeatingTimer(Timer):
def run(self):
while not self.finished.is_set():
self.function(*self.args, **self.kwargs)
self.finished.wait(self.interval)
t = RepeatingTimer(30.0, self.mqConn.heartbeat_tick)
t.start() # every 30 seconds, call heartbeat_tick
# later
t.cancel() # cancels execution
Or you could use the Timer class in the threading module:
from threading import Timer
def hello():
print "hello, world"
t = Timer(30.0, hello)
t.start() # after 30 seconds, "hello, world" will be printed
t.cancel() # cancels execution, this only works before the 30 seconds is elapsed
This will not start every x seconds, rather it delays the thread for execution in x seconds. But you can still put that in a loop and use t.is_alive() to see its status.
A quick followup to Eric's answer: you can't subclass Timer in python 2, since it's actually a light function wrapper around the true class: _Timer. If you do you'll get the issue that pops up in this post.
Using _Timer instead fixes it:
from threading import _Timer
class RepeatingTimer(_Timer):
def run(self):
while not self.finished.is_set():
self.function(*self.args, **self.kwargs)
self.finished.wait(self.interval)
t = RepeatingTimer(30.0, self.mqConn.heartbeat_tick)
t.start() # every 30 seconds, call heartbeat_tick
# later
t.cancel() # cancels execution
One way to do this would be to use the circuits application framework like this:
from circuits import Component, Event, Timer
class App(Component):
def init(self, mqConn):
self.mqConn = mqConn
Timer(30, Event.create("heartbeat"), persist=True).register(self)
def heartbeat(self):
self.mqConn.heartbeat_tick()
App().run()
Note: I'm the author of circuits :)
This is just a basic idea and structure -- You would need to adapt this to suit your exact application and requirements!
the following code:
import time
import threading
tasks = dict()
class newTask(object):
def __init__(self, **kw):
[setattr(self, x, kw[x]) for x in kw]
self.object_ret()
def object_ret(self): return self
def task_create(name, timeout, function):
task = newTask(**{
'timeout': int(timeout),
'function': function,
'start': time.time()
})
def set_timeout(v):
while True:
if (time.time() - v.start) > v.timeout:
v.function()
v.start = time.time()
tasks[name] = threading.Thread(target=set_timeout, args=(task,))
tasks[name].start()
def stop(x):
#stops the thread in tasks[x]
is a simple task system that i am using for minor tasks such as pings and timeouts. This works for my needs but if i ever wanted to stop a ping or task that was running, there is no way for me to do so. Is there a way for me to delete or stop that thread that i created using any means possible? I do not care if it is bad or messy to do so, i just want it stopped.
I suggest the following:
In your newTask.init function, add a line "self.alive = True"
In the set_timeout function, replace "while True:" with "while v.alive:"
Store newTask objects in your "tasks" dictionary, not thread objects.
The stop(x) function has one line: "tasks[x].alive = False"
This will cause the thread to die when you call stop(x), where x is the thread's name. It provides a mechanism that allows a thread to die without killing it in some bogus way. I know you said you don't care, but you really should care if you want your multithreaded programs to work.
Second suggestion: read Ulrich Eckhardt's comment carefully and take it seriously; all of his points are well taken.
Signal handler:::
def signal_handler(signal, frame):
print('You pressed Ctrl+C!')
tasks[name].stop()
sys.exit(0)
in the main script, register the handler:::
signal.signal(signal.SIGINT, signal_handler)
signal.pause()
I'm making a wxPython app that I need to update a value from the internet every 15 seconds. Is there any way I can have a function to set the value, and make it run in the background at this interval, without interrupting the program?
EDIT: Here's what I'm trying:
import thread
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
UpdateThread.__init__(self)
def run(self):
while not self.stopped:
downloadValue()
time.sleep(15)
def downloadValue():
print x
UpdateThread.__init__()
What you want is to add a thread that runs your task at a specified pace.
You may have a look at this great answer here : https://stackoverflow.com/a/12435256/667433 to help you achieve this.
EDIT : Here is the code that should work for you :
import time
from threading import Thread # This is the right package name
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
Thread.__init__(self) # Call the super construcor (Thread's one)
def run(self):
while not self.stopped:
self.downloadValue()
time.sleep(15)
def downloadValue(self):
print "Hello"
myThread = UpdateThread()
myThread.start()
for i in range(10):
print "MainThread"
time.sleep(2)
Hope it helps
I have made something similar to this:
-you need a thread to run in the background .
-And a define a 'custom' event , so that the tread can notify the UI when needed
Create the custom WX event
(MyEVENT_CHECKSERVER, EVT_MYEVENT_CHECKSERVER) =
wx.lib.newevent.NewEvent()
on UI "init" you can bind the event , and start the thread
# bind the custom event
self.Bind(EVT_MYEVENT_CHECKSERVER, self.foo)
# and start the worker thread
checkServerThread = threading.Thread(target=worker_checkServerStatus
,args=(self,) )
checkServerThread.daemon = True
checkServerThread.start()
the worker thread can be something like this ,ps. caller is the UI instance
def worker_checkServerStatus(caller):
while True:
# check the internet code here
evt = MyEVENT_CHECKSERVER(status='Some internet Status' ) #make a new event
wx.PostEvent(caller, evt) # send the event to the UI
time.sleep(15) #ZZZzz for a bit
Edit: miss read the question...
Another way to do that is with a timer:
import threading
stopNow = 0
def downloadValue():
print("Running downloadValue")
if not stopNow: threading.Timer(15,downloadValue).start()
downloadValue()
This is a classic pattern for repeating a function: the function itself adds a timed call to itself. To start the cycle, call the function (it returns immediately). To break the cycle set stopNow to 1.
I've got an event-driven chatbot and I'm trying to implement spam protection. I want to silence a user who is behaving badly for a period of time, without blocking the rest of the application.
Here's what doesn't work:
if user_behaving_badly():
ban( user )
time.sleep( penalty_duration ) # Bad! Blocks the entire application!
unban( user )
Ideally, if user_behaving_badly() is true, I want to start a new thread which does nothing but ban the user, then sleep for a while, unban the user, and then the thread disappears.
According to this I can accomplish my goal using the following:
if user_behaving_badly():
thread.start_new_thread( banSleepUnban, ( user, penalty ) )
"Simple" is usually an indicator of "good", and this is pretty simple, but everything I've heard about threads has said that they can bite you in unexpected ways. My question is: Is there a better way than this to run a simple delay loop without blocking the rest of the application?
instead of starting a thread for each ban, put the bans in a priority queue and have a single thread do the sleeping and unbanning
this code keeps two structures a heapq that allows it to quickly find the soonest ban to expire and a dict to make it possible to quickly check if a user is banned by name
import time
import threading
import heapq
class Bans():
def __init__(self):
self.lock = threading.Lock()
self.event = threading.Event()
self.heap = []
self.dict = {}
self.thread = threading.thread(target=self.expiration_thread)
self.thread.setDaemon(True)
self.thread.start()
def ban_user(self, name, duration):
with self.lock:
now = time.time()
expiration = (now+duration)
heapq.heappush(self.heap, (expiration, user))
self.dict[user] = expiration
self.event.set()
def is_user_banned(self, user):
with self.lock:
now = time.time()
return self.dict.get(user, None) > now
def expiration_thread(self):
while True:
self.event.wait()
with self.lock:
next, user = self.heap[0]
now = time.time()
duration = next-now
if duration > 0:
time.sleep(duration)
with self.lock:
if self.heap[0][0] = next:
heapq.heappop(self.heap)
del self.dict(user)
if not self.heap:
self.event.clear()
and is used like this:
B = Bans()
B.ban_user("phil", 30.0)
B.is_user_banned("phil")
Use a threading timer object, like this:
t = threading.Timer(30.0, unban)
t.start() # after 30 seconds, unban will be run
Then only unban is run in the thread.
Why thread at all?
do_something(user):
if(good_user(user)):
# do it
else
# don't
good_user():
if(is_user_baned(user)):
if(past_time_since_ban(user)):
user_good_user(user)
elif(is_user_bad()):
ban_user()
ban_user(user):
# add a user/start time to a hash
is_user_banned()
# check hash
# could check if expired now too, or do it seperately if you care about it
is_user_bad()
# check params or set more values in a hash
This is language agnostic, but consider a thread to keep track of stuff. The thread keeps a data structure that has something like "username" and "banned_until" in a table. The thread is always running in the background checking the table, if banned_until is expired, it unblocks the user. Other threads go on normally.
If you're using a GUI,
most GUI modules have a timer function which can abstract all the yuck multithreading stuff,
and execute code after a given time,
though still allowing the rest of the code to be executed.
For instance, Tkinter has the 'after' function.