We Keep Coding

How to continue code after thread? Confused about the flow of this code

Sorry I'm new to programming, and don't really understand how this Thread thing works. My goal was for this input to be timed, and I found some code that does that. However, I'm confused about the structure of this Thread because if you are "too slow", the program never continues on to print "checkpoint" as desired. It just sort of... freezes... Why is it getting stuck?
import time
from threading import Thread
answer = None
def check():
# waits for user input for 3 seconds
for i in range(3):
time.sleep(1)
if answer != None:
return
print('too slow')
Thread(target = check).start()
answer = input("Input something: ")
print('checkpoint')
One thing I tried is:
t = Thread(target = check)
t.start()
answer = input("Input something: ")
# also tried t.join()
if t.is_alive:
print('hi')
I tried to solve this program by trying to raise and catch an exception. However, I couldn't catch the exception. How do I catch it? (Or is there another solution to the problem I am having?)
import time
from threading import Thread
answer = None
def check():
# waits for user input for 3 seconds
for i in range(3):
time.sleep(1)
if answer != None:
return
print('too slow')
# was hoping to catch this as an exception
raise TimeoutError
# starts new thread
Thread(target = check).start()
# prompts user for an input
answer = input("Input something: ")
print('checkpoint')
What's good:
When you type something into the input prompt within 3 seconds, it prints "checkpoint" and continues on with code.
What's bad:
If you take "too long", the program prints "too slow!" as expected, BUT then it stops executing code and just sort of... freezes. So to try to fix this, I was hoping to raise a Timeout Error and then catch it, but I don't know how to catch it. This didn't catch the error:
try:
Thread(target = check).start()
except:
pass
This didn't either:
try:
answer = input("Input something: ")
except:
pass
Could I get some help? Thank you!
Edit: Forgot to mention that I am using linux so a lot of the solutions for my application did not work for me like msvcrt or keyboard. And modules that do work for Linux seem not to be "non-blocking."

You should think of the two threads as two separate programs, but sharing the same variables.
Thread 1 consists of everything that isn't indented in your code. It launches a thread, then it waits for user input, then it prints "checkpoint". Then it's done.
Thread 2 consists of the function check. It checks to see if the variable isn't None. If that happens it's done. If that doesn't happen in three seconds, it prints "too slow" and now it's done.
Neither thread "knows" what the other one is doing, except they share one variable, answer.
The program as a whole will exit when all its threads are finished.
That's it. That's what you've written. So if you type something, the program exits because Thread 1 will always exit after you type something. Thread 2 exits once it sees the variable isn't None.
If you don't type anything, Thread 1 will just sit there and wait for you, forever. That's how the input function works. Thread 2 will exit after 3 seconds or less, but that doesn't affect Thread 1.
You can't throw an Exception from one Thread to another. So you can't throw an exception from Thread 2 and have Thread 1 handle it.
Have you tried typing something AFTER the message "too slow" appears? When you do, Thread 1 (and therefore your program) will exit.
The bottom line is that you can't use the input function in cases like this, because that function blocks the flow of its thread until the user types something. There is nothing any other thread can do to make it continue.

DISCLAIMER: THIS DOESN'T ANSWER THE QUESTION BUT IN CASE YOU WANT TO KNOW HOW I GOT AROUND THE "INPUT" THING, HERE IS MY SOLUTION TO THE PROBLEM.
Actually I found something that works! It's a little strange but it works for what I am trying to do, thanks to #rayryeng 's answer here: detect key press in python?.
Problem Statement: Continue the program when 'enter' is pressed, and timeout if input takes too long. This does exactly that, although it displays strangely to the console... PS. I had to run this in my terminal as 'sudo' or it wouldn't work in my scratch file for whatever reason.
import curses
import os
from time import time, sleep
def main(win):
win.nodelay(True) # True turns on "non-blocking"
key=""
win.clear()
win.addstr("Please press 'Enter' to continue:")
start_time = time()
while 1:
end_time = time()
try:
if end_time-start_time > 5: # 5 seconds
return 'You are too slow!'
else:
key = win.getkey()
if key == os.linesep:
return 'OK. Continuing on...'
except Exception as e:
# No input
pass
p = curses.wrapper(main)
print(p) #-> either 'You are too slow!' or 'OK. Continuing on...'
I guess if you actually wanted to store the input you can modify it to be something like this:
def main(win):
win.nodelay(True) # True turns on "non-blocking"
key=""
win.clear()
win.addstr("Please press 'Enter' to continue:")
start_time = time()
while 1:
end_time = time()
try:
if end_time-start_time > 5: # 5 seconds
return 'You are too slow!'
else:
key = win.getkey() # gets a single char
if key: # == os.linesep:
return str(key) # returns that single char
except Exception as e:
# No input
pass
p = curses.wrapper(main)
print(p) #-> either 'You are too slow!' or character entered
And if you want to store more characters you could do something like this (just be aware that it also stores the "enter" key in the resulting string):
import curses
import os
from time import time, sleep
def main(win):
win.nodelay(True) # True turns on "non-blocking"
key=""
win.clear()
win.addstr("Please press 'Enter' to continue:")
start_time = time()
result = key # empty string
while 1:
end_time = time()
try:
if end_time-start_time > 5: # 5 seconds
return 'You are too slow!'
else:
key = win.getkey() # gets single char
result = result + str(key) # adds characters to the empty string
if key == os.linesep: # "new line"
return result
except Exception as e:
# No input
pass
p = curses.wrapper(main)
print(p) #-> either 'You are too slow!' or characters entered

Multiple stdout w/ flush going on in Python threading

I have a small piece of code that I made to test out and hopefully debug the problem without having to modify the code in my main applet in Python. This has let me to build this code:
#!/usr/bin/env python
import sys, threading, time
def loop1():
count = 0
while True:
sys.stdout.write('\r thread 1: ' + str(count))
sys.stdout.flush()
count = count + 1
time.sleep(.3)
pass
pass
def loop2():
count = 0
print ""
while True:
sys.stdout.write('\r thread 2: ' + str(count))
sys.stdout.flush()
count = count + 2
time.sleep(.3)
pass
if __name__ == '__main__':
try:
th = threading.Thread(target=loop1)
th.start()
th1 = threading.Thread(target=loop2)
th1.start()
pass
except KeyboardInterrupt:
print ""
pass
pass
My goal with this code is to be able to have both of these threads displaying output in stdout format (with flushing) at the same time and have then side by side or something. problem is that I assume since it is flushing each one, it flushes the other string by default. I don't quite know how to get this to work if it is even possible.
If you just run one of the threads, it works fine. However I want to be able to run both threads with their own string running at the same time in the terminal output. Here is a picture displaying what I'm getting:
terminal screenshot
let me know if you need more info. thanks in advance.

Instead of allowing each thread to output to stdout, a better solution is to have one thread control stdout exclusively. Then provide a threadsafe channel for the other threads to dispatch data to be output.
One good method to achieve this is to share a Queue between all threads. Ensure that only the output thread is accessing data after it has been added to the queue.
The output thread can store the last message from each other thread and use that data to format stdout nicely. This can include clearing output to display something like this, and update it as each thread generates new data.
Threads
#1: 0
#2: 0
Example
Some decisions were made to simplify this example:
There are gotchas to be wary of when giving arguments to threads.
Daemon threads terminate themselves when the main thread exits. They are used to avoid adding complexity to this answer. Using them on long-running or large applications can pose problems. Other
questions discuss how to exit a multithreaded application without leaking memory or locking system resources. You will need to think about how your program needs to signal an exit. Consider using asyncio to save yourself these considerations.
No newlines are used because \r carriage returns cannot clear the whole console. They only allow the current line to be rewritten.
import queue, threading
import time, sys
q = queue.Queue()
keepRunning = True
def loop_output():
thread_outputs = dict()
while keepRunning:
try:
thread_id, data = q.get_nowait()
thread_outputs[thread_id] = data
except queue.Empty:
# because the queue is used to update, there's no need to wait or block.
pass
pretty_output = ""
for thread_id, data in thread_outputs.items():
pretty_output += '({}:{}) '.format(thread_id, str(data))
sys.stdout.write('\r' + pretty_output)
sys.stdout.flush()
time.sleep(1)
def loop_count(thread_id, increment):
count = 0
while keepRunning:
msg = (thread_id, count)
try:
q.put_nowait(msg)
except queue.Full:
pass
count = count + increment
time.sleep(.3)
pass
pass
if __name__ == '__main__':
try:
th_out = threading.Thread(target=loop_output)
th_out.start()
# make sure to use args, not pass arguments directly
th0 = threading.Thread(target=loop_count, args=("Thread0", 1))
th0.daemon = True
th0.start()
th1 = threading.Thread(target=loop_count, args=("Thread1", 3))
th1.daemon = True
th1.start()
# Keep the main thread alive to wait for KeyboardInterrupt
while True:
time.sleep(.1)
except KeyboardInterrupt:
print("Ended by keyboard stroke")
keepRunning = False
for th in [th0, th1]:
th.join()
Example Output:
(Thread0:110) (Thread1:330)

Dynamic Threading

Hello I am having an issue with Python Threads.
What I am aiming to do is create a function that launches other functions as a thread when called. Unfortunately I am running into two issues (Besides my lack of knowledge in Python.)
1: If I put quotes around: "globals()[T_Name[i]]()" it treats it as a string and executes the code normally.
2: If I omit the quotes around globals()[T_Name[i]]() it launches the first function immediately and does not process through the rest of the script to launch it as a thread.
If anyone could provide some insight I apologize for the formatting I will be bumping it up to PEP 8 standards eventually.
Code:
import threading
import time
T_Name=("Write_Done", "Write_Pin")
T_Time=[]
Tr=[]
for i, Nu in enumerate(T_Name):
Tr.append("T" + str(i))
T_Time.append("0")
def Write_Done():
while True:
print("Done")
time.sleep(5)
def Write_Pin():
while True:
print("Pin")
time.sleep(15)
def Thread_Checker():
while True:
time.sleep(5)
for i, TH in enumerate(T_Time):
if (time.time() - int(TH)) < 30:
pass
#thread is still rocking
else:
#thread has failed Time to get her done.
Tr[i] = threading.Thread(target=("globals()[T_Name[i]]()"))
print("starting" + T_Name[i])
Tr[i].daemon = True
Tr[i].start()
print("Test if alive")
if Tr[0].is_alive():
print("I LIVE!")
else:
print("I ded")
Thread_Checker()

Use a lambda function to create something that is actually callable as the target, but defers the call of what you want until the target is called.
Tr[i] = threading.Thread(target=lambda: globals()[T_Name[i]]())

Consistent assert for parallel runs in python?

it seems to me that in a python code that runs in parallel, an assert that is failed by at least one processor should abort all the processors, so that:
1) the error message is clearly visible (with the stack trace)
2) the remaining processors do not keep waiting forever.
However this is not what the standard assert does.
This question has already been asked in
python script running with mpirun not stopping if assert on processor 0 fails
but I am not satisfied by the answer. There it is suggested to use the comm.Abort() function, but this only answers point 2) above.
So I was wondering: is there a standard "assert" function for parallel codes (eg with mpi4py), or should I write my own assert for that purpose?
Thanks!
Edit -- here is my try (in a class but could be outside), which can surely be improved:
import mpi4py.MPI as mpi
import traceback
class My_code():
def __init__(self, some_parameter=None):
self.current_com = mpi.COMM_WORLD
self.rank = self.current_com.rank
self.nb_procs = self.current_com.size
self.my_assert(some_parameter is not None)
self.parameter = some_parameter
print "Ok, parameter set to " + repr(self.parameter)
# some class functions here...
def my_assert(self, assertion):
"""
this is a try for an assert function that kills
every process in a parallel run
"""
if not assertion:
print 'Traceback (most recent call last):'
for line in traceback.format_stack()[:-1]:
print(line.strip())
print 'AssertionError'
if self.nb_procs == 1:
exit()
else:
self.current_com.Abort()

I think that the following piece of code answers the question. It is derived from the discussion pointed by Dan D.
import mpi4py.MPI as mpi
import sys
# put this somewhere but before calling the asserts
sys_excepthook = sys.excepthook
def mpi_excepthook(type, value, traceback):
sys_excepthook(type, value, traceback)
if mpi.COMM_WORLD.size > 1:
mpi.COMM_WORLD.Abort(1)
sys.excepthook = mpi_excepthook
# example:
if mpi.COMM_WORLD.rank == 0:
# with sys.excepthook redefined as above this will kill every processor
# otherwise this would only kill processor 0
assert 1==0
# assume here we have a lot of print messages
for i in range(50):
print "rank = ", mpi.COMM_WORLD.rank
# with std asserts the code would be stuck here
# and the error message from the failed assert above would hardly be visible
mpi.COMM_WORLD.Barrier()

python watchdog for threads

Im writing simple app, which reads (about a million) lines from file, copy those lines into list, and if next line will be different then previous it runs a thread, to do some job with that list. Thread job is based on tcp sockets, sending and receiving commands via telnet lib.
Sometimes my application hangs and does nothing. All telnet operations I wrapped into try-except statements, also read and write into sockets has timeouts.
I thought about writing watchdog, which will do sys.exit() or something similiar on that hang condtition. But, for now I'm thinking how to create it, and still got no idea how to do it. So if You can trace me, it would be great.
For that file I'm creating 40 threads. Pseudo code looks:
lock = threading.Lock()
no_of_jobs = 0
class DoJob(threading.Thread):
def start(self, cond, work):
self.work = work
threading.Thread.start(self)
def run(self)
global lock
global no_of_jobs
lock.acquire()
no_of_jobs += 1
lock.release()
# do some job, if error or if finished, decrement no_of_jobs under lock
(...)
main:
#starting conditions:
with open(sys.argv[1]) as targetsfile:
head = [targetsfile.next() for x in xrange(1)]
s = head[0]
prev_cond = s[0]
work = []
for line in open(sys.argv[1], "r"):
cond = line([0])
if prev_cond != cond:
while(no_of_jobs>= MAX_THREADS):
time.sleep(1)
DoJob(cond, work)
prev_cond = cond
work = None
work = []
work.append(line)
#last job:
DoJob(cond, work)
while threading.activeCount() > 1:
time.sleep(1)
best regards
J

I have successfully used code like below in the past (from a python 3 program I wrote):
import threading
def die():
print('ran for too long. quitting.')
for thread in threading.enumerate():
if thread.isAlive():
try:
thread._stop()
except:
pass
sys.exit(1)
if __name__ == '__main__':
#bunch of app-specific code...
# setup max runtime
die = threading.Timer(2.0, die) #quit after 2 seconds
die.daemon = True
die.start()
#after work is done
die.cancel()