I have some background task that i want to start an be able to safely quit by user input.
To do that i have a thread in which a process pool with a continous task is started. There is a input lock to stop printing and to wait for user input and a event to stop the whole process.
What surprises me is that the processes seem to start and do their work if there is a time.sleep after the start of the thread with processpool (6th line in main).
import multiprocessing as mp
import time
import threading as tr
def init(e, l):
global stop_event
global input_lock
stop_event = e
input_lock = l
def stupid_task(n):
while not stop_event.is_set():
with input_lock:
print(n)
time.sleep(2)
def test_mng(n, event, lock):
with mp.Pool(n, initializer=init, initargs=(event, lock,)) as p:
print("before")
p.map(stupid_task, range(1, n + 1))
print("after")
p.close()
p.join()
def main():
i_lock = mp.Lock()
s_event = mp.Event()
thread = tr.Thread(target=test_mng, args=(3, s_event, i_lock))
init(s_event,i_lock)
thread.start()
time.sleep(1) # if this line is commented out only "before" is printed
while not stop_event.is_set():
input("")
with input_lock:
print("stopped")
eingabe = input("type q to quit")
if eingabe == "q":
stop_event.set()
if __name__ == "__main__":
main()
I ask myself what is stopping the process pool from doing it's work. Do i do something fundamentally wrong? The time.sleep seems a little bit hacky.
I my opinion, you are running your script from an IDE (like PyCharm), but not from the Console. Your IDE is catching the keyboard events.
You can simplify the processing:
The main process can wait for the user input,
The thread can do the "stupid task".
Here is a possible solution:
# coding: utf-8
import multiprocessing as mp
import threading as tr
import time
stop_event = None
def init(event):
global stop_event
stop_event = event
def stupid_task(n):
while not stop_event.is_set():
print(n)
time.sleep(2)
def test_mng(n, event):
with mp.Pool(n, initializer=init, initargs=(event,)) as p:
print("before")
p.map(stupid_task, range(1, n + 1))
print("after")
p.close()
p.join()
def main():
print("type 'q' <ENTER> to quit")
s_event = mp.Event()
init(s_event)
thread = tr.Thread(target=test_mng, args=(3, s_event,))
thread.start()
while not stop_event.is_set():
c = input("")
if c in "qQ":
stop_event.set()
if __name__ == "__main__":
main()
Related
I checked threads and solutions about multiprocessing on pyhton 3, but could not adapt it to my case because its containing a loop:
import time
anotherFunctionRunning = False
def anotherFunction():
global anotherFunctionRunning
anotherFunctionRunning = True
print("Another function started")
time.sleep(5)
print("Another function stopped running")
anotherFunctionRunning = False
def mainLoop():
global anotherFunctionRunning
while True:
print("running")
time.sleep(1)
if (anotherFunctionRunning == False):
anotherFunction()
else:
print("loop running, another function running")
print("loop ended")
mainLoop()
My problem here is when anotherFunction starts running, script waits it to be over (in the example 5 seconds) and continues the loop.
I want it to continue the loop while anotherFunction running.
I saw this common solution but could not adapt it to my case and dont know how to do because its becoming too complex:
from multiprocessing import Process
def func1:
#does something
def func2:
#does something
if __name__=='__main__':
p1 = Process(target = func1)
p1.start()
p2 = Process(target = func2)
p2.start()
Any ideas ?
Thanks for support
I'm not sure I understand what you want, but I think this should do the job
import time
from threading import Thread
anotherFunctionRunning = False
def anotherFunction():
global anotherFunctionRunning
anotherFunctionRunning = True
print("Another function started")
time.sleep(5)
print("Another function stopped running")
anotherFunctionRunning = False
def mainLoop():
global anotherFunctionRunning
thread = None
counter = 0
while counter < 5:
counter += 1
print("running")
time.sleep(1)
if anotherFunctionRunning == False:
thread = Thread(target= anotherFunction, name="AnotherThread")
thread.start()
else:
print("loop running, another function running")
print("loop ended")
thread.join()
mainLoop()
Beware however I have royally ignored the dangers of competitive access
So, i would claim that i understand how Asyncio, Multiprocessing, Threading etc. works, basically. I know how to listen for keystrokes too - there are many good examples on this site.
However i was unable to combine both into one. I have a programm that runs continously in a loop, until it runs into certain cases where it stops. In these cases, it uses a Multiprocessing.Queue() to prompt for user input on wether it should continue or not.
All of this works, so far so good. Now i want to add a second catch case here: The programm should, once it starts running, immediatly cease working as soon as i press a certain button (lets say Escape).
This is the very dumbed down version of my programm:
test.py:
from test3 import Counter
from multiprocessing import Process, Queue
import sys
def main(q, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p1.join()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, user_Input)
else:
sys.exit()
if __name__ == "__main__":
q = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, passed_variable=user_Input)
test3.py:
import time
class Counter:
def counting(self, q, user_input):
x = 0
while True:
print(str(x) + " " + user_input)
if x == 4:
q.put(False)
break
time.sleep(1)
x += 1
I tried everything i could think of, in no case did i get the desired result, and no question i found here was able to help me in this specific case.
You can solve this using keyboard and then creating a second Queue():
from test3 import Counter
from multiprocessing import Process, Queue
import sys
import keyboard
def main(q, queue2, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p2 = Process(target=keyCatcher, args=(queue2,))
p2.start()
if queue2.get() == False:
p1.terminate()
print("Terminating Programm")
sys.exit()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, queue2, user_Input)
else:
sys.exit()
def keyCatcher(queue2):
while True:
if keyboard.is_pressed('q'): # if key 'q' is pressed
queue2.put(False)
if __name__ == "__main__":
q = Queue()
queue2 = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, queue2, passed_variable=user_Input)
The crux is:
p1.start()
p1.join()
Which means after main() starts p1, it waits for it to finish. So there's no chance to interrupt it while processing.
You need to:
wait for p1 to finish
while waiting, see if the main process gets a 'q'
if the main process gets a 'q', stop it.
Something like:
p1.start()
while p1.is_alive():
k = keyboard.read_key()
if k == 'q':
p1.terminate()
I have a script that runs but it has a tendency of hanging. I am trying to make another script to run it using threads, but the auto-restarting function does not seem to be working.
What I want this script to do is to restart the function every 900 seconds, or if it is possible, restart it whenever it hangs.
import time
import threading
import os
import functionname
def restart():
time.sleep(900)
os.execl('currentfilepath')
def res():
while True:
try:
# do your works
t = threading.Thread(target=restart, args=(), name='reset')
t.start()
functionname()
except:
print('error')
finally:
print('done')
if __name__ == '__main__':
res()
so I managed to find a way to solve this.
import multiprocessing
import time
t=600 # time to wait
def bar():
# do stuff
def barn():
for i in range(t*500): # runs for 500 times
# print ("Tick"+str(i)+'n') # just for tracking progress
time.sleep(1)
if i % t == 0:
print('restarting')
p = multiprocessing.Process(target=bar)
p.terminate
p = multiprocessing.Process(target=bar)
p.start()
if __name__ == '__main__':
p = multiprocessing.Process(target=bar)
p.start()
p1 = multiprocessing.Process(target=barn)
p1.start()
Probably not the most elegant but hey, it works.
Your answer works but it seems to be a one shot; on my system it stops running after 1 time. I changed the code to use a while loop instead; now it's working on my system:
def barn():
p = multiprocessing.Process(target=bar)
p.start()
i = 0
while(t > i):
print("running for " + str(i))
i += 1
time.sleep(1)
if i == t:
print('restarting')
p = multiprocessing.Process(target=bar)
p.terminate
p = multiprocessing.Process(target=bar)
p.start()
i = 0
Hey I am trying to have a loop be pausable from user input like having a input box in the terminal that if you type pause it will pause the loop and then if you type start it will start again.
while True:
#Do something
pause = input('Pause or play:')
if pause == 'Pause':
#Paused
Something like this but having the #Do something continually happening without waiting for the input to be sent.
Ok I get it now, here is a solution with Threads:
from threading import Thread
import time
paused = "play"
def loop():
global paused
while not (paused == "pause"):
print("do some")
time.sleep(3)
def interrupt():
global paused
paused = input('pause or play:')
if __name__ == "__main__":
thread2 = Thread(target = interrupt, args = [])
thread = Thread(target = loop, args = [])
thread.start()
thread2.start()
You can't directly, as input blocks everything until it returns.
The _thread module, though, can help you with that:
import _thread
def input_thread(checker):
while True:
text = input()
if text == 'Pause':
checker.append(True)
break
else:
print('Unknown input: "{}"'.format(text))
def do_stuff():
checker = []
_thread.start_new_thread(input_thread, (checker,))
counter = 0
while not checker:
counter += 1
return counter
print(do_stuff())
This question already has answers here:
Is there any way to kill a Thread?
(31 answers)
Closed 6 years ago.
i've got this code, how can I stop func2 from func1? something like Thread(target = func1).stop() doesn't work
import threading
from threading import Thread
def func1():
while True:
print 'working 1'
def func2():
while True:
print 'Working2'
if __name__ == '__main__':
Thread(target = func1).start()
Thread(target = func2).start()
It's better to ask your other thread to stop, using a message queue for instance.
import time
import threading
from threading import Thread
import Queue
q = Queue.Queue()
def func1():
while True:
try:
item = q.get(True, 1)
if item == 'quit':
print 'quitting'
break
except:
pass
print 'working 1'
def func2():
time.sleep(10)
q.put("quit")
while True:
time.sleep(1)
print 'Working2'
if __name__ == '__main__':
Thread(target = func1).start()
Thread(target = func2).start()
You cannot tell a thread to stop, you have to make it return in its target function
from threading import Thread
import Queue
q = Queue.Queue()
def thread_func():
while True:
# checking if done
try:
item = q.get(False)
if item == 'stop':
break # or return
except Queue.Empty:
pass
print 'working 1'
def stop():
q.put('stop')
if __name__ == '__main__':
Thread(target=thread_func).start()
# so some stuff
...
stop() # here you tell your thread to stop
# it will stop the next time it passes at (checking if done)