I am trying to create a program that runs an infinite loop in parallel and exits the loops when it is told too. Specifically, the infinite loop is in the square function and the exiting signal is given when shv='STOP'. When all processes read that signal will have to exit the infinite loop and return.
The problem is that the Processes do not close even after giving the STOP signal.
Some notes:
As many instances of multiprocessing code, this code runs in the terminal rather than in IDEs.
The code:
import multiprocessing as mp
import time
import ctypes
def square(x, shv):
while shv.value != 'STOP':
time.sleep(3)
print(shv.value)
else:
print('stopped')
return
if __name__ == '__main__':
stopphrase = 'STOP'
proecess_num = 2
shv = mp.Value(ctypes.c_wchar_p, '')
processes = [mp.Process(target=square, args=(i, shv)) for i in range(proecess_num)]
for p in processes:
p.start()
print('Mapped & Started')
print(processes)
while shv.value != stopphrase:
inp = input('Type STOP and press Enter to terminate: ')
if inp == stopphrase:
shv.value = stopphrase
time.sleep(2)
p.terminate()
print(processes)
For some reason this code gives the following in both cases of print(processes) even though I set the shv.value = stopphrase:
[<Process name='Process-1' pid=9664 parent=6084 started>, <Process name='Process -2' pid=10052 parent=6084 started>]
Please let me know for further improvements or details of the question.
I think you meant to have a loop calling ‘join()’ on each process.
for p in processes:
p.join()
instead of calling terminate on just one of them.
Related
I'm looking to terminate a thread after a certain amount of time. The thread is called from an infinite while loop that waits for users input. Thread starts after user inputs number 4 from command line, after which thread writes some text to the console in another infinite loop. After x amount of time the started thread should terminate. How can I make the thread stop after the set time while keeping the main thread with user input responsive (active)?
Here is some sample code:
def loop():
while True:
print("ON")
print("OFF")
if __name__ == "__main__":
t = threading.Thread(target=loop)
try:
while True:
print(">>", end='')
command = int(input())
if command == 1:
...
elif command == 2:
...
elif command == 3:
...
elif command == 4:
print("Insert time:")
time = float(input())
t.start()
elif command == 5:
...
It is possible to pass an argument to the loop and that can be used to stop the thread, something like this:
def loop(running):
while running[0]:
print("HELLO")
if __name__ == "__main__":
running = [True]
t = threading.Thread(target=loop, args=(running,))
t.start()
while True:
command = input()
running[0] = False
running is shared by both the main thread and the other thread, so changing the value in one changes it in the other.
i used multiprocessing but i don't know how to do it
the logic : a variable sign is equal to 0, with a function called timer count 20 seconds and each second check if sign is equal to 1 then it'll print something and breaks the loop, at the same time with a function called waiting waits for an input from another library as example "discord" or "socket" so if the input is equal to my key flip the variable sign to 1 and that affects the first function timer
import multiprocessing
from time import sleep
sign = 0
def timer():
for s in range(20):
if sign == 1: # if the input is equal to the secret key then break the timer
print("Yes!")
break
else:
sleep(1) #if not then continue timing
def waiting():
# maybe it waits for an input or a message from "discord or whatsapp"
if message == "secret_key":
sign = 1
p1 = multiprocessing.Process(target=timer)
p2 = multiprocessing.Process(target=waiting)
p1.start()
p2.start()
I mentioned it above in a comment, but here is how you would use an event
import time
import multiprocessing as mp
def timer(exit_event):
for s in range(20):
if exit_event.is_set():
print("Yes!")
break
else:
time.sleep(1) #if not then continue timing
def waiting(exit_event):
# maybe it waits for an input or a message from "discord or whatsapp"
time.sleep(5)
exit_event.set()
if __name__ == '__main__':
exit_event = mp.Event()
p1 = mp.Process(target=timer, args=(exit_event,))
p2 = mp.Process(target=waiting, args=(exit_event,))
p1.start()
p2.start()
p1.join()
p2.join()
However the real way to use an Event is to just to wait() for it to become true. No need for a sleep loop.
def timer(exit_event):
if exit_event.wait(timeout=20)
# returns True if the event is True. False if timed out
print("Yes!")
Multiprocess job is running the tasks, I want to stop the rest of the parallel or dependent tasks if one of them fails or completes all the tasks.
The problem is with 1st print, where it should check if job failed with non-zero exit code and already not completed then enter the loop and stop the rest of the jobs by breaking the while loop.
however, even the execution completed successfully with exit 0, it enters the loop intermittently, stops the rest of the jobs by breaking the loop.
What is going wrong here.
Failed one
enter image description here
Passed one
enter image description here
Main job triggering multiprocess tasks.
def run_block(index):
print index
# do some execution
def run_blocks(target, dict_blocks):
process = []
for (index, (block_id, depend_on)) in \
enumerate(dict_blocks.items()):
proc = multiprocessing.Process(target=run_block, args=index)
process.append(proc)
proc.start()
check_exit(process)
def check_exit(process):
done = False
process_count = len(process)
count = 0
completed = []
while not done:
for proc in process:
if proc.exitcode != 0 and proc.exitcode != None:
print ('1st', proc, count, done, proc.exitcode)
done = True
break
if proc.exitcode == 0 and proc.pid not in completed:
print ('2nd', proc, count, done, proc.exitcode)
completed.append(proc.pid)
count += 1
if count == process_count:
print ('3rd', proc, count, done)
done = True
break
stop_process_exit(process, count, process_count, done)
def stop_process_exit(
process,
count,
process_count,
done,
):
print (process_count, count, done, process)
for proc in process:
if proc.is_alive():
proc.terminate()
if done == True and count != process_count:
exit(1)
Your processes are running independently, so the variable proc.exitcode must be dynamic. In other words, it might change at any moment because the process has just finished. In this statement:
if proc.exitcode != 0 and proc.exitcode != None
you access the variable twice. Suppose proc.exitcode is None when you begin to execute this line. Python does the first comparison and it evaluates True. Now suppose that the process finishes at that exact moment, and now proc.exitcode becomes zero. Python performs the second comparison, and now that is also True! So your print statement fires, and then you break out of the loop when you really don't want to.
Of course I don't know this is what's happening since I can't run your program, but the evidence points that way.
I would change the loop like this:
for proc in process:
if proc.is_alive():
continue
if proc.exitcode != 0:
print ('1st', proc, count, done, proc.exitcode)
done = True
break
# ... everything else is not changed
So, i would claim that i understand how Asyncio, Multiprocessing, Threading etc. works, basically. I know how to listen for keystrokes too - there are many good examples on this site.
However i was unable to combine both into one. I have a programm that runs continously in a loop, until it runs into certain cases where it stops. In these cases, it uses a Multiprocessing.Queue() to prompt for user input on wether it should continue or not.
All of this works, so far so good. Now i want to add a second catch case here: The programm should, once it starts running, immediatly cease working as soon as i press a certain button (lets say Escape).
This is the very dumbed down version of my programm:
test.py:
from test3 import Counter
from multiprocessing import Process, Queue
import sys
def main(q, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p1.join()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, user_Input)
else:
sys.exit()
if __name__ == "__main__":
q = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, passed_variable=user_Input)
test3.py:
import time
class Counter:
def counting(self, q, user_input):
x = 0
while True:
print(str(x) + " " + user_input)
if x == 4:
q.put(False)
break
time.sleep(1)
x += 1
I tried everything i could think of, in no case did i get the desired result, and no question i found here was able to help me in this specific case.
You can solve this using keyboard and then creating a second Queue():
from test3 import Counter
from multiprocessing import Process, Queue
import sys
import keyboard
def main(q, queue2, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p2 = Process(target=keyCatcher, args=(queue2,))
p2.start()
if queue2.get() == False:
p1.terminate()
print("Terminating Programm")
sys.exit()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, queue2, user_Input)
else:
sys.exit()
def keyCatcher(queue2):
while True:
if keyboard.is_pressed('q'): # if key 'q' is pressed
queue2.put(False)
if __name__ == "__main__":
q = Queue()
queue2 = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, queue2, passed_variable=user_Input)
The crux is:
p1.start()
p1.join()
Which means after main() starts p1, it waits for it to finish. So there's no chance to interrupt it while processing.
You need to:
wait for p1 to finish
while waiting, see if the main process gets a 'q'
if the main process gets a 'q', stop it.
Something like:
p1.start()
while p1.is_alive():
k = keyboard.read_key()
if k == 'q':
p1.terminate()
I created threads, added delay in the function, but all threads are executing at the same time. Instead i want threads to start one by one. Is that possible ?
Below is my code
from _thread import start_new_thread
import time
def mul(n):
time.sleep(1)
res = n * n
return res
while 1:
m = input("Enter number ")
t = input("Enter the number of times the function should be executed:")
max_threads = int(t)
for n in range(0, max_threads):
start_new_thread(mul, (m,))
except:
pass
print("Please type only digits (0-9)")
continue
print(f"Started {max_threads} threads.")
First of all, you added the delay inside the thread, causing it to pause after it started. Thus, you are starting all the threads one by one without delay and when each thread starts it waits 1 second before continuing.
So if you want a specific delay - add after you start each thread, in the main thread.
If you want each thread to start after the previous thread finished, you can do the following:
import threading
.
.
.
for n in range(0, max_threads):
t = threading.Thread(target = mul, args=(m,))
t.start()
t.join() # Waits until it is finished