Multiprocess job is running the tasks, I want to stop the rest of the parallel or dependent tasks if one of them fails or completes all the tasks.
The problem is with 1st print, where it should check if job failed with non-zero exit code and already not completed then enter the loop and stop the rest of the jobs by breaking the while loop.
however, even the execution completed successfully with exit 0, it enters the loop intermittently, stops the rest of the jobs by breaking the loop.
What is going wrong here.
Failed one
enter image description here
Passed one
enter image description here
Main job triggering multiprocess tasks.
def run_block(index):
print index
# do some execution
def run_blocks(target, dict_blocks):
process = []
for (index, (block_id, depend_on)) in \
enumerate(dict_blocks.items()):
proc = multiprocessing.Process(target=run_block, args=index)
process.append(proc)
proc.start()
check_exit(process)
def check_exit(process):
done = False
process_count = len(process)
count = 0
completed = []
while not done:
for proc in process:
if proc.exitcode != 0 and proc.exitcode != None:
print ('1st', proc, count, done, proc.exitcode)
done = True
break
if proc.exitcode == 0 and proc.pid not in completed:
print ('2nd', proc, count, done, proc.exitcode)
completed.append(proc.pid)
count += 1
if count == process_count:
print ('3rd', proc, count, done)
done = True
break
stop_process_exit(process, count, process_count, done)
def stop_process_exit(
process,
count,
process_count,
done,
):
print (process_count, count, done, process)
for proc in process:
if proc.is_alive():
proc.terminate()
if done == True and count != process_count:
exit(1)
Your processes are running independently, so the variable proc.exitcode must be dynamic. In other words, it might change at any moment because the process has just finished. In this statement:
if proc.exitcode != 0 and proc.exitcode != None
you access the variable twice. Suppose proc.exitcode is None when you begin to execute this line. Python does the first comparison and it evaluates True. Now suppose that the process finishes at that exact moment, and now proc.exitcode becomes zero. Python performs the second comparison, and now that is also True! So your print statement fires, and then you break out of the loop when you really don't want to.
Of course I don't know this is what's happening since I can't run your program, but the evidence points that way.
I would change the loop like this:
for proc in process:
if proc.is_alive():
continue
if proc.exitcode != 0:
print ('1st', proc, count, done, proc.exitcode)
done = True
break
# ... everything else is not changed
Related
I'm new to python, this below funtion is not getting stop at if condition where it needs to stop.
def check_pi_installation_status():
proc1cmd = "grep " + AppName
p1 = subprocess.Popen(['kubectl', 'get', 'pi', '-A'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(proc1cmd, shell=True, stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p1.stdout.close()
stdout_list = p2.communicate()[0]
stdout_list = stdout_list.decode()
imagenamepi = stdout_list.split()
print("pi imagename ", imagenamepi)
if (imagenamepi[2] == 'Installed'):
print("Successfully Installed")
return
if (imagenamepi[2] == 'Failed'):
print(" Installation Failed")
return
if(imagenamepi[2] != 'Installed' and imagenamepi[2] != 'Failed'):
time.sleep(30)
# count = count + 1
# print("count", count)
write_eventslog()
check_pi_installation_status()
Below are the case:
when i started this function where its still not reached to 'Failed',it prints 'Installation Failed' on reaching state 'Failed' but it continues even when it reaches 'Failed' state.
I ran the function when state is already 'Failed' its works as expected (it prints 'Installation Failed' and come out.
Instead of using multiple if statements, why not use if, elif and else? The main difference here is that when you have multiple if statements, all of them will run no matter what, so change it to this:
if (imagenamepi[2] == 'Installed'):
print("Successfully Installed")
return
elif (imagenamepi[2] == 'Failed'):
print(" Installation Failed")
return
else:
time.sleep(30)
# count = count + 1
# print("count", count)
write_eventslog()
check_pi_installation_status()
Over here, we first used if, then elif. elif is a mixture between IF and ELSE, which basically tells the compiler: If the statement above did not run, check this one. You can have multiple elif's in one go. Lastly, we have else. Else tells the compiler: If none of the other statements ran, run this one. Unlike Elif, you can only have one Else condition.
The main difference between the two methods are that when you use If, Elif and Else, the entire statement is exited when a condition is run.
Although im not too sure why return doesnt exit out the function immediately, the use of elif and else would help you do the same thing. More than likely, it works on the fail state because none of the other two statements are applicable.
I have a simple Python/Selenium script that has to wait for 4 checks to happen. Each of these on a separated thread. On the first thread that returns information, the program should continue and the remaining threads killed (if possible).
I have a program that is creating the threads just fine, but I have no idea on how to continue on the first that have executed.
these are my function definitions:
def wait_by(search_type, locator, timeout):
WebDriverWait(driver, timeout).until(EC.presence_of_element_located((search_type, locator)))
def multi_locator_verify_element(xpath_locator, css_locator, name_locator, id_locator, timeout):
element = ""
if xpath_locator != "":
tx = Thread(target=wait_by, args=(By.XPATH, xpath_locator, timeout))
tx.start()
if css_locator != "":
tc = Thread(target=wait_by, args=(By.CSS, css_locator, timeout))
tc.start()
if name_locator != "":
tn = Thread(target=wait_by, args=(By.NAME, name_locator, timeout))
tn.start()
if id_locator != "":
ti = Thread(target=wait_by, args=(By.ID, id_locator, timeout))
ti.start()
The multi_locator_verify_element function should finish after the first thread has been able to retrive the right element.
Could someone please help me with that?
Regards,
Using the script below, I cannot seem to exit the threads. The script runs smoothly without issues but never exits when done. I can still see the thread alive, I have to use htop to kill them or completely exit the command line.
How can I get this script to exit and the threads to die?
def async_dns():
s = adns.init()
while True:
dname = q.get()
response = s.synchronous(dname,adns.rr.NS)[0]
if response == 0:
dot_net.append("Y")
print(dname + ", is Y")
elif response == 300 or response == 30 or response == 60:
dot_net.append("N")
print(dname + ", is N")
elif q.empty() == True:
q.task_done()
q = queue.Queue()
threads = []
for i in range(20):
t = threading.Thread(target=async_dns)
threads.append(t)
t.start()
for name in names:
q.put_nowait(name)
Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. Otherwise (block is false), return an item if one is immediately available, else raise the Empty exception (timeout is ignored in that case).
Remember to check your queue.
See the document of queue.
Your threads are stuck in dname = q.get()
If you reaching empty queue, q.get() will wait forever for value to arrive.
You can replace get with get_nowait() but get ready to catch Queue.Empty execption
I'm new to Python and have been googling for a couple of days and read all I can find on this forum. Might be that I don't understand it all but I haven't found a solution to my problem yet. Ask for forgiveness already if there's an answer already to my problem, then I haven't understood it.
I want to make a Pause function for my program Tennismatch. The program will when it's being run print the score of a tennis match like this: "15-0, 15-15 etc ongoing till the match ends. It will print the score line by line.
I want the user to be able to pause after x number of balls, games, etc. So I don't know when the user wants to pause and after the user has paused I want the user to be able to resume the tennismatch where it was.
Have seen the time.sleep() but as I have understood it you must know when you want to pause to use this and it also ain't an indefinetie pause like I want. With input() it's the same.
Am going to make a GUI later on when the code is finished. Happy for anything that leads me to solving my problem.
I use Windows and Python 3.42 and run the program in Shell.
A piece of the code (haven't written it all yet, it's more of a general situation when something is being printed line after line for some time and want to be able do pause in the CIL:
#self.__points = [0,0]
def playGame(self):
if self.server == True: #self.server is either True or False when someone calls playGame()
server = self.player_1.get_win_serve() #self.player_1 = an object of a class Player():
else:
server = self.player_2.get_win_serve() #get_win_serve() method returns the probability to win his serv (1-0)
while (0 < self.__points[0] - self.__points[1] >= 2 or 0 < self.__points[1] - self.__points[0] >= 2) and (self.__points[1] >= 4 or self.__points[0] >= 4):
x = random.uniform(0,1)
if x > 0 and x < server:
self.__points[0] += 1
else:
self.__points[1] += 1
# print('The score, by calling a score() function that I haven't written yet')
For dealing with events in main loop you need to make a separated thread which capture input or any other event.
import sys
from sys import stdin
from time import sleep
from threading import Thread
from Queue import Queue, Empty
def do_something():
sleep(1)
print 42
def enqueue_output(queue):
while True:
# reading line from stdin and pushing to shared queue
input = stdin.readline()
print "got input ", input
queue.put(input)
queue = Queue()
t = Thread(target=enqueue_output, args=(queue,))
t.daemon = True
t.start()
pause = False
try:
while True:
try:
command = queue.get_nowait().strip()
print 'got from queue ', command
except Empty:
print "queue is empty"
command = None
if command:
if command == 'p':
pause = True
if command == 'u':
pause = False
if not pause:
print pause
do_something()
except KeyboardInterrupt:
sys.exit(0)
I came up with the following.
while True:
try:
## Keep doing something here
## your regular code
print '.',
except KeyboardInterrupt:
## write or call pause function which could be time.sleep()
print '\nPausing... (Hit ENTER to continue, type quit to exit.)'
try:
response = raw_input()
if response.lower() == 'quit':
break
print 'Quitting...'
except KeyboardInterrupt:
print 'Resuming...'
continue
The Event loop might as well be the code I wrote with.
I don't see any user input so I assume that x emulates it. To pause the game if x < 0.1 and to unpause(/resume) it if x > 0.9, you could:
while your_condition(self.__points):
x = random.random()
if x < 0.1: # pause
self.pause()
elif x > 0.9: # resume
self.resume()
if self.is_paused:
continue # do nothing else only wait for input (`x`)
# assume your_condition() has no side-effects
# here's what the resumed version does:
print("...")
# change self.__points, etc
where pause(), resume(), is_paused() methods could be implemented as:
def __init__(self):
self.is_paused = False
def pause(self):
self.is_paused = True
def resume(self):
self.is_paused = False
as you can see the implementation is very simple.
I am trying to create a program that runs an infinite loop in parallel and exits the loops when it is told too. Specifically, the infinite loop is in the square function and the exiting signal is given when shv='STOP'. When all processes read that signal will have to exit the infinite loop and return.
The problem is that the Processes do not close even after giving the STOP signal.
Some notes:
As many instances of multiprocessing code, this code runs in the terminal rather than in IDEs.
The code:
import multiprocessing as mp
import time
import ctypes
def square(x, shv):
while shv.value != 'STOP':
time.sleep(3)
print(shv.value)
else:
print('stopped')
return
if __name__ == '__main__':
stopphrase = 'STOP'
proecess_num = 2
shv = mp.Value(ctypes.c_wchar_p, '')
processes = [mp.Process(target=square, args=(i, shv)) for i in range(proecess_num)]
for p in processes:
p.start()
print('Mapped & Started')
print(processes)
while shv.value != stopphrase:
inp = input('Type STOP and press Enter to terminate: ')
if inp == stopphrase:
shv.value = stopphrase
time.sleep(2)
p.terminate()
print(processes)
For some reason this code gives the following in both cases of print(processes) even though I set the shv.value = stopphrase:
[<Process name='Process-1' pid=9664 parent=6084 started>, <Process name='Process -2' pid=10052 parent=6084 started>]
Please let me know for further improvements or details of the question.
I think you meant to have a loop calling ‘join()’ on each process.
for p in processes:
p.join()
instead of calling terminate on just one of them.