On windows, I need to stop a python method execution after a specific time using Python 3.7 version. I tried with multiprocessing.Process but it is not able to call the method.
from multiprocessing import Process
import time
def func1(name):
print("Hi from func1")
i = 0
while True:
i += 1
print('hello', name)
time.sleep(1)
if __name__ == '__main__':
p = Process(target=func1, args=('bob',))
p.start()
time.sleep(5)
p.terminate()
In the above code, it is not able to invoke the func1 method. Please let me know if I am missing something or is there any better way to achieve this.
Related
When I run this and input something it goes into the main function but then again asks for input. Why is that even happening?
I am running using command prompt in windows. version is 3.8
import multiprocessing
from concurrent.futures import ProcessPoolExecutor
import concurrent.futures
input('?')
def pp(id,lock):
with lock:
for i in range(5):
print(f'{id}=>{i}')
def main():
pool = ProcessPoolExecutor()
m = multiprocessing.Manager()
lock = m.Lock()
futures = [pool.submit(pp, num,lock) for num in range(10)]
with concurrent.futures.ThreadPoolExecutor(max_workers=100) as executor:
executor.map(main, list(range(10)),[lock]*10)
if __name__=='__main__':
main()
Here is the output:
?abc
?abd
????
How to solve this problem so it runs the input just once?
I cannot reproduce, it only runs once on my local Python.
What is your Python version ?
However, I can recommend putting input inside the if __name__ == "main". The problem is that your input is called whenever you import your module, which could be done by a Thread when importing the main function.
Note: sorry to not post a comment, but I can't with a lower reputation than 50.
On a machine running Windows Server 2012 R2, in the Spyder IDE from Anaconda and running Python 3.7 with the following code:
import time
import multiprocessing
start=time.perf_counter()
def do_something():
print('func start')
time.sleep(1)
print('func end')
if __name__=='__main__':
print('name is main')
p1=multiprocessing.Process(target=do_something)
p1.start()
p1.join()
finish=time.perf_counter()
print('\n\nProgram completed in '+str(round((finish-start),2))+'s')
print('Goodbye!')
And I get the output
name is main
Program completed in 0.13s
Goodbye!
My expectation was that I would see the two print statements
func start
func end
and also (because .join was envoked) that the program would take >1s to complete.
I suspect that the .start() call did not successfully call the do_something function.
FYI, I am following this tutorial, which I know needs to be modified to include the if statement on windows. I am also seeing similar code on other sites, but doesn't seem to work on my end.
Any suggestions on troubleshooting would be much appreciated.
**EDIT: Per the comment below from Azy_Crw4282, the code seems to work on his end and, per his suggestion, it seems to work from the cmd prompt. So this seems to be a bug specifically with the Spyder IDE.
FYI, I wanted to understand whether the issue was that the process was getting kicked off but the IDE wasn't capturing the output OR the process wasn't getting kicked off. I tried two things, 1) the code below writes a dataframe to csv. When doing this in the multiprocessing function, it does NOT write the file. 2) I created a global variable and changed variable value in the function. Spyder keeps the variable values after the code runs, and when I printed the variable it was unchanged.
So, in summary - it seems that the Spyder IDE does not work with the multiprocessing module.**
import time
import multiprocessing
start=time.perf_counter()
df=pd.DataFrame(data={'Col1':[1.1,2.1,3.1],
'Col2':[1.2,2.2,3.2],
'Col3':[1.3,2.3,3.3]}, columns=['Col1','Col2','Col3'])
def do_something():
print('func start')
df.to_csv('C:/testMp.csv')
time.sleep(1)
print('func end')
if __name__=='__main__':
print('name is main')
p1=multiprocessing.Process(target=do_something)
p1.start()
p1.join()
finish=time.perf_counter()
print('\n\nProgram completed in '+str(round((finish-start),2))+'s')
print('Goodbye!')
When I ran your code, I get the following output. Can you try running your code in another ide/ cmd line/ terminal?
import multiprocessing
start=time.perf_counter()
def do_something():
print('func start')
time.sleep(1)
print('func end')
if __name__=='__main__':
print('name is main')
p1=multiprocessing.Process(target=do_something)
p1.start()
p1.join()
finish=time.perf_counter()
print('\n\nProgram completed in '+str(round((finish-start),2))+'s')
print('Goodbye!')
Outputs:
name is main
Program completed in 0.0s
Goodbye!
func start
func end
Program completed in 1.27s
Goodbye!
The above result is not probably what you expect. In order to achieve, what you want, you need to indent the outer print section so that it appears within the main call.
import time
import multiprocessing
start=time.perf_counter()
def do_something():
print('func start')
time.sleep(1)
print('func end')
if __name__=='__main__':
print('name is main')
p1=multiprocessing.Process(target=do_something)
p1.start()
p1.join()
finish=time.perf_counter()
print('\n\nProgram completed in '+str(round((finish-start),2))+'s')
print('Goodbye!')
Outputs:
name is main
func start
func end
Program completed in 1.33s
Goodbye!
Python v3.5, Windows 10
I'm using multiple processes and trying to captures user input. Searching everything I see there are odd things that happen when using input() with multiple processes. After 8 hours+ of trying, nothing I implement worked, I'm positive I am doing it wrong but I can't for the life of me figure it out.
The following is a very stripped down program that demonstrates the issue. Now it works fine when I run this program within PyCharm, but when I use pyinstaller to create a single executable it fails. The program constantly is stuck in a loop asking the user to enter something as shown below:.
I am pretty sure it has to do with how Windows takes in standard input from things I've read. I've also tried passing the user input variables as Queue() items to the functions but the same issue. I read you should put input() in the main python process so I did that under if __name__ = '__main__':
from multiprocessing import Process
import time
def func_1(duration_1):
while duration_1 >= 0:
time.sleep(1)
print('Duration_1: %d %s' % (duration_1, 's'))
duration_1 -= 1
def func_2(duration_2):
while duration_2 >= 0:
time.sleep(1)
print('Duration_2: %d %s' % (duration_2, 's'))
duration_2 -= 1
if __name__ == '__main__':
# func_1 user input
while True:
duration_1 = input('Enter a positive integer.')
if duration_1.isdigit():
duration_1 = int(duration_1)
break
else:
print('**Only positive integers accepted**')
continue
# func_2 user input
while True:
duration_2 = input('Enter a positive integer.')
if duration_2.isdigit():
duration_2 = int(duration_2)
break
else:
print('**Only positive integers accepted**')
continue
p1 = Process(target=func_1, args=(duration_1,))
p2 = Process(target=func_2, args=(duration_2,))
p1.start()
p2.start()
p1.join()
p2.join()
You need to use multiprocessing.freeze_support() when you produce a Windows executable with PyInstaller.
Straight out from the docs:
multiprocessing.freeze_support()
Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.)
One needs to call this function straight after the if name == 'main' line of the main module. For example:
from multiprocessing import Process, freeze_support
def f():
print('hello world!')
if __name__ == '__main__':
freeze_support()
Process(target=f).start()
If the freeze_support() line is omitted then trying to run the frozen executable will raise RuntimeError.
Calling freeze_support() has no effect when invoked on any operating system other than Windows. In addition, if the module is being run normally by the Python interpreter on Windows (the program has not been frozen), then freeze_support() has no effect.
In your example you also have unnecessary code duplication you should tackle.
Python v3.5, Windows 10
I'm using multiple processes and trying to captures user input. Searching everything I see there are odd things that happen when using input() with multiple processes. After 8 hours+ of trying, nothing I implement worked, I'm positive I am doing it wrong but I can't for the life of me figure it out.
The following is a very stripped down program that demonstrates the issue. Now it works fine when I run this program within PyCharm, but when I use pyinstaller to create a single executable it fails. The program constantly is stuck in a loop asking the user to enter something as shown below:.
I am pretty sure it has to do with how Windows takes in standard input from things I've read. I've also tried passing the user input variables as Queue() items to the functions but the same issue. I read you should put input() in the main python process so I did that under if __name__ = '__main__':
from multiprocessing import Process
import time
def func_1(duration_1):
while duration_1 >= 0:
time.sleep(1)
print('Duration_1: %d %s' % (duration_1, 's'))
duration_1 -= 1
def func_2(duration_2):
while duration_2 >= 0:
time.sleep(1)
print('Duration_2: %d %s' % (duration_2, 's'))
duration_2 -= 1
if __name__ == '__main__':
# func_1 user input
while True:
duration_1 = input('Enter a positive integer.')
if duration_1.isdigit():
duration_1 = int(duration_1)
break
else:
print('**Only positive integers accepted**')
continue
# func_2 user input
while True:
duration_2 = input('Enter a positive integer.')
if duration_2.isdigit():
duration_2 = int(duration_2)
break
else:
print('**Only positive integers accepted**')
continue
p1 = Process(target=func_1, args=(duration_1,))
p2 = Process(target=func_2, args=(duration_2,))
p1.start()
p2.start()
p1.join()
p2.join()
You need to use multiprocessing.freeze_support() when you produce a Windows executable with PyInstaller.
Straight out from the docs:
multiprocessing.freeze_support()
Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.)
One needs to call this function straight after the if name == 'main' line of the main module. For example:
from multiprocessing import Process, freeze_support
def f():
print('hello world!')
if __name__ == '__main__':
freeze_support()
Process(target=f).start()
If the freeze_support() line is omitted then trying to run the frozen executable will raise RuntimeError.
Calling freeze_support() has no effect when invoked on any operating system other than Windows. In addition, if the module is being run normally by the Python interpreter on Windows (the program has not been frozen), then freeze_support() has no effect.
In your example you also have unnecessary code duplication you should tackle.
for some reason the code does not get to the main function. I am using cloud9 to run the code so that might be the issue.
from multiprocessing import Process, Value
import time
def main():
print "main function"
def market_price_thread():
while True:
market_price()
time.sleep(5)
def market_price():
#do something
print "end"
def start_threads():
thread = Process(target=market_price_thread())
thread.start()
time.sleep(5)
if __name__ == '__main__':
start_threads()
main() #does not seem to get to this
You've asked Python to call market_price_thread:
thread = Process(target=market_price_thread())
and then use whatever it returns, as the target value. So, before calling Process, we'll have to wait for market_price_thread to return. What value does it return, and when?
(Compare with Process(target=market_price_thread), which does not call market_price_thread yet, but rather, passes the function to Process so that Process can call it.)