We Keep Coding

Print in real time without waiting for the script to complete in python

I want to run external script(demo_print.py) and print the output in real time.
But it doesn't work for me. The program wait until the script end print all output.
My goal is Print in real time without waiting for the script to complete. How to fix it ?
0
1
2
....
9
demo_print.py
import time
for i in range(10):
print(f"print {i}")
time.sleep(1)
main.py
def run_command(command):
process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
while True:
output = process.stdout.readline()
if output == '' and process.poll() is not None:
break
if output:
print(output.decode('UTF-8').strip())
rc = process.poll()
return rc
def main(argv):
cmd = "demo_print.py"
run_command(cmd)
if __name__ == '__main__':
main(sys.argv)

Why Python console hangs when printed text is highlighted

I'm facing a problem with Python and Threads: I have a daemon thread that continuously prints "TEST" in the console, every 5 seconds.
When the console is open, and I try to highlight a few lines of text, the thread stops and it doesn't continue until I press a key on the keyboard.
Here is my code:
import threading
import time
def thread_function():
t = threading.current_thread()
while getattr(t, 'do_run', True):
print('TEST')
time.sleep(5)
def main():
x = threading.Thread(target = thread_function)
x.daemon = True
x.start()
time.sleep(1000)
x.do_run = False
x.join()
if __name__ == '__main__':
main()
Which is the cause of this behavior?

Python - creating a simple killswitch for a function that runs with multiprocessing

So, i would claim that i understand how Asyncio, Multiprocessing, Threading etc. works, basically. I know how to listen for keystrokes too - there are many good examples on this site.
However i was unable to combine both into one. I have a programm that runs continously in a loop, until it runs into certain cases where it stops. In these cases, it uses a Multiprocessing.Queue() to prompt for user input on wether it should continue or not.
All of this works, so far so good. Now i want to add a second catch case here: The programm should, once it starts running, immediatly cease working as soon as i press a certain button (lets say Escape).
This is the very dumbed down version of my programm:
test.py:
from test3 import Counter
from multiprocessing import Process, Queue
import sys
def main(q, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p1.join()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, user_Input)
else:
sys.exit()
if __name__ == "__main__":
q = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, passed_variable=user_Input)
test3.py:
import time
class Counter:
def counting(self, q, user_input):
x = 0
while True:
print(str(x) + " " + user_input)
if x == 4:
q.put(False)
break
time.sleep(1)
x += 1
I tried everything i could think of, in no case did i get the desired result, and no question i found here was able to help me in this specific case.

You can solve this using keyboard and then creating a second Queue():
from test3 import Counter
from multiprocessing import Process, Queue
import sys
import keyboard
def main(q, queue2, passed_variable):
foo = Counter()
p1 = Process(target=foo.counting, args=(q,passed_variable))
p1.start()
p2 = Process(target=keyCatcher, args=(queue2,))
p2.start()
if queue2.get() == False:
p1.terminate()
print("Terminating Programm")
sys.exit()
if q.get() == False:
x = input("Keep going?")
print(x)
if x == "y":
main(q, queue2, user_Input)
else:
sys.exit()
def keyCatcher(queue2):
while True:
if keyboard.is_pressed('q'): # if key 'q' is pressed
queue2.put(False)
if __name__ == "__main__":
q = Queue()
queue2 = Queue()
user_Input = ("What you want from me, man?")
print("Starting")
main(q, queue2, passed_variable=user_Input)

The crux is:
p1.start()
p1.join()
Which means after main() starts p1, it waits for it to finish. So there's no chance to interrupt it while processing.
You need to:
wait for p1 to finish
while waiting, see if the main process gets a 'q'
if the main process gets a 'q', stop it.
Something like:
p1.start()
while p1.is_alive():
k = keyboard.read_key()
if k == 'q':
p1.terminate()

What is the best way to stop a loop from outside it?

I have a python script that prints a spinner. This spinner, hopefully, will last until stopped.
spinnerFrames = [
"/",
"-",
"\\",
"|",
]
def spinner():
i = 0
while True:
clearScreen() #function to clear the screen
print(spinnerFrames[i])
i = i + 1
if (i == 3):
i = 0
sleep(0.15)
spinner()
sleep(3)
# break out here
print("Done!")
I know you can do sys.stdout.write() and then only delete that line, but that's beside the point.
I can't figure out the best way to stop the loop and exit the function. (To continue on in my code)I'd like to be able to break from the loop down where you call it, as I hope to make this a Pip package.
This, I assume is possible, though I don't know how to do it. Thanks for your help!

You need to run it asynchronously, like how the multiprocessing library allows you to do. When you create a separate thread, you'll be left with a handle on it that you can use to kill it when you want it to stop.
from multiprocessing import Process
from time import sleep
spinnerFrames = [
"/",
"-",
"\\",
"|",
]
def spinner():
i = 0
while True:
print(spinnerFrames[i], end='\r')
i = i + 1
if (i == 3):
i = 0
sleep(0.15)
if __name__ == '__main__':
p = Process(target=spinner)
p.start()
sleep(3)
p.terminate()
print("Done!")

Here is a reference implementation from one of my projects. It prints dots instead of a spinner, but it is trivial to change:
import threading
import time
def indicate_wait(func):
active = threading.Lock()
def dot_printer():
while active.locked():
print('.', end='', flush=True)
time.sleep(1)
def wrapper(*args, **kwargs):
t = threading.Thread(target=dot_printer)
active.acquire()
t.start()
res = func(*args, **kwargs)
active.release()
return res
return wrapper
Example:
#indicate_wait
def test():
time.sleep(5)

record when it started, then break loop if current time - start time >
duration.
import time
spinnerFrames = [
"/",
"-",
"\\",
"|",
]
def spinner():
i = 0
startTime = time.time()# record starting time
duration = 3
while True:
clearScreen() #function to clear the screen
print(spinnerFrames[i])
i = i + 1
if (i == 3):
i = 0
if time.time() - startTime > duration:
break
sleep(0.15)
spinner()
print("Done!")

Why are my multiprocesses not starting without time.sleep in main?

I have some background task that i want to start an be able to safely quit by user input.
To do that i have a thread in which a process pool with a continous task is started. There is a input lock to stop printing and to wait for user input and a event to stop the whole process.
What surprises me is that the processes seem to start and do their work if there is a time.sleep after the start of the thread with processpool (6th line in main).
import multiprocessing as mp
import time
import threading as tr
def init(e, l):
global stop_event
global input_lock
stop_event = e
input_lock = l
def stupid_task(n):
while not stop_event.is_set():
with input_lock:
print(n)
time.sleep(2)
def test_mng(n, event, lock):
with mp.Pool(n, initializer=init, initargs=(event, lock,)) as p:
print("before")
p.map(stupid_task, range(1, n + 1))
print("after")
p.close()
p.join()
def main():
i_lock = mp.Lock()
s_event = mp.Event()
thread = tr.Thread(target=test_mng, args=(3, s_event, i_lock))
init(s_event,i_lock)
thread.start()
time.sleep(1) # if this line is commented out only "before" is printed
while not stop_event.is_set():
input("")
with input_lock:
print("stopped")
eingabe = input("type q to quit")
if eingabe == "q":
stop_event.set()
if __name__ == "__main__":
main()
I ask myself what is stopping the process pool from doing it's work. Do i do something fundamentally wrong? The time.sleep seems a little bit hacky.

I my opinion, you are running your script from an IDE (like PyCharm), but not from the Console. Your IDE is catching the keyboard events.
You can simplify the processing:
The main process can wait for the user input,
The thread can do the "stupid task".
Here is a possible solution:
# coding: utf-8
import multiprocessing as mp
import threading as tr
import time
stop_event = None
def init(event):
global stop_event
stop_event = event
def stupid_task(n):
while not stop_event.is_set():
print(n)
time.sleep(2)
def test_mng(n, event):
with mp.Pool(n, initializer=init, initargs=(event,)) as p:
print("before")
p.map(stupid_task, range(1, n + 1))
print("after")
p.close()
p.join()
def main():
print("type 'q' <ENTER> to quit")
s_event = mp.Event()
init(s_event)
thread = tr.Thread(target=test_mng, args=(3, s_event,))
thread.start()
while not stop_event.is_set():
c = input("")
if c in "qQ":
stop_event.set()
if __name__ == "__main__":
main()