We Keep Coding

Threading and target function in external file (python)

I want to move some functions to an external file for making it clearer.
lets say i have this example code (which does indeed work):
import threading
from time import sleep
testVal = 0
def testFunc():
while True:
global testVal
sleep(1)
testVal = testVal + 1
print(testVal)
t = threading.Thread(target=testFunc, args=())
t.daemon = True
t.start()
try:
while True:
sleep(2)
print('testval = ' + str(testVal))
except KeyboardInterrupt:
pass
now i want to move testFunc() to a new python file. My guess was the following but the global variables don't seem to be the same.
testserver.py:
import threading
import testclient
from time import sleep
testVal = 0
t = threading.Thread(target=testclient.testFunc, args=())
t.daemon = True
t.start()
try:
while True:
sleep(2)
print('testval = ' + str(testVal))
except KeyboardInterrupt:
pass
and testclient.py:
from time import sleep
from testserver import testVal as val
def testFunc():
while True:
global val
sleep(1)
val = val + 1
print(val)
my output is:
1
testval = 0
2
3
testval = 0 (testval didn't change)
...
while it should:
1
testval = 1
2
3
testval = 3
...
any suggestions? Thanks!

Your immediate problem is not due to multithreading (we'll get to that) but due to how you use global variables. The thing is, when you use this:
from testserver import testVal as val
You're essentially doing this:
import testserver
val = testserver.testVal
i.e. you're creating a local reference val that points to the testserver.testVal value. This is all fine and dandy when you read it (the first time at least) but when you try to assign its value in your function with:
val = val + 1
You're actually re-assigning the local (to testclient.py) val variable, not setting the value of testserver.testVal. You have to directly reference the actual pointer (i.e. testserver.testVal += 1) if you want to change its value.
That being said, the next problem you might encounter might stem directly from multithreading - you can encounter a race-condition oddity where GIL pauses one thread right after reading the value, but before actually writing it, and the next thread reading it and overwriting the current value, then the first thread resumes and writes the same value resulting in single increase despite two calls. You need to use some sort of mutex to make sure that all non-atomic operations execute exclusively to one thread if you want to use your data this way. The easiest way to do it is with a Lock that comes with the threading module:
testserver.py:
# ...
testVal = 0
testValLock = threading.Lock()
# ...
testclient.py:
# ...
with testserver.testValLock:
testserver.testVal += 1
# ...
A third and final problem you might encounter is a circular dependency (testserver.py requires testclient.py, which requires testserver.py) and I'd advise you to re-think the way you want to approach this problem. If all you want is a common global store - create it separately from modules that might depend on it. That way you ensure proper loading and initializing order without the danger of unresolveable circular dependencies.

Print array data 1 by 1 sleeping between each iteration? [duplicate]

Is there a way to print a spinning cursor in a terminal using Python?

Something like this, assuming your terminal handles \b
import sys
import time
def spinning_cursor():
while True:
for cursor in '|/-\\':
yield cursor
spinner = spinning_cursor()
for _ in range(50):
sys.stdout.write(next(spinner))
sys.stdout.flush()
time.sleep(0.1)
sys.stdout.write('\b')

Easy to use API (this will run the spinner in a separate thread):
import sys
import time
import threading
class Spinner:
busy = False
delay = 0.1
#staticmethod
def spinning_cursor():
while 1:
for cursor in '|/-\\': yield cursor
def __init__(self, delay=None):
self.spinner_generator = self.spinning_cursor()
if delay and float(delay): self.delay = delay
def spinner_task(self):
while self.busy:
sys.stdout.write(next(self.spinner_generator))
sys.stdout.flush()
time.sleep(self.delay)
sys.stdout.write('\b')
sys.stdout.flush()
def __enter__(self):
self.busy = True
threading.Thread(target=self.spinner_task).start()
def __exit__(self, exception, value, tb):
self.busy = False
time.sleep(self.delay)
if exception is not None:
return False
Now use it in a with block anywhere in the code:
with Spinner():
# ... some long-running operations
# time.sleep(3)

A nice pythonic way is to use itertools.cycle:
import itertools, sys
spinner = itertools.cycle(['-', '/', '|', '\\'])
while True:
sys.stdout.write(next(spinner)) # write the next character
sys.stdout.flush() # flush stdout buffer (actual character display)
sys.stdout.write('\b') # erase the last written char
Also, you might want to use threading to display the spinner during a long function call, as in http://www.interclasse.com/scripts/spin.php

For completeness I want to add the great package halo. It offers a lot of preset spinners and higher level customization options.
Extract from their readme
from halo import Halo
spinner = Halo(text='Loading', spinner='dots')
spinner.start()
# Run time consuming work here
# You can also change properties for spinner as and when you want
spinner.stop()
Alternatively, you can use halo with Python's with statement:
from halo import Halo
with Halo(text='Loading', spinner='dots'):
# Run time consuming work here
Finally, you can use halo as a decorator:
from halo import Halo
#Halo(text='Loading', spinner='dots')
def long_running_function():
# Run time consuming work here
pass
long_running_function()

A solution:
import sys
import time
print "processing...\\",
syms = ['\\', '|', '/', '-']
bs = '\b'
for _ in range(10):
for sym in syms:
sys.stdout.write("\b%s" % sym)
sys.stdout.flush()
time.sleep(.5)
The key is to use the backspace character '\b' and flush stdout.

Improved version from #Victor Moyseenko
as the original version had few issues
was leaving spinner's characters after spinning is complete
and sometimes lead to removing following output's first character too
avoids a rare race condition by putting threading.Lock() on output
falls back to simpler output when no tty is available (no spinning)
import sys
import threading
import itertools
import time
class Spinner:
def __init__(self, message, delay=0.1):
self.spinner = itertools.cycle(['-', '/', '|', '\\'])
self.delay = delay
self.busy = False
self.spinner_visible = False
sys.stdout.write(message)
def write_next(self):
with self._screen_lock:
if not self.spinner_visible:
sys.stdout.write(next(self.spinner))
self.spinner_visible = True
sys.stdout.flush()
def remove_spinner(self, cleanup=False):
with self._screen_lock:
if self.spinner_visible:
sys.stdout.write('\b')
self.spinner_visible = False
if cleanup:
sys.stdout.write(' ') # overwrite spinner with blank
sys.stdout.write('\r') # move to next line
sys.stdout.flush()
def spinner_task(self):
while self.busy:
self.write_next()
time.sleep(self.delay)
self.remove_spinner()
def __enter__(self):
if sys.stdout.isatty():
self._screen_lock = threading.Lock()
self.busy = True
self.thread = threading.Thread(target=self.spinner_task)
self.thread.start()
def __exit__(self, exception, value, tb):
if sys.stdout.isatty():
self.busy = False
self.remove_spinner(cleanup=True)
else:
sys.stdout.write('\r')
example of usage of the Spinner class above:
with Spinner("just waiting a bit.. "):
time.sleep(3)
uploaded code to https://github.com/Tagar/stuff/blob/master/spinner.py

Nice, simple, and clean...
while True:
for i in '|\\-/':
print('\b' + i, end='')

Sure, it's possible. It's just a question of printing the backspace character (\b) in between the four characters that would make the "cursor" look like it's spinning ( -, \, |, /).

I have found py-spin package on GitHub. It has many nice spinning Styles. Here are some sample about how to use, Spin1 is the \-/ style:
from __future__ import print_function
import time
from pyspin.spin import make_spin, Spin1
# Choose a spin style and the words when showing the spin.
#make_spin(Spin1, "Downloading...")
def download_video():
time.sleep(10)
if __name__ == '__main__':
print("I'm going to download a video, and it'll cost much time.")
download_video()
print("Done!")
time.sleep(0.1)
It is also possible to control the spin manualy:
from __future__ import print_function
import sys
import time
from pyspin.spin import Spin1, Spinner
# Choose a spin style.
spin = Spinner(Spin1)
# Spin it now.
for i in range(50):
print(u"\r{0}".format(spin.next()), end="")
sys.stdout.flush()
time.sleep(0.1)
Other styles in the below gif.

Grab the awesome progressbar module - http://code.google.com/p/python-progressbar/
use RotatingMarker.

I built a generic Singleton, shared by the entire application
from itertools import cycle
import threading
import time
class Spinner:
__default_spinner_symbols_list = ['|-----|', '|#----|', '|-#---|', '|--#--|', '|---#-|', '|----#|']
def __init__(self, spinner_symbols_list: [str] = None):
spinner_symbols_list = spinner_symbols_list if spinner_symbols_list else Spinner.__default_spinner_symbols_list
self.__screen_lock = threading.Event()
self.__spinner = cycle(spinner_symbols_list)
self.__stop_event = False
self.__thread = None
def get_spin(self):
return self.__spinner
def start(self, spinner_message: str):
self.__stop_event = False
time.sleep(0.3)
def run_spinner(message):
while not self.__stop_event:
print("\r{message} {spinner}".format(message=message, spinner=next(self.__spinner)), end="")
time.sleep(0.3)
self.__screen_lock.set()
self.__thread = threading.Thread(target=run_spinner, args=(spinner_message,), daemon=True)
self.__thread.start()
def stop(self):
self.__stop_event = True
if self.__screen_lock.is_set():
self.__screen_lock.wait()
self.__screen_lock.clear()
print("\r", end="")
print("\r", end="")
if __name__ == '__main__':
import time
# Testing
spinner = Spinner()
spinner.start("Downloading")
# Make actions
time.sleep(5) # Simulate a process
#
spinner.stop()

You can write '\r\033[K' to clear the current line. And the following is a example modified from #nos.
import sys
import time
def spinning_cursor():
while True:
for cursor in '|/-\\':
yield cursor
spinner = spinning_cursor()
for _ in range(1, 10):
content = f'\r{next(spinner)} Downloading...'
print(content, end="")
time.sleep(0.1)
print('\r\033[K', end="")
For anyone who interested in nodejs, I also write a nodejs example.
function* makeSpinner(start = 0, end = 100, step = 1) {
let iterationCount = 0;
while (true) {
for (const char of '|/-\\') {
yield char;
}
}
return iterationCount;
}
async function sleep(seconds) {
return new Promise((resolve) => {
setTimeout(resolve, seconds * 1000);
});
}
(async () => {
const spinner = makeSpinner();
for (let i = 0; i < 10; i++) {
content = `\r${spinner.next().value} Downloading...`;
process.stdout.write(content);
await sleep(0.1);
process.stdout.write('\r\033[K');
}
})();

curses module. i'd have a look at the addstr() and addch() functions. Never used it though.

For more advanced console manipulations, on unix you can use the curses python module, and on windows, you can use WConio which provides equivalent functionality of the curses library.

#!/usr/bin/env python
import sys
chars = '|/-\\'
for i in xrange(1,1000):
for c in chars:
sys.stdout.write(c)
sys.stdout.write('\b')
sys.stdout.flush()
CAVEATS:
In my experience this doesn't work in all terminals. A more robust way to do this under Unix/Linux, be it more complicated is to use the curses module, which doesn't work under Windows.
You probably want to slow it down some how with actual processing that is going on in the background.

Here ya go - simple and clear.
import sys
import time
idx = 0
cursor = ['|','/','-','\\'] #frames of an animated cursor
while True:
sys.stdout.write(cursor[idx])
sys.stdout.write('\b')
idx = idx + 1
if idx > 3:
idx = 0
time.sleep(.1)

Crude but simple solution:
import sys
import time
cursor = ['|','/','-','\\']
for count in range(0,1000):
sys.stdout.write('\b{}'.format(cursor[count%4]))
sys.stdout.flush()
# replace time.sleep() with some logic
time.sleep(.1)
There are obvious limitations, but again, crude.

I propose a solution using decorators
from itertools import cycle
import functools
import threading
import time
def spinner(message, spinner_symbols: list = None):
spinner_symbols = spinner_symbols or list("|/-\\")
spinner_symbols = cycle(spinner_symbols)
global spinner_event
spinner_event = True
def start():
global spinner_event
while spinner_event:
symbol = next(spinner_symbols)
print("\r{message} {symbol}".format(message=message, symbol=symbol), end="")
time.sleep(0.3)
def stop():
global spinner_event
spinner_event = False
print("\r", end="")
def external(fct):
#functools.wraps(fct)
def wrapper(*args):
spinner_thread = threading.Thread(target=start, daemon=True)
spinner_thread.start()
result = fct(*args)
stop()
spinner_thread.join()
return result
return wrapper
return external
Simple usage
#spinner("Downloading")
def f():
time.sleep(10)

import sys
def DrowWaitCursor(counter):
if counter % 4 == 0:
print("/",end = "")
elif counter % 4 == 1:
print("-",end = "")
elif counter % 4 == 2:
print("\\",end = "")
elif counter % 4 == 3:
print("|",end = "")
sys.stdout.flush()
sys.stdout.write('\b')
This can be also another solution using a function with a parameter.

I just started with python about a week ago and found this posting. I've combined a bit of what I found here with stuff I learned about threads and queues elsewhere to provide a much better implementation in my opinion. In my solution, writing to the screen is handled by a thread that checks a queue for content. If that queue has content, the cursor spinning thread knows to stop. On the flipside, the cursor spinning thread uses a queue as a lock so the printing thread knows not to print until a full pass of the spinner code is complete. This prevents race conditions and a lot of excess code people are using to keep the console clean.
See below:
import threading, queue, itertools, sys, time # all required for my version of spinner
import datetime #not required for spinning cursor solution, only my example
console_queue = queue.Queue() # this queue should be initialized before functions
screenlock = queue.Queue() # this queue too...
def main():
threading.Thread(target=spinner).start()
threading.Thread(target=consoleprint).start()
while True:
# instead of invoking print or stdout.write, we just add items to the console_queue
# The next three lines are an example of my code in practice.
time.sleep(.5) # wait half a second
currenttime = "[" + datetime.datetime.now().strftime("%d/%m/%Y %H:%M:%S") + "] "
console_queue.put(currenttime) # The most important part. Substitute your print and stdout functions with this.
def spinner(console_queue = console_queue, screenlock = screenlock):
spinnerlist = itertools.cycle(['|', '/', '-', '\\'])
while True:
if console_queue.empty():
screenlock.put("locked")
sys.stdout.write(next(spinnerlist))
sys.stdout.flush()
sys.stdout.write('\b')
sys.stdout.flush()
screenlock.get()
time.sleep(.1)
def consoleprint(console_queue = console_queue, screenlock = screenlock):
while True:
if not console_queue.empty():
while screenlock.empty() == False:
time.sleep(.1)
sys.stdout.flush()
print(console_queue.get())
sys.stdout.flush()
if __name__ == "__main__":
main()
Having said all I said and written all I've written, I've only been doing python stuff for a week. If there are cleaner ways of doing this or I missed some best practices I'd love to learn. Thanks.

import requests
import time
import sys
weathercity = input("What city are you in? ")
weather = requests.get('http://api.openweathermap.org/data/2.5/weather?q='+weathercity+'&appid=886705b4c1182eb1c69f28eb8c520e20')
url = ('http://api.openweathermap.org/data/2.5/weather?q='+weathercity+'&appid=886705b4c1182eb1c69f28eb8c520e20')
def spinning_cursor():
while True:
for cursor in '|/-\\':
yield cursor
data = weather.json()
temp = data['main']['temp']
description = data['weather'][0]['description']
weatherprint ="In {}, it is currently {}°C with {}."
spinner = spinning_cursor()
for _ in range(25):
sys.stdout.write(next(spinner))
sys.stdout.flush()
time.sleep(0.1)
sys.stdout.write('\b')
convert = int(temp - 273.15)
print(weatherprint.format(weathercity, convert, description))

if you wanna python text spinner you can look picture
Simple:print_spinner("Hayatta en hakiki mürşit ilimdir.")

Here is the simplest loading spinner for python:
import time
spin=["loading...... ", "|", "/","-", "\"]
for i in spin:
print("\b"+i, end=" ")
time.sleep (0.2)
Output:
loading...... |

Lock with timeout in Python2.7

The accepted solution here doesn't work for all situations,
How to implement a Lock with a timeout in Python 2.7
(In particular the last thread who owns the lock calls cond.notify() when no one holds the conditional variable)
Then, I've tried a spin lock like this:
import threading
import time
class TimeLock(object):
def __init__(self):
self._lock = threading.Lock()
def acquire_lock(self, timeout = 0):
''' If timeout = 0, do a blocking lock
else, return False at [timeout] seconds
'''
if timeout == 0:
return self._lock.acquire() # Block for the lock
current_time = start_time = time.time()
while current_time < start_time + timeout:
if self._lock.acquire(False): # Contend for the lock, without blocking
return True
else:
time.sleep(1)
current_time = time.time()
# Time out
return False
def release_lock(self):
self._lock.release()
However after trying, the spin lock will almost always starve against the blocking lock.
Is there other solutions?

Turns out that python queues have a timeout feature in their
Queue module in 2.7
I can simulate a lock with time out by doing this
lock.acquire() -> Queue.get(block=True, timeout=timeout)
lock.release() -> Queue.put(1, block=False)

Python wait x secs for a key and continue execution if not pressed

I'm a n00b to python, and I'm looking a code snippet/sample which performs the following:
Display a message like "Press any key to configure or wait X seconds to continue"
Wait, for example, 5 seconds and continue execution, or enter a configure() subroutine if a key is pressed.
Thank you for your help!
Yvan Janssens

If you're on Unix/Linux then the select module will help you.
import sys
from select import select
print "Press any key to configure or wait 5 seconds..."
timeout = 5
rlist, wlist, xlist = select([sys.stdin], [], [], timeout)
if rlist:
print "Config selected..."
else:
print "Timed out..."
If you're on Windows, then look into the msvcrt module. (Note this doesn't work in IDLE, but will in cmd prompt)
import sys, time, msvcrt
timeout = 5
startTime = time.time()
inp = None
print "Press any key to configure or wait 5 seconds... "
while True:
if msvcrt.kbhit():
inp = msvcrt.getch()
break
elif time.time() - startTime > timeout:
break
if inp:
print "Config selected..."
else:
print "Timed out..."
Edit Changed the code samples so you could tell whether there was a timeout or a keypress...

Python doesn't have any standard way to catch this, it gets keyboard input only through input() and raw_input().
If you really want this you could use Tkinter or pygame to catch the keystrokes as "events". There are also some platform-specific solutions like pyHook. But if it's not absolutely vital to your program, I suggest you make it work another way.

If you combine time.sleep, threading.Thread, and sys.stdin.read you can easily wait for a specified amount of time for input and then continue.
t = threading.Thread(target=sys.stdin.read(1) args=(1,))
t.start()
time.sleep(5)
t.join()

Here's how I did it:
import threading
import time
import sys
class MyThread(threading.Thread):
def __init__(self, threadID, name, counter, f):
super().__init__()
self.threadID = threadID
self.name = name
self.counter = counter
self.func = f
def run(self):
self.func()
class KeyboardMonitor:
def __init__(self):
# Setting a boolean flag is atomic in Python.
# It's hard to imagine a boolean being
# anything else, with or without the GIL.
# If inter-thread communication is anything more complicated than
# a couple of flags, you should replace low level variables with
# a thread safe buffer.
self.keepGoing = True
def wait4KeyEntry(self):
while self.keepGoing:
s = input("Type q to quit: ")
if s == "q":
self.keepGoing = False
def mainThread(self, f, *args, **kwargs):
"""Pass in some main function you want to run, and this will run it
until keepGoing = False. The first argument of function f must be
this class, so that that function can check the keepGoing flag and
quit when keepGoing is false."""
keyboardThread = MyThread(1, "keyboard_thread", 0, self.wait4KeyEntry)
keyboardThread.start()
while self.keepGoing:
f(self, *args, **kwargs)
def main(keyMonitorInst, *args, **kwargs):
while keyMonitorInst.keepGoing:
print("Running again...")
time.sleep(1)
if __name__ == "__main__":
uut = KeyboardMonitor()
uut.mainThread(main)
Rather than make a blocking call time out, my approach is to start a thread that waits for the user to enter input, while another thread does something else. The two processes communicate through a small number of atomic operations: in this case, setting a boolean flag. For anything more complicated than atomic operations, obviously you should replace the atomic variable with a threadsafe buffer of some kind.

kill a function after a certain time in windows

I've read a lot of posts about using threads, subprocesses, etc.. A lot of it seems over complicated for what I'm trying to do...
All I want to do is stop executing a function after X amount of time has elapsed.
def big_loop(bob):
x = bob
start = time.time()
while True:
print time.time()-start
This function is an endless loop that never throws any errors or exceptions, period.
I"m not sure the difference between "commands, shells, subprocesses, threads, etc.." and this function, which is why I'm having trouble manipulating subprocesses.
I found this code here, and tried it but as you can see it keeps printing after 10 seconds have elapsed:
import time
import threading
import subprocess as sub
import time
class RunCmd(threading.Thread):
def __init__(self, cmd, timeout):
threading.Thread.__init__(self)
self.cmd = cmd
self.timeout = timeout
def run(self):
self.p = sub.Popen(self.cmd)
self.p.wait()
def Run(self):
self.start()
self.join(self.timeout)
if self.is_alive():
self.p.terminate()
self.join()
def big_loop(bob):
x = bob
start = time.time()
while True:
print time.time()-start
RunCmd(big_loop('jimijojo'), 10).Run() #supposed to quit after 10 seconds, but doesn't
x = raw_input('DONEEEEEEEEEEEE')
What's a simple way this function can be killed. As you can see in my attempt above, it doesn't terminate after 20 seconds and just keeps on going...
***OH also, I've read about using signal, but I"m on windows so I can't use the alarm feature.. (python 2.7)
**assume the "infinitely running function" can't be manipulated or changed to be non-infinite, if I could change the function, well I'd just change it to be non infinite wouldn't I?
Here are some similar questions, which I haven't able to port over their code to work with my simple function:
Perhaps you can?
Python: kill or terminate subprocess when timeout
signal.alarm replacement in Windows [Python]
Ok I tried an answer I received, it works.. but how can I use it if I remove the if __name__ == "__main__": statement? When I remove this statement, the loop never ends as it did before..
import multiprocessing
import Queue
import time
def infinite_loop_function(bob):
var = bob
start = time.time()
while True:
time.sleep(1)
print time.time()-start
print 'this statement will never print'
def wrapper(queue, bob):
result = infinite_loop_function(bob)
queue.put(result)
queue.close()
#if __name__ == "__main__":
queue = multiprocessing.Queue(1) # Maximum size is 1
proc = multiprocessing.Process(target=wrapper, args=(queue, 'var'))
proc.start()
# Wait for TIMEOUT seconds
try:
timeout = 10
result = queue.get(True, timeout)
except Queue.Empty:
# Deal with lack of data somehow
result = None
finally:
proc.terminate()
print 'running other code, now that that infinite loop has been defeated!'
print 'bla bla bla'
x = raw_input('done')

Use the building blocks in the multiprocessing module:
import multiprocessing
import Queue
TIMEOUT = 5
def big_loop(bob):
import time
time.sleep(4)
return bob*2
def wrapper(queue, bob):
result = big_loop(bob)
queue.put(result)
queue.close()
def run_loop_with_timeout():
bob = 21 # Whatever sensible value you need
queue = multiprocessing.Queue(1) # Maximum size is 1
proc = multiprocessing.Process(target=wrapper, args=(queue, bob))
proc.start()
# Wait for TIMEOUT seconds
try:
result = queue.get(True, TIMEOUT)
except Queue.Empty:
# Deal with lack of data somehow
result = None
finally:
proc.terminate()
# Process data here, not in try block above, otherwise your process keeps running
print result
if __name__ == "__main__":
run_loop_with_timeout()
You could also accomplish this with a Pipe/Connection pair, but I'm not familiar with their API. Change the sleep time or TIMEOUT to check the behaviour for either case.

There is no straightforward way to kill a function after a certain amount of time without running the function in a separate process. A better approach would probably be to rewrite the function so that it returns after a specified time:
import time
def big_loop(bob, timeout):
x = bob
start = time.time()
end = start + timeout
while time.time() < end:
print time.time() - start
# Do more stuff here as needed

Can't you just return from the loop?
start = time.time()
endt = start + 30
while True:
now = time.time()
if now > endt:
return
else:
print end - start

import os,signal,time
cpid = os.fork()
if cpid == 0:
while True:
# do stuff
else:
time.sleep(10)
os.kill(cpid, signal.SIGKILL)
You can also check in the loop of a thread for an event, which is more portable and flexible as it allows other reactions than brute killing. However, this approach fails if # do stuff can take time (or even wait forever on some event).