I am making a simple project to learn about threading and this is my code:
import time
import threading
x = 0
def printfunction():
while x == 0:
print("process running")
def timer(delay):
while True:
time.sleep(delay)
break
x = 1
return x
t1 = threading.Thread(target = timer,args=[3])
t2 = threading.Thread(target = printfunction)
t1.start()
t2.start()
t1.join()
t2.join()
It is supposed to just print out process running in the console for three seconds but it never stops printing. The console shows me no errors and I have tried shortening the time to see if I wasn't waiting long enough but it still doesn't work. Then I tried to delete the t1.join()and t2.join()but I still have no luck and the program continues running.
What am I doing wrong?
Add
global x
to the top of timer(). As is, because timer() assigns to x, x is considered to be local to timer(), and its x = 1 has no effect on the module-level variable also named x. The global x remains 0 forever, so the while x == 0: in printfunction() always succeeds. It really has nothing to do with threading :-)
Related
I'd like to run multiple python scripts in parallel and start them from a master script. I did find solutions for this in previously asked questions, however, none of these worked if the scripts running in parallel contained loops.
Let's for example define two scripts.
Script 1:
array_1 = []
x = 0
while True:
array_1.append(x)
x = x + 1
Script 2:
array_2 = []
x = 0
while True:
array_2.append(x)
x = x + 1
Now I want to run both processes simultaneously. Previous solutions suggested the following code for a master script:
import script_1, script_2
exec(open(script_1))
exec(open(script_2))
While this is a solution for starting scripts from within another script, however, this will not run the two scripts in parallel.
What should such a master script actually look like ?
Thanks for your suggestions!
Edit
I tried the following threading approach:
def function_1():
print('function 1 started...')
while True:
print('1')
sleep(1)
def function_2():
print('function 2 started...')
while True:
print('2')
sleep(1)
thread_1 = Thread(target=function_1())
thread_2 = Thread(target=function_2())
thread_1.start()
thread_2.start()
thread_1.join()
thread_2.join()
print("thread finished")
It doesn't work, only the first function gets started so I get the following output:
function 1 started...
1
1
1
1
1
1
When you want to spawn a new thread, you need to pass the address of the function you want the thread to execute, and not to call it. What you are doing here is essentially spawning a new thread that immediately calls function_1() which of course runs forever.
Also, you won't be able to reach this line of code:
print("thread finished")
As the threads are executing a while loop - forever, so it is redundent..
from time import sleep
from threading import Thread
def function_1():
print('function 1 started...')
while True:
print('1')
sleep(1)
def function_2():
print('function 2 started...')
while True:
print('2')
sleep(1)
thread_1 = Thread(target=function_1)
thread_2 = Thread(target=function_2)
thread_1.start()
thread_2.start()
thread_1.join()
thread_2.join()
# print("thread finished") - redundant
I want to start thread multiple times, but only when it is not running.
There is a simple model what I am trying:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
if not H.isAlive():
H.start()
print(threading.active_count())
Also what i don't understand is that:
When I run program it prints: 0. Then after 3 seconds it prints:1 and so on after every 3 second it is increased by 1.
But I thought it would print: 0. Then after 3 second it would print: 1. And then immediately increasing fast.
Because after starting first thread it would immediately start the next one and so on. why does this happen?
How not to start a thread 'up', if it's already running?
Not sure if I got your question completely, but here are some thoughts.
When I run your code I get an increasing number of active threads, as you are creating a new thread every time, checking its status (which will always be not alive) and then starting it.
What you want to do instead is to check the status of the last run thread and if that's not alive start a new one. In order to do that you should create a new thread if the old one is done:
def up (x, r):
time.sleep(3)
r['h'] = x + 1
def main():
hum = {'h' : 0}
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
while True:
# print(hum['h'])
if not H.isAlive():
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
print(threading.active_count())
What happens in your code:
Print the value of hum['h']
Create a thread (note you create it, you are not starting it yet)
Set the value of a property
If the thread is not started then start it
Print the count of active threads (active, NOT started)
Since you replace the H variable every time, you'll have a new thread every time that gets immediately started.
If you add a print that says "starting" in the if for the is alive, you'll see that it gets called every time.
You can use join() to wait for the thread to finish:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
H.join()
print(threading.active_count())
If you don't want to wait, you can just save the current running thread in a variable and check it in the loop:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
current_thread = None
while True:
print(hum['h'])
if current_thread is None:
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()
elif not current_thread.isAlive():
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()
I am suppose to modify this code without changing the main function to stop it from deadlocking. It is deadlocking because of how the locks end up waiting for each other but I cannot figure out how to stop it. My professors lecture talks about os.fork which I can't use since I am on windows.
I was looking into the pool thing with multiprocessing but can't see how to implement that without changing the main function. I am pretty sure I am supposed to use subprocess, but again, she didn't include any information about it and I can't find a relevant example online.
import threading
x = 0
def task(lock1, lock2, count):
global x
for i in range(count):
lock1.acquire()
lock2.acquire()
# Assume that a thread can update the x value
# only after both locks have been acquired.
x+=1
print(x)
lock2.release()
lock1.release()
# Do not modify the main method
def main():
global x
count = 1000
lock1 = threading.Lock()
lock2 = threading.Lock()
T1 = threading.Thread(target = task, args = (lock1, lock2, count))
T2 = threading.Thread(target = task, args = (lock2, lock1, count))
T1.start()
T2.start()
T1.join()
T2.join()
print(f"x = {x}")
main()
Edit: Changing task to this seems to have fixed it, although I do not think it was done the way she wanted...
def task(lock1, lock2, count):
global x
for i in range(count):
lock1.acquire(False)
lock2.acquire(False)
# Assume that a thread can update the x value
# only after both locks have been acquired.
x+=1
print(x)
if lock2.locked():
lock2.release()
if lock1.locked():
lock1.release()
Your threads need to lock the locks in a consistent order. You can do this by locking the one with the lower id value first:
def task(lock1, lock2, count):
global x
if id(lock1) > id(lock2):
lock1, lock2 = lock2, lock1
for i in range(count):
lock1.acquire()
lock2.acquire()
# Assume that a thread can update the x value
# only after both locks have been acquired.
x+=1
print(x)
lock2.release()
lock1.release()
With a consistent lock order, it's impossible for two threads to each be holding a lock the other needs.
(multiprocessing, subprocess, and os.fork are all unhelpful here. They would just add more issues.)
The provided code is about 2 thread trying to access the function increment() to increment the value of a global variable x. I have designed a semaphore class for process synchronization. So the expected increment of each thread is expected to be 1000000 summing up to 2000000. But actual output is not reaching up to 2000000. The output is reaching up to 1800000 - 1950000. Why are all loop not executing?
import threading as th
x=0
class Semaphore:
def __init__(self):
self.__s = 1
def wait(self):
while(self.__s==0):
pass
self.__s-=1
def signal(self):
self.__s+=1
def increament(s):
global x
s.wait()
x+=1
s.signal()
def task1(s):
for _ in range(1000000):
increament(s)
def task2(s):
for _ in range(1000000):
increament(s)
def main():
s = Semaphore()
t1 = th.Thread(target=task1,name="t1",args=(s,))
t2 = th.Thread(target=task2,name="t1",args=(s,))
t1.start()
t2.start()
#Checking Synchronization
for _ in range(10):
print("Value of X: %d"%x)
#waiting for termination of thread
t2.join()
t1.join()
if __name__=="__main__":
main()
print("X = %d"%x) #Final Output
Output:
Value of X: 5939
Value of X: 14150
Value of X: 25036
Value of X: 50490
Value of X: 54136
Value of X: 57674
Value of X: 69994
Value of X: 84912
Value of X: 94284
Value of X: 105895
X = 1801436
The threads are working fine and they're completing correctly. It's your 'z' variable that's the problem.
In general using a global variable as a container for your shared memory between two threads is a bad way to go about it.
Check out this answer to see why.
I made the following changes to your code. I made 'z' the shared variable and 'x' and 'y' are data for each thread alone.
x=0
y=0
z=0
def increament1(s):
global x,z
s.wait()
x+=1
z+=1
s.signal()
def increament2(s):
global y,z
s.wait()
y+=1
z+=1
s.signal()
def task1(s):
for somei in range(1000000):
increament1(s)
def task2(s):
for somej in range(1000000):
increament2(s)
This is the output I got:
X = 1000000
Y = 1000000
Z = 1961404
As you can see there's nothing wrong with the threads themselves, as they're completing their execution. But the shared data Z is a little wonky. Z will change randomly each time you run the script. Hence as you can see using global variables as shared memory is a bad idea.
A much better option would be using some python supported sharing tool such as Queue provided by python's library itself. It's a multi-producer, multi-consumer message queue and helps when it comes to shared data such as the data you're using now.
Let me show you how it can be done with Queue:
import threading as th
from Queue import Queue
def task1(q):
global x,z
for somei in range(1000000):
q.put(q.get() + 1)
def task2(q):
global y,z
for somei in range(1000000):
q.put(q.get() + 1)
def main():
queue = Queue()
queue.put(0)
t1 = th.Thread(target=task1,name="t1",args=(queue, ))
t2 = th.Thread(target=task2,name="t1",args=(queue, ))
t1.start()
t2.start()
#Checking Synchronization
t1.join()
t2.join()
return queue.get()
if __name__=="__main__":
print("Queue = %d"%main()) #Final Output
You don't even need to create a semaphore here as the Queue will automatically take care of synchronization.
The output of this final program is this:
Queue = 2000000
I've read a lot of posts about using threads, subprocesses, etc.. A lot of it seems over complicated for what I'm trying to do...
All I want to do is stop executing a function after X amount of time has elapsed.
def big_loop(bob):
x = bob
start = time.time()
while True:
print time.time()-start
This function is an endless loop that never throws any errors or exceptions, period.
I"m not sure the difference between "commands, shells, subprocesses, threads, etc.." and this function, which is why I'm having trouble manipulating subprocesses.
I found this code here, and tried it but as you can see it keeps printing after 10 seconds have elapsed:
import time
import threading
import subprocess as sub
import time
class RunCmd(threading.Thread):
def __init__(self, cmd, timeout):
threading.Thread.__init__(self)
self.cmd = cmd
self.timeout = timeout
def run(self):
self.p = sub.Popen(self.cmd)
self.p.wait()
def Run(self):
self.start()
self.join(self.timeout)
if self.is_alive():
self.p.terminate()
self.join()
def big_loop(bob):
x = bob
start = time.time()
while True:
print time.time()-start
RunCmd(big_loop('jimijojo'), 10).Run() #supposed to quit after 10 seconds, but doesn't
x = raw_input('DONEEEEEEEEEEEE')
What's a simple way this function can be killed. As you can see in my attempt above, it doesn't terminate after 20 seconds and just keeps on going...
***OH also, I've read about using signal, but I"m on windows so I can't use the alarm feature.. (python 2.7)
**assume the "infinitely running function" can't be manipulated or changed to be non-infinite, if I could change the function, well I'd just change it to be non infinite wouldn't I?
Here are some similar questions, which I haven't able to port over their code to work with my simple function:
Perhaps you can?
Python: kill or terminate subprocess when timeout
signal.alarm replacement in Windows [Python]
Ok I tried an answer I received, it works.. but how can I use it if I remove the if __name__ == "__main__": statement? When I remove this statement, the loop never ends as it did before..
import multiprocessing
import Queue
import time
def infinite_loop_function(bob):
var = bob
start = time.time()
while True:
time.sleep(1)
print time.time()-start
print 'this statement will never print'
def wrapper(queue, bob):
result = infinite_loop_function(bob)
queue.put(result)
queue.close()
#if __name__ == "__main__":
queue = multiprocessing.Queue(1) # Maximum size is 1
proc = multiprocessing.Process(target=wrapper, args=(queue, 'var'))
proc.start()
# Wait for TIMEOUT seconds
try:
timeout = 10
result = queue.get(True, timeout)
except Queue.Empty:
# Deal with lack of data somehow
result = None
finally:
proc.terminate()
print 'running other code, now that that infinite loop has been defeated!'
print 'bla bla bla'
x = raw_input('done')
Use the building blocks in the multiprocessing module:
import multiprocessing
import Queue
TIMEOUT = 5
def big_loop(bob):
import time
time.sleep(4)
return bob*2
def wrapper(queue, bob):
result = big_loop(bob)
queue.put(result)
queue.close()
def run_loop_with_timeout():
bob = 21 # Whatever sensible value you need
queue = multiprocessing.Queue(1) # Maximum size is 1
proc = multiprocessing.Process(target=wrapper, args=(queue, bob))
proc.start()
# Wait for TIMEOUT seconds
try:
result = queue.get(True, TIMEOUT)
except Queue.Empty:
# Deal with lack of data somehow
result = None
finally:
proc.terminate()
# Process data here, not in try block above, otherwise your process keeps running
print result
if __name__ == "__main__":
run_loop_with_timeout()
You could also accomplish this with a Pipe/Connection pair, but I'm not familiar with their API. Change the sleep time or TIMEOUT to check the behaviour for either case.
There is no straightforward way to kill a function after a certain amount of time without running the function in a separate process. A better approach would probably be to rewrite the function so that it returns after a specified time:
import time
def big_loop(bob, timeout):
x = bob
start = time.time()
end = start + timeout
while time.time() < end:
print time.time() - start
# Do more stuff here as needed
Can't you just return from the loop?
start = time.time()
endt = start + 30
while True:
now = time.time()
if now > endt:
return
else:
print end - start
import os,signal,time
cpid = os.fork()
if cpid == 0:
while True:
# do stuff
else:
time.sleep(10)
os.kill(cpid, signal.SIGKILL)
You can also check in the loop of a thread for an event, which is more portable and flexible as it allows other reactions than brute killing. However, this approach fails if # do stuff can take time (or even wait forever on some event).