The story begin with two threads and a global variable that change.. a lot of time :)
Thread number one (for simplicity we will call t1) generates a random number and store it in a global variable GLB.
Thread number two (aka t2) check the value of the global variable and when it reaches a value starts to print his value until a period of time.
BUT if t1 changes the value of that global variable, also change the value inside the loop and I don't want this!
I try to write pseudocode:
import random
import time
import threading
GLB = [0,0]
#this is a thread
def t1():
while True:
GLB[0] = random.randint(0, 100)
GLB[1] = 1
print GLB
time.sleep(5)
#this is a thread
def t2():
while True:
if GLB[0]<=30:
static = GLB
for i in range(50):
print i," ",static
time.sleep(1)
a = threading.Thread(target=t1)
a.start()
b = threading.Thread(target=t2)
b.start()
while True:
time.sleep(1)
The question is: why variable static change inside the loop for? It should be remain constant unitl it escapes from loop!
Could I create a lock to the variable? Or there is any other way to solve the problem?
Thanks regards.
GLB is a mutable object. To let one thread see a consistent value while another thread modifies it you can either protect the object temporarily with a lock (the modifier will wait) or copy the object. In your example, a copy seems the best option. In python, a slice copy is atomic so does not need any other locking.
import random
import time
import threading
GLB = [0,0]
#this is a thread
def t1():
while True:
GLB[0] = random.randint(0, 100)
GLB[1] = 1
print GLB
time.sleep(5)
#this is a thread
def t2():
while True:
static = GLB[:]
if static[0]<=30:
for i in range(50):
print i," ",static
time.sleep(1)
a = threading.Thread(target=t1)
a.start()
b = threading.Thread(target=t2)
b.start()
while True:
time.sleep(1)
Related
I am making a simple project to learn about threading and this is my code:
import time
import threading
x = 0
def printfunction():
while x == 0:
print("process running")
def timer(delay):
while True:
time.sleep(delay)
break
x = 1
return x
t1 = threading.Thread(target = timer,args=[3])
t2 = threading.Thread(target = printfunction)
t1.start()
t2.start()
t1.join()
t2.join()
It is supposed to just print out process running in the console for three seconds but it never stops printing. The console shows me no errors and I have tried shortening the time to see if I wasn't waiting long enough but it still doesn't work. Then I tried to delete the t1.join()and t2.join()but I still have no luck and the program continues running.
What am I doing wrong?
Add
global x
to the top of timer(). As is, because timer() assigns to x, x is considered to be local to timer(), and its x = 1 has no effect on the module-level variable also named x. The global x remains 0 forever, so the while x == 0: in printfunction() always succeeds. It really has nothing to do with threading :-)
I want to start thread multiple times, but only when it is not running.
There is a simple model what I am trying:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
if not H.isAlive():
H.start()
print(threading.active_count())
Also what i don't understand is that:
When I run program it prints: 0. Then after 3 seconds it prints:1 and so on after every 3 second it is increased by 1.
But I thought it would print: 0. Then after 3 second it would print: 1. And then immediately increasing fast.
Because after starting first thread it would immediately start the next one and so on. why does this happen?
How not to start a thread 'up', if it's already running?
Not sure if I got your question completely, but here are some thoughts.
When I run your code I get an increasing number of active threads, as you are creating a new thread every time, checking its status (which will always be not alive) and then starting it.
What you want to do instead is to check the status of the last run thread and if that's not alive start a new one. In order to do that you should create a new thread if the old one is done:
def up (x, r):
time.sleep(3)
r['h'] = x + 1
def main():
hum = {'h' : 0}
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
while True:
# print(hum['h'])
if not H.isAlive():
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
print(threading.active_count())
What happens in your code:
Print the value of hum['h']
Create a thread (note you create it, you are not starting it yet)
Set the value of a property
If the thread is not started then start it
Print the count of active threads (active, NOT started)
Since you replace the H variable every time, you'll have a new thread every time that gets immediately started.
If you add a print that says "starting" in the if for the is alive, you'll see that it gets called every time.
You can use join() to wait for the thread to finish:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
H.join()
print(threading.active_count())
If you don't want to wait, you can just save the current running thread in a variable and check it in the loop:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
current_thread = None
while True:
print(hum['h'])
if current_thread is None:
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()
elif not current_thread.isAlive():
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()
The provided code is about 2 thread trying to access the function increment() to increment the value of a global variable x. I have designed a semaphore class for process synchronization. So the expected increment of each thread is expected to be 1000000 summing up to 2000000. But actual output is not reaching up to 2000000. The output is reaching up to 1800000 - 1950000. Why are all loop not executing?
import threading as th
x=0
class Semaphore:
def __init__(self):
self.__s = 1
def wait(self):
while(self.__s==0):
pass
self.__s-=1
def signal(self):
self.__s+=1
def increament(s):
global x
s.wait()
x+=1
s.signal()
def task1(s):
for _ in range(1000000):
increament(s)
def task2(s):
for _ in range(1000000):
increament(s)
def main():
s = Semaphore()
t1 = th.Thread(target=task1,name="t1",args=(s,))
t2 = th.Thread(target=task2,name="t1",args=(s,))
t1.start()
t2.start()
#Checking Synchronization
for _ in range(10):
print("Value of X: %d"%x)
#waiting for termination of thread
t2.join()
t1.join()
if __name__=="__main__":
main()
print("X = %d"%x) #Final Output
Output:
Value of X: 5939
Value of X: 14150
Value of X: 25036
Value of X: 50490
Value of X: 54136
Value of X: 57674
Value of X: 69994
Value of X: 84912
Value of X: 94284
Value of X: 105895
X = 1801436
The threads are working fine and they're completing correctly. It's your 'z' variable that's the problem.
In general using a global variable as a container for your shared memory between two threads is a bad way to go about it.
Check out this answer to see why.
I made the following changes to your code. I made 'z' the shared variable and 'x' and 'y' are data for each thread alone.
x=0
y=0
z=0
def increament1(s):
global x,z
s.wait()
x+=1
z+=1
s.signal()
def increament2(s):
global y,z
s.wait()
y+=1
z+=1
s.signal()
def task1(s):
for somei in range(1000000):
increament1(s)
def task2(s):
for somej in range(1000000):
increament2(s)
This is the output I got:
X = 1000000
Y = 1000000
Z = 1961404
As you can see there's nothing wrong with the threads themselves, as they're completing their execution. But the shared data Z is a little wonky. Z will change randomly each time you run the script. Hence as you can see using global variables as shared memory is a bad idea.
A much better option would be using some python supported sharing tool such as Queue provided by python's library itself. It's a multi-producer, multi-consumer message queue and helps when it comes to shared data such as the data you're using now.
Let me show you how it can be done with Queue:
import threading as th
from Queue import Queue
def task1(q):
global x,z
for somei in range(1000000):
q.put(q.get() + 1)
def task2(q):
global y,z
for somei in range(1000000):
q.put(q.get() + 1)
def main():
queue = Queue()
queue.put(0)
t1 = th.Thread(target=task1,name="t1",args=(queue, ))
t2 = th.Thread(target=task2,name="t1",args=(queue, ))
t1.start()
t2.start()
#Checking Synchronization
t1.join()
t2.join()
return queue.get()
if __name__=="__main__":
print("Queue = %d"%main()) #Final Output
You don't even need to create a semaphore here as the Queue will automatically take care of synchronization.
The output of this final program is this:
Queue = 2000000
I want to move some functions to an external file for making it clearer.
lets say i have this example code (which does indeed work):
import threading
from time import sleep
testVal = 0
def testFunc():
while True:
global testVal
sleep(1)
testVal = testVal + 1
print(testVal)
t = threading.Thread(target=testFunc, args=())
t.daemon = True
t.start()
try:
while True:
sleep(2)
print('testval = ' + str(testVal))
except KeyboardInterrupt:
pass
now i want to move testFunc() to a new python file. My guess was the following but the global variables don't seem to be the same.
testserver.py:
import threading
import testclient
from time import sleep
testVal = 0
t = threading.Thread(target=testclient.testFunc, args=())
t.daemon = True
t.start()
try:
while True:
sleep(2)
print('testval = ' + str(testVal))
except KeyboardInterrupt:
pass
and testclient.py:
from time import sleep
from testserver import testVal as val
def testFunc():
while True:
global val
sleep(1)
val = val + 1
print(val)
my output is:
1
testval = 0
2
3
testval = 0 (testval didn't change)
...
while it should:
1
testval = 1
2
3
testval = 3
...
any suggestions? Thanks!
Your immediate problem is not due to multithreading (we'll get to that) but due to how you use global variables. The thing is, when you use this:
from testserver import testVal as val
You're essentially doing this:
import testserver
val = testserver.testVal
i.e. you're creating a local reference val that points to the testserver.testVal value. This is all fine and dandy when you read it (the first time at least) but when you try to assign its value in your function with:
val = val + 1
You're actually re-assigning the local (to testclient.py) val variable, not setting the value of testserver.testVal. You have to directly reference the actual pointer (i.e. testserver.testVal += 1) if you want to change its value.
That being said, the next problem you might encounter might stem directly from multithreading - you can encounter a race-condition oddity where GIL pauses one thread right after reading the value, but before actually writing it, and the next thread reading it and overwriting the current value, then the first thread resumes and writes the same value resulting in single increase despite two calls. You need to use some sort of mutex to make sure that all non-atomic operations execute exclusively to one thread if you want to use your data this way. The easiest way to do it is with a Lock that comes with the threading module:
testserver.py:
# ...
testVal = 0
testValLock = threading.Lock()
# ...
testclient.py:
# ...
with testserver.testValLock:
testserver.testVal += 1
# ...
A third and final problem you might encounter is a circular dependency (testserver.py requires testclient.py, which requires testserver.py) and I'd advise you to re-think the way you want to approach this problem. If all you want is a common global store - create it separately from modules that might depend on it. That way you ensure proper loading and initializing order without the danger of unresolveable circular dependencies.
Hi i would like to know what method i could use to call a function a few times and each call is processed in parallel and NOT in a queue based processing.
Something along this line
import time
import random
def run(incoming):
time.sleep(5)
print incoming
break
while True:
hash = random.getrandbits(128)
run(hash)
time.sleep(1)
import time
import random
import threading
def run(incoming):
time.sleep(5)
print incoming
while True:
hash = random.getrandbits(128)
threading.Thread(target = run,args = (hash,)).start()
time.sleep(1)
note that this is restricted by the gil where it interleaves the processes ... but for your purposes you can probably call it parallel and since your thread count keeps growing it may eventually break down
there are much better ways to do this lets check it out
def do_hard_work(hash):
time.sleep(1)
def Run(data_pipe):
while True:
while data_pipe.poll():
hash = data_pipe.recv()
if hash == "QUIT":
break
threading.Thread(target=do_hard_work,args=(hash)).start()
time.sleep(1)
local,remote = multiprocessing.Pipe()
worker_process = multiprocessing.Process(target=run,args=(local,))
worker_process.start()
while True:
remote.send(random.getrandbits(128))
time.sleep(1)
if some_condition:
remote.send("QUIT")
break