We Keep Coding

Using Thread.join() method with threads that inside inside a class

I am solving this question from LeetCode: 1116. Print Zero Even Odd
I am running this solution in VS Code with my own main function to understand the issue in depth.
After reading this question and the suggested solutions. In addition to reading this explanation.
I added this code to the code from the solution:
from threading import Semaphore
import threading
def threaded(fn):
def wrapper(*args, **kwargs):
threading.Thread(target=fn, args=args, kwargs=kwargs).start()
return wrapper
and before those functions from the question I added: #threaded
I added a printNumber function and main function to run it on VS Code.
def printNumber(num):
print(num, end="")
if __name__ == "__main__":
a = ZeroEvenOdd(7)
handle = a.zero(printNumber)
handle = a.even(printNumber)
handle = a.odd(printNumber)
Running this code gives me a correct answer but I do not get a new line printed in the terminal after that, I mean for input 7 in my main function, the output is: 01020304050607hostname and not what I want it to be:
01020304050607
hostname
So, I added print("\n") in the main and I saw that I get a random output like:
0102
0304050607
or
0
1020304050607
still without a new line in the end.
When I try to use the join function handle.join() then I get the error:
Exception has occurred: AttributeError 'NoneType' object has no
attribute 'join'
I tried to do this:
handle1 = a.zero(printNumber)
handle2 = a.even(printNumber)
handle3 = a.odd(printNumber)
handle1.join()
handle2.join()
handle3.join()
Still got the same error.
Where in the code should I do the waiting until the threads will terminate?
Thanks.

When I try to use...handle.join()...I get the error: "...'NoneType' object has no attribute, 'join'
The error message means that the value of handle was None at the point in your program where your code tried to call handle.join(). There is no join() operation available on the None value.
You probably wanted to join() a thread (i.e., the object returned by Threading.thread(...). For a single thread, you could do this:
t = Threading.thread(...)
t.start()
...
t.join()
Your program creates three threads, so you won't be able to just use a single variable t. You could use three separate variables, or you could create a list, or... I'll leave that up to you.

Pygame return variable from ThreadPool , local variable referenced before assignment with try, except

I'm trying to receive some data from thread, but every time it pass through exception, it not pass inside Try, i don't know what is wrong. I did it once, and i've searched every where. If someone please could help.
def receving(name, sock):
while run:
try:
tLock.acquire()
data = sock.recvfrom(1024)
except:
pass
finally:
tLock.release()
return data
host = socket.gethostbyname(socket.gethostname())
server = (host,5000)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind((host,port))
s.setblocking(0)
pool = ThreadPool(processes=1)
async_result = pool.apply_async(receving, ('arg qualquer', s))
return_val = async_result.get()
print(return_val)
run = True
while run:
return_val = async_result.get()
print(return_val)
The error message is this:
return data
UnboundLocalError: local variable 'data' referenced before assignment
I've already tried to initialize before try: but the output is the same as default, it jumps Try: same way.
Also tried to make it global but no success.

The exception you describe is very straight forward. It's all in the function at the top of your code:
def receving(name, sock):
while run:
try:
tLock.acquire()
data = sock.recvfrom(1024)
except:
pass
finally:
tLock.release()
return data
If the code in the try block causes an exception, the assignment to data won't have run. So when you try to return data later on, the local variable has no value and so it doesn't work.
It's not hard to fix that specific issue. Try putting data = None or something similar in the except clause, instead of just pass. That way, data will be defined (albeit perhaps with a value that's not very useful) regardless of whether there was an exception or not.
You should however consider tightening up the except clause so that you're not ignoring all exceptions. That's often a bad idea, since it can cause the program to run even with really broken code in it. For instance, you've never defined tLock in the code you've shown, and the try would catch the NameError caused by trying to acquire it (you'd still get an exception though when the finally clause tries to release it, so I'm guessing this isn't a real issue in your code). Normally you should specify the exception types you want to catch. I'm not exactly sure which ones would be normal for your current code, so I'll leave picking them to you.
You might also consider not having an except clause at all, if there's no reasonable result to return. That way, the exception would "bubble out" of your function and it would be the caller's responsibility to deal with it. For some kinds of exceptions (e.g. ones cause by programming bugs, not expected situations), this is usually the best way to go.
There's a lot of other weird stuff in your code though, so I'd expect you'll run into other issues after fixing the first one. For instance, you always return from the first iteration of your while loop (assuming I fixed your messed up indentation correctly), so there's not really much point in having it at all. If the return data line is actually indented less (it's at the same level as while run, then the loop will make the code inside run more than once, but it will never stop running since nothing inside it will ever change the value of the global run variable.
There may be other issues too, but it's not entirely obvious to me what you're trying to do, so I can't help with those. Multi-threaded code and network programming can be very tough to get right even for experienced programmers, so if you're new it might be a better idea to start with something simpler first.

Python module function returning immediately in thread

I've written a couple of twitter scrapers in python, and am writing another script to keep them running even if they suffer a timeout, disconnection, etc.
My current solution is as follows:
Each scraper file has a doScrape/1 function in it, which will start up a scraper and run it once, eg:
def doScrape(logger):
try:
with DBWriter(logger=logger) as db:
logger.log_info("starting", __name__)
s = PastScraper(db.getKeywords(), TwitterAuth(), db, logger)
s.run()
finally:
logger.log_info("Done", __name__)
Where run is a near-infinite loop, which won't break unless there is an exception.
In order to run one of each kind of scraper at once, I'm using this code (with a few extra imports):
from threading import Thread
class ScraperThread(Thread):
def __init__(self, module, logger):
super(ScraperThread, self).__init__()
self.module = module # Module should contain a doScrape(logger) function
self.logger = logger
def run(self):
while True:
try:
print "Starting!"
print self.module.doScrape
self.module.doScrape(self.logger)
except: # if for any reason we get disconnected, reconnect
self.logger.log_debug("Restarting scraper", __name__)
if __name__ == "__main__":
with Logger(level="all", handle=open(sys.argv[1], "a")) as l:
past = ScraperThread(PastScraper, l)
stream = ScraperThread(StreamScraper, l)
past.start()
stream.start()
past.join()
stream.join()
However, it appears that my call of doScrape from above is returning immediately, hence "Starting!" is printed in the console repeatedly, and that "Done" message in the finally block is not written to the log, whereas when run individually like so:
if __name__ == "__main__":
# Example instantiation
from Scrapers.Logging import Logger
with Logger(level="all", handle=open(sys.argv[1], "a")) as l:
doScrape(l)
The code runs forever, as expected. I'm a bit stumped.
Is there anything silly that I might have missed?

get rid of the diaper pattern in your run() method, as in: get rid of that catch-all exception handler. You'll probably get the error printed there then. I think there may be something wrong in the DBWriter or other code you're calling from your doScrape function. Perhaps it is not thread-safe. That would explain why running it from the main program directly works, but calling it from a thread fails.

Aha, solved it! It was actually that I didn't realise that a default argument (here in TwitterAuth()) is evaluated at definition time. TwitterAuth reads the API key settings from a file handle, and the default argument opens up the default config file. Since this file handle is generated at definition time, both threads had the same handle, and once one had read it, the other one tried to read from the end of the file, throwing an exception. This is remedied by resetting the file before use, and using a mutex.
Cheers to Irmen de Jong for pointing me in the right direction.

Python, run exec() on a string 'return' to exit a function or thread

I would like to run exec() on 'return' in my function, so that my function returns and stops, here is my code:
def Function():#recieves time consuming function to do
#checks if main thread should be closed, if so returns:
return 'return "it worked"'
#otherwise returns 'Null' so main thread keeps going
def MainThread():
#Wants to do some function, so
exec(Function())
return "didnt work"
when I run MainThread(), it says:
SyntaxError: 'return' outside function
And I am not sure what is wrong there, I have simplified it and found that running exec('return') will also not work in a function.
Why I want to do this:
I have a thread that controls instruments, and before each command that it sends to the instruments I want it to check if it should abort, since controlling the instruments can be time consuming, and there are safety hazards. I don't want to copy-paste an if statement many times through my code, so thought of wrapping each command to an instrument with a check.
It seems quite messy, if there are other approaches I would love to hear.
My current solution is:
def Function(stuff):#recieves time consuming function to do
#does things to stuff
return
def check(thing,skip):
if skip==true:
return
else:
Function(thing)
return
def MainThread():
skip = False #will be true or false if need to skip
#Wants to do some function, so
check("the thing to do",skip)
If its necessary to abort the thread, it actually just skips through and does nothing at each function. At the end of the thread is aborts, but I didn't like simply keeping the thread there doing nothing and hoped for a better idea :)

exec() does not execute the code in the context of the function in which it is called. The "return "it worked"" does not become a part of Function(), it is executed on it's own.

Multitask to update the file contept every second with python

Im trying to write a chat logic, so here what I want to do
def chatlogic():
talk=True
while talk:
if __name__ == '__main__':
Thread(target = fluse).start()
message = raw_input('enter a message: ')
if not message: #Ending Conversation if empty message was sent
talk=False
conv_file.write('[%s]%s: %s\n' %(msgtime,user,message))
#conv_file.flush()
return 'Conversation Ended'
def fluse():
while True:
time.sleep(1)
conv_file.fluse()
the file must update every second, nomatter what you are doing.
What am i doing wrong?
note: I have never used multitasking before

There are many problems in this code, but the one you seem to be asking about is here:
while talk:
if __name__ == '__main__':
Thread(target = fluse).start()
This means that each time through the loop—that is, once per message—you're going to fire off a new fluse thread. And that thread loops forever, calling conv_file.fluse() every second until the end of time.
So, if you type messages at 0.3, 2.7, and 5.1 seconds after app startup, you're going to get a fluse at 5.3 seconds, another at 5.7, another at 6.1, and so on.
If you want this to only happen once/second, just start a single thread, instead of starting a new one each time through the loop. For example:
if __name__ == '__main__':
Thread(target = fluse).start()
while talk:
# etc.
If, instead, you want it to happen a second after each write, but not every second again after that, just take the loop out of the function:
def fluse():
time.sleep(1)
conv_file.fluse()
Although in this case, threading.Timer is an easier way to do the same thing.
Anyway, even with this fix, as I said, there are a number of other problems:
You're calling a file method that doesn't exist—presumably you meant flush instead of fluse?
The file doesn't exist either. Maybe you meant it to be a global, created outside the functions? Or an argument to them?
You're trying to loop forever until an empty message, but you call return each time through the loop, which means you're only going to loop once.
There's no way to exit your program, because you fire off a non-daemon background thread that runs forever.
You never call the chatlogic function anyway, so the program just exits immediately without doing any work.
If I fix all of the other errors in your program, and add something that shows me whether the fluse thread is doing its job, it does something… and maybe you can tell us whether it's the thing you wanted.
from threading import Thread
import time
def chatlogic(conv_file):
user = 'Nobody'
t = Thread(target=flusher, args=[conv_file])
t.daemon=True
t.start()
while True:
message = raw_input('enter a message: ')
if not message: #Ending Conversation if empty message was sent
break
msgtime = time.asctime()
conv_file.write('[%s]%s: %s\n' %(msgtime,user,message))
return 'Conversation Ended'
def flusher(conv_file):
while True:
time.sleep(1)
print 'Flushing the file now'
conv_file.flush()
if __name__ == '__main__':
conv_file = open('convfile.txt', 'w')
chatlogic(conv_file)
This is still a strange design. The file is being flushed once/second even if you didn't write anything, and that there's no guarantee it actually gets flushed at the end, and so on. But this is what it sounds like you were attempting to do.