Python: subclassing run method of threading gives an error - python

I encountered weird behavior when subclassing Thread of the threading-module of Python 2.7.3.
Consider the next code, called test.py:
import threading
def target_function(): print 'Everything OK'
class SpecificThread(threading.Thread):
def run(self):
try:
if self.__target:
self.__target(*self.__args, **self.__kwargs)
finally:
# Avoid a refcycle if the thread is running a function with
# an argument that has a member that points to the thread.
del self.__target, self.__args, self.__kwargs
def check():
thread = SpecificThread(target=target_function)
#thread = threading.Thread(target=target_function)
thread.run()
print thread.name, 'is running:', thread.is_alive()
This code raises the following error when check() is run:
>>> check()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "test.py", line 18, in check
thread.run()
File "test.py", line 13, in run
del self.__target, self.__args, self.__kwargs
AttributeError: _SpecificThread__target
Although, the run() method of SpecificThread is exactly the same as the code in the original threading.py module.
If threading.Thread is used or when SpecificThread does not overwrite the run() method, the script runs flawless. I do not understand why overwriting does not work, considering that the Python documentation states that it is allowed.
Thanks!

The thing you've encountered is called name mangling in Python.
It means that all non-system attributes (attributes like "__attrname__") starting with double underscore are automatically renamed by interpreter as _Classname__attrname). That's a kind of protection mechanizm and such design usually means that you souldn't even touch those fields (they are already handled in a proper way), and usually referred to as "private fields".
So, if you want for some reason to get to those fields, use notation above:
self._Thread__target
Note, that field starts with _Thread, not with _SpecificThread, because this attribute was defined in Thread class.

Related

Using Thread.join() method with threads that inside inside a class

I am solving this question from LeetCode: 1116. Print Zero Even Odd
I am running this solution in VS Code with my own main function to understand the issue in depth.
After reading this question and the suggested solutions. In addition to reading this explanation.
I added this code to the code from the solution:
from threading import Semaphore
import threading
def threaded(fn):
def wrapper(*args, **kwargs):
threading.Thread(target=fn, args=args, kwargs=kwargs).start()
return wrapper
and before those functions from the question I added: #threaded
I added a printNumber function and main function to run it on VS Code.
def printNumber(num):
print(num, end="")
if __name__ == "__main__":
a = ZeroEvenOdd(7)
handle = a.zero(printNumber)
handle = a.even(printNumber)
handle = a.odd(printNumber)
Running this code gives me a correct answer but I do not get a new line printed in the terminal after that, I mean for input 7 in my main function, the output is: 01020304050607hostname and not what I want it to be:
01020304050607
hostname
So, I added print("\n") in the main and I saw that I get a random output like:
0102
0304050607
or
0
1020304050607
still without a new line in the end.
When I try to use the join function handle.join() then I get the error:
Exception has occurred: AttributeError 'NoneType' object has no
attribute 'join'
I tried to do this:
handle1 = a.zero(printNumber)
handle2 = a.even(printNumber)
handle3 = a.odd(printNumber)
handle1.join()
handle2.join()
handle3.join()
Still got the same error.
Where in the code should I do the waiting until the threads will terminate?
Thanks.
When I try to use...handle.join()...I get the error: "...'NoneType' object has no attribute, 'join'
The error message means that the value of handle was None at the point in your program where your code tried to call handle.join(). There is no join() operation available on the None value.
You probably wanted to join() a thread (i.e., the object returned by Threading.thread(...). For a single thread, you could do this:
t = Threading.thread(...)
t.start()
...
t.join()
Your program creates three threads, so you won't be able to just use a single variable t. You could use three separate variables, or you could create a list, or... I'll leave that up to you.

Python program bombs out after finishing

I'm getting a strange Python error. I'm executing a file that looks like this.
if __name__ == '__main__':
MyClass().main()
print('Done 1')
print('Done 2')
The preceding runs successfully. But when I change it to this, I get the strange result.
if __name__ == '__main__':
myObject = MyClass()
myObject.main()
print('Done 1')
print('Done 2')
The output looks like this.
Done 1
Done 2
Exception ignored in: <function Viewer.__del__ at 0x0000021569EF72F0>
Traceback (most recent call last):
File "C:\...\lib\site-packages\gym\envs\classic_control\rendering.py", line 143, in __del__
File "C:\...\lib\site-packages\gym\envs\classic_control\rendering.py", line 62, in close
File "C:\...\lib\site-packages\pyglet\window\win32\__init__.py", line 305, in close
File "C:\...\lib\site-packages\pyglet\window\__init__.py", line 770, in close
ImportError: sys.meta_path is None, Python is likely shutting down
Process finished with exit code 0
There is a blank line after the final print line. The same thing happens when the final line does not have an end-of-line marker.
I get the same result whether I run it from within PyCharm using the run command or from the terminal.
As you can probably tell from the error lines, the program generates an animation. (It's the cart-pole problem from OpenAI gym.)
Since the program completes before the error, it's not a disaster. But I'd like to understand what's happening.
Thanks.
Python provides a __del__ dunder method for classes that will be called as the instances are garbage collected, if they're garbage collected.
When it's used, the __del__ method typically performs some sort of cleanup.
Due to the fact that it's fairly easy to inadvertently prevent an object from being collected, reliance on the __del__ to perform cleanup (instead of say, a context manager's __exit__ or an explicit .close() method) is generally advised against.
Your error highlights a different reason for avoiding relying on __del__, however: that during shutdown __del__ will be called but possibly after other things that it relies on are freed.
The proposed workarounds on the github issue linked in the comments should be instructive, as they all ensure that the cleanup is done at a time where the things that that cleanup relies on (e.g. sys.meta_path) are still in defined/not yet freed, e.g.:
try:
del env
except ImportError:
pass
and
env = gym.make('CartPole-v0')
...
env.env.close()
and (likely, but much less efficient or clear)
import gc; gc.collect()

Error im receiving in Python (Tkinter)

Currently trying to construct a program with multiple windows (Main screen -> A Landlord / Customer section -> calendar/calculator ect)
I am very much a beginner at this moment in time, i keep coming across two errors:
Exception in Tkinter callback Traceback (most recent call last):
File
"C:\Users\HP\AppData\Local\Programs\Python\Python35-32\lib\idlelib\run.py",
line 119, in main
seq, request = rpc.request_queue.get(block=True, timeout=0.05) File
"C:\Users\HP\AppData\Local\Programs\Python\Python35-32\lib\queue.py",
line 172, in get
raise Empty queue.Empty
Also another query; a error i receive a lot is how to define "Self" ("self is not defined")
EDIT
My code is very much dis-functional - i think looking at my code will probably give you a heart attack. When running the code i want there to be 1 screen at one time, currently 3 come up at the start, im assuming this is too me using wrong inheritance or something
It was too big to place in here so you can easily view the code here
http://textuploader.com/525p5
To be honest any help will really be appreciated. My first time doing something complex on python such as a working program with features such as a calendar, calculator ect
Cheers
Ross
File "C:\Users\HP\AppData\Local\Programs\Python\Python35-32\lib\idlelib\run.py",
line 119, in main seq, request = rpc.request_queue.get(block=True, timeout=0.05)
File "C:\Users\HP\AppData\Local\Programs\Python\Python35-32\lib\queue.py", line 172,
in get raise Empty queue.Empty
these two lines
request = rpc.request_queue.get(block=True, timeout=0.05)
raise Empty queue.Empty
Suggest you are trying to get data from an empty queue, and an exception is raised. The proper way to deal with this is to put this code in a try..except block, catch the exception and deal with it accordingly.
Here is a decent tutorial on this matter.
try:
....
request = rpc.request_queue.get(block=True, timeout=0.05)
....
except Exception,e:
# this will cath all exceptions derived from the exception class
About your second query, I strongly advise you to post a new question instead of bundling them up.
But I'll try and give some advice: self is used to address an instance of a class, it's the current object in use. You can't 'define' self, you use it in a class implementation to tell python that your method is to be used with a specific instance, not the global scope.
class demo:
def __init__(self):
self.a = 5
def foo(self):
self.a = 6
def global_foo():
print 'global_foo'

Why is using a while loop better?

I am making a program that uses a loop to run forever and I have a snippet of code below to show you how I acheive the loop. This is just an example and not the actual program. It is the same idea though.
import time
def start():
print "hello"
time.sleep(0.2)
start()
start()
All of my programmer friends tell me not to do this, and use a while loop instead. Like this:
import time
def start():
while True:
print "Hello"
time.sleep(0.2)
start()
Why should I use the while loop instead when both methods work perfectly fine?
Each time you are recursing, you are pushing a frame context onto your program stack. Soon you would have used up your entire allotted stack space causing stackoverflow, no pun intended ;)
The second approach has no such flaws. Hence by the looks of it 2nd is better than 1st approach (unless more of the program is presented).
If you run the program continuously with the recursion, you will get RuntimeError:
Traceback (most recent call last):
File "t.py", line 8, in <module>
start()
File "t.py", line 7, in start
start()
File "t.py", line 7, in start
start()
...
File "t.py", line 7, in start
start()
RuntimeError: maximum recursion depth exceeded
>>> import sys
>>> sys.getrecursionlimit()
1000
the while loop amd recursion both have their advantages and disadvantages but while loop is just a test and conditional jump. wheras ecursion involves pushing a stack frame, jumping, returning, and popping back from the stack.So they might have preferred while loop :)
Python does not have Tail Call Optimization (TCO) and cannot elide the stack. As such, recursion should not be used for an unbound depth (in languages without TCO) lest you get a stack-overflow error!
While the current code might appear to be fine, consider this simple change which will quickly reveal a problem.
def start():
print "hello"
# let's go faster! -- time.sleep(0.2)
start()
start()

Share objects with file handle attribute between processes

I have a question about shared resource with file handle between processes.
Here is my test code:
from multiprocessing import Process,Lock,freeze_support,Queue
import tempfile
#from cStringIO import StringIO
class File():
def __init__(self):
self.temp = tempfile.TemporaryFile()
#print self.temp
def read(self):
print "reading!!!"
s = "huanghao is a good boy !!"
print >> self.temp,s
self.temp.seek(0,0)
f_content = self.temp.read()
print f_content
class MyProcess(Process):
def __init__(self,queue,*args,**kwargs):
Process.__init__(self,*args,**kwargs)
self.queue = queue
def run(self):
print "ready to get the file object"
self.queue.get().read()
print "file object got"
file.read()
if __name__ == "__main__":
freeze_support()
queue = Queue()
file = File()
queue.put(file)
print "file just put"
p = MyProcess(queue)
p.start()
Then I get a KeyError like below:
file just put
ready to get the file object
Process MyProcess-1:
Traceback (most recent call last):
File "D:\Python26\lib\multiprocessing\process.py", line 231, in _bootstrap
self.run()
File "E:\tmp\mpt.py", line 35, in run
self.queue.get().read()
File "D:\Python26\lib\multiprocessing\queues.py", line 91, in get
res = self._recv()
File "D:\Python26\lib\tempfile.py", line 375, in __getattr__
file = self.__dict__['file']
KeyError: 'file'
I think when I put the File() object into queue , the object got serialized, and file handle can not be serialized, so, i got the KeyError:
Anyone have any idea about that? if I want to share objects with file handle attribute, what should I do?
I have to object (at length, won't just fit in a commentl;-) to #Mark's repeated assertion that file handles just can't be "passed around between running processes" -- this is simply not true in real, modern operating systems, such as, oh, say, Unix (free BSD variants, MacOSX, and Linux, included -- hmmm, I wonder what OS's are left out of this list...?-) -- sendmsg of course can do it (on a "Unix socket", by using the SCM_RIGHTS flag).
Now the poor, valuable multiprocessing is fully right to not exploit this feature (even assuming there might be black magic to implement it on Windows too) -- most developers would no doubt misuse it anyway (having multiple processes access the same open file concurrently and running into race conditions). The only proper way to use it is for a process which has exclusive rights to open certain files to pass the opened file handles to another process which runs with reduced privileges -- and then never use that handle itself again. No way to enforce that in the multiprocessing module, anyway.
Back to #Andy's original question, unless he's going to work on Linux only (AND with local processes only, too) and willing to play dirty tricks with the /proc filesystem, he's going to have to define his application-level needs more sharply and serialize file objects accordingly. Most files have a path (or can be made to have one: path-less files are pretty rare, actually non-existent on Windows I believe) and thus can be serialized via it -- many others are small enough to serialize by sending their content over -- etc, etc.

Categories