I have a threaded program in Python that works fine except that __del__ does not get called once the thread is running:
class tt(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.stop_event = threading.Event()
def __del__(self):
print "Deleting"
self.stop_event.set()
time.sleep(5)
def run(self):
self.stop_event.clear()
while not self.stop_event.isSet():
self.do_stuff()
threading.Thread.__init__(self)
def stop(self):
self.stop_event.set()
For example if I do
tc = aa()
del tc
It works fine (I get the deleting message pause etc). However if I do:
tc = aa()
tc.start()
del tc
Then the __del__ does not get run (Note it does execute __del__ when I do tc.start(); tc.stop(); del tc.
I'm running this in ipython
The __del__() method is executed when an instance is garbage collected, not when you call del on a name that happens to point to the instance. The latter will only delete the name (which might in some cases result in the instance pointed to by the name being gc'ed).
A threading.Thread instance will never be garbage collected while the thread is still running, since the instance is still in use in this case. And ditching a name that happens to point to the instance certainly won't stop the thread.
See the documentation:
del x doesn’t directly call x.__del__() — the former decrements the reference count for x by one, and the latter is only called when x‘s reference count reaches zero.
In your case, the thread is still running and there is still some (internal) reference to it. Therefore it has a reference count greater than one, and still greater than zero when you delete it from your module namespace. Since it is not garbage-collected, __del__ is not called.
Related
I have created a class A using the following pattern
class A:
def __init__(self):
self.worker = threading.Thread(target=self.workToDo)
self.worker.setDaemon(daemonic=True)
self.worker.start()
def workToDo(self):
while True:
print("Work")
However, this design gets not garbage collected. I assume that this is due to a circular dependency between the running thread and its parent.
How can i design a class that starts a periodic thread that does some work, stops this thread on destruction and gets destructed as soon as all obvious references to the parent object get out of scope.
I tried to stop the thread in the ___del___ method, but this method is never called (i assume due to the circular dependency).
There is no circular dependence, and the garbage collector is doing exactly what it is supposed to do. Look at the method workToDo:
def workToDo(self):
while True:
print("Work")
Once you start the thread, this method will run forever. It contains a variable named self: the instance of class A that originally launched the thread. As long as this method continues to run, there is an active reference to the instance of A and therefore it cannot be garbage collected.
This can easily be demonstrated with the following little program:
import threading
import time
def workToDo2():
while True:
print("Work2")
time.sleep(0.5)
class A:
def __init__(self):
self.worker = threading.Thread(target=workToDo2, daemon=True)
self.worker.start()
def workToDo(self):
while True:
print("Work")
time.sleep(0.5)
def __del__(self):
print("del")
A()
time.sleep(5.0)
If you change the function that starts the thread from self.workToDo to workToDo2, the __del__ method fires almost immediately. In that case the thread does not reference the object created by A(), so it can be safely garbage collected.
Your statement of the problem is based on a false assumption about how the garbage collector works. There is no such concept as "obvious reference" - there is either a reference or there isn't.
The threads continue to run whether the object that launched them is garbage collected or not. You really should design Python threads so there is a mechanism to exit from them cleanly, unless they are true daemons and can continue to run without harming anything.
I understand the urge to avoid trusting your users to call some sort of explicit close function. But the Python philosophy is "we're all adults here," so IMO this problem is not a good use of your time.
Syntax of destructor declaration:
def __del__(self):
# body of destructor
Note: A reference to objects is also deleted when the object goes out of reference or when the program ends.
Example 1: Here is the simple example of destructor. By using del keyword we deleted the all references of object ‘obj’, therefore destructor invoked automatically
Python program to illustrate destructor:
class Employee:
# Initializing
def __init__(self):
print('Employee created.')
# Deleting (Calling destructor)
def __del__(self):
print('Destructor called, Employee deleted.')
obj = Employee()
del obj
Below is the code. I did not manually del object reference but then also in VScode __del__ gets invoked. But the same piece of code is working in Jupyter Notebook. This means, __del__ is not called automatically. Is there any problem with the code? Confused. Does VScode handle Garbage Collector (automatically)? Googled a lot, but did not get an appropriate answer.
class Student:
# constructor
def __init__(self, name):
print('Inside Constructor')
self.name = name
print('Object initialized')
def show(self):
print('Hello, my name is', self.name)
# destructor
def __del__(self):
print('Inside destructor')
print('Object destroyed')
# create object
s1 = Student('Emma')
s2 = s1;
s3 = s1;
print("Id of S1: " , id(s1));
It's better to call __del__ a finalizer method because this method gets called exactly when the object is about to be garbage collected. (like when all references to that object are gone.)
In VSCode or even when you run your script in terminal, when the interpreter reaches the end of the file it will terminate the execution and the process, therefore any objects will be destroyed. So you can see that this method gets called. So even if you didn't del the references to the object explicitly, all the references will be gone at the end.
Jupyter notebook on the other hand is still running, and also maybe it keeps a reference to your object for internal works...
Put a simple input() in the last line of your code, you can see that the __del__ wont get called in VScode or terminal as well.
I'm running this code:
import random
class Pound:
def __init__(self):
self.value=1.00
self.color="gold"
self.num_ages=1
self.diameter=22.5
self.thickness=3.15
self.heads=True
self.num_pocket=10
def __del__(self):
print("Coin spent!")
def rust(self):
self.color="greenish"
def clean(self):
self.color="clean"
def flip(self):
heads_options=[True,False]
choice=random.choice(heads_options)
self.heads=choice
coin1=Pound()
print(coin1.value)
When I run it in the Python IDLE I have no issues whatsoever. But when I run it in ATOM (using the atom-python-run or the script package) it always shows me this output:
1.0
Coin spent!
The destructor seems to call itself. I have no clue how to solve the issues and I haven't found anything asked about this anywhere.
In this code, when interpreted as a script, the object goes out of scope right after print since after that, the program ends. __del__ is called in that case because at that point, "the instance is about to be destroyed" (docs).
When run from a REPL though (like IDLE), it doesn't go out of scope until you kill the REPL, so __del__ isn't called right away.
This is the expected behavior.
Also note, the docs don't prefer the word "destructor":
This is also called a finalizer or (improperly) a destructor.
I have a python class object and I want to assign the value of one class variable
class Groupclass(Workerclass):
"""worker class"""
count = 0
def __init__(self):
"""initialize time"""
Groupclass.count += 1
self.membercount = 0;
self.members = []
def __del__(self):
"""delte a worker data"""
Groupclass.count -= 1
if __name__ == "__main__":
group1 = Groupclass()
This execution result is correct, but there's an error message that says:
Exception AttributeError: "'NoneType' object has no attribute 'count'" in <bound method Groupclass.__del__ of <__main__.Groupclass instance at 0x00BA6710>> ignored
Can someone tell me what me I did wrong?
Your __del__ method assumes that the class is still present by the time it is called.
This assumption is incorrect. Groupclass has already been cleared when your Python program exits and is now set to None.
Test if the global reference to the class still exists first:
def __del__(self):
if Groupclass:
Groupclass.count -= 1
or use type() to get the local reference:
def __del__(self):
type(self).count -= 1
but do note that this means that the semantics for count change if Groupclass is subclassed (each subclass gets a .count attribute versus only Groupclass having a .count attribute).
Quoting from the __del__ hook documentation:
Warning: Due to the precarious circumstances under which __del__() methods are invoked, exceptions that occur during their execution are ignored, and a warning is printed to sys.stderr instead. Also, when __del__() is invoked in response to a module being deleted (e.g., when execution of the program is done), other globals referenced by the __del__() method may already have been deleted or in the process of being torn down (e.g. the import machinery shutting down). For this reason, __del__() methods should do the absolute minimum needed to maintain external invariants. Starting with version 1.5, Python guarantees that globals whose name begins with a single underscore are deleted from their module before other globals are deleted; if no other references to such globals exist, this may help in assuring that imported modules are still available at the time when the __del__() method is called.
If you are using Python 3, two additional notes apply:
CPython 3.3 automatically applies a randomized hash salt to the str keys used in a globals dictionary; this also affects the order in which globals are cleaned up, and it could be that you see the problem on only some of the runs.
CPython 3.4 no longer sets globals to None (in most cases), as per Safe Object Finalization; see PEP 442.
When __del__() method called, the Groupclass may be recovered by the garbage collection mechanism, so using Groupclass.xxx may failed. But you can access the count variable through self.__class__.count. Code likes below:
def __del__(self):
self.__class__.count -= 1
I'm trying to understand how the Python garbage collector functions and if there is anything I can do to control when an object is collected. I wrote this test:
>>> class Test:
... def __del__(self):
... print 'Delete ' + str(self)
...
>>> def fun():
... return Test()
...
>>> fun()
<__main__.Test instance at 0x0000000002989E48>
>>> fun()
Delete <__main__.Test instance at 0x0000000002989E48>
<__main__.Test instance at 0x00000000023E2488>
>>> fun()
Delete <__main__.Test instance at 0x00000000023E2488>
<__main__.Test instance at 0x0000000002989C48>
As you can see, the Test instance, although I do not keep an instance to it, is not deleted until the next time I call fun. Is this simply an accident (could it have been deleted at any other point) or is there a specific reason why it is deleted only when I call fun again? Is there anything I can do to ensure it gets deleted if I don't keep a reference to it?
The "contact" of the Python garbage collector (like all garbage collectors) is that it will release an object sometime after the last reachable reference to it disappears.
Because CPython uses reference counting, as an implementation detail it will release most garbage objects (specifically non-cyclic objects) immediately after the last reachable reference to them disappears. This is not a guarantee of the Python language, and is not true of other Python implementations like PyPy, Jython, IronPython, so relying on it is generally considered to be poor practice.
In your case, what you're observing with the object being collected after the function is called again has little to do with the behaviour of the garbage collector, but is rather due to the way the interactive interpreter shell works.
When you evaluate an expression in the interactive prompt, the resulting value is automatically saved in the variable _, so you can get it back if you discover that you still want it only after you've seen it printed. So after your fun() calls, there is still a reference to the return value. Then when you evaluate another expression (anything else, it doesn't have to involve fun again), _ is overwritten with the new value, allowing the old one to be garbage collected.
This only happens for expressions directly entered at the interactive prompt, so it won't delay collection of objects within functions or when your Python code is imported or run as a script.
Try explicitly calling del on the returned value:
returned_value = fun()
del returned_value
But finalizers like __del__ can be problematic; as you have already seen, one issue is that when they get called is not deterministic. Also, it is possible within a finalizer to reinstantiate a deleted object, such as sticking a reference to it in a global list.
If you need to release resources (besides just raw memory) - things like unlocking locks, closing files, or releasing database connections, use a context manager, and bound its life span using the with statement. Many of these resource are already context managers. For example, a threading.Lock can be locked and unlocked implicitly using with:
# "with" statement will call the __enter__ method of self.lock,
# which will block until self.lock can be locked
with self.lock:
# do thread-synchronized stuff here
# self.lock is automatically released here - at then end of
# the "with" block, the lock's __exit__ method is called, which
# releases the lock. This will get called even if the block is
# exited by a raised exception