Below is the code. I did not manually del object reference but then also in VScode __del__ gets invoked. But the same piece of code is working in Jupyter Notebook. This means, __del__ is not called automatically. Is there any problem with the code? Confused. Does VScode handle Garbage Collector (automatically)? Googled a lot, but did not get an appropriate answer.
class Student:
# constructor
def __init__(self, name):
print('Inside Constructor')
self.name = name
print('Object initialized')
def show(self):
print('Hello, my name is', self.name)
# destructor
def __del__(self):
print('Inside destructor')
print('Object destroyed')
# create object
s1 = Student('Emma')
s2 = s1;
s3 = s1;
print("Id of S1: " , id(s1));
It's better to call __del__ a finalizer method because this method gets called exactly when the object is about to be garbage collected. (like when all references to that object are gone.)
In VSCode or even when you run your script in terminal, when the interpreter reaches the end of the file it will terminate the execution and the process, therefore any objects will be destroyed. So you can see that this method gets called. So even if you didn't del the references to the object explicitly, all the references will be gone at the end.
Jupyter notebook on the other hand is still running, and also maybe it keeps a reference to your object for internal works...
Put a simple input() in the last line of your code, you can see that the __del__ wont get called in VScode or terminal as well.
Related
I have created a class A using the following pattern
class A:
def __init__(self):
self.worker = threading.Thread(target=self.workToDo)
self.worker.setDaemon(daemonic=True)
self.worker.start()
def workToDo(self):
while True:
print("Work")
However, this design gets not garbage collected. I assume that this is due to a circular dependency between the running thread and its parent.
How can i design a class that starts a periodic thread that does some work, stops this thread on destruction and gets destructed as soon as all obvious references to the parent object get out of scope.
I tried to stop the thread in the ___del___ method, but this method is never called (i assume due to the circular dependency).
There is no circular dependence, and the garbage collector is doing exactly what it is supposed to do. Look at the method workToDo:
def workToDo(self):
while True:
print("Work")
Once you start the thread, this method will run forever. It contains a variable named self: the instance of class A that originally launched the thread. As long as this method continues to run, there is an active reference to the instance of A and therefore it cannot be garbage collected.
This can easily be demonstrated with the following little program:
import threading
import time
def workToDo2():
while True:
print("Work2")
time.sleep(0.5)
class A:
def __init__(self):
self.worker = threading.Thread(target=workToDo2, daemon=True)
self.worker.start()
def workToDo(self):
while True:
print("Work")
time.sleep(0.5)
def __del__(self):
print("del")
A()
time.sleep(5.0)
If you change the function that starts the thread from self.workToDo to workToDo2, the __del__ method fires almost immediately. In that case the thread does not reference the object created by A(), so it can be safely garbage collected.
Your statement of the problem is based on a false assumption about how the garbage collector works. There is no such concept as "obvious reference" - there is either a reference or there isn't.
The threads continue to run whether the object that launched them is garbage collected or not. You really should design Python threads so there is a mechanism to exit from them cleanly, unless they are true daemons and can continue to run without harming anything.
I understand the urge to avoid trusting your users to call some sort of explicit close function. But the Python philosophy is "we're all adults here," so IMO this problem is not a good use of your time.
Syntax of destructor declaration:
def __del__(self):
# body of destructor
Note: A reference to objects is also deleted when the object goes out of reference or when the program ends.
Example 1: Here is the simple example of destructor. By using del keyword we deleted the all references of object ‘obj’, therefore destructor invoked automatically
Python program to illustrate destructor:
class Employee:
# Initializing
def __init__(self):
print('Employee created.')
# Deleting (Calling destructor)
def __del__(self):
print('Destructor called, Employee deleted.')
obj = Employee()
del obj
I'm running this code:
import random
class Pound:
def __init__(self):
self.value=1.00
self.color="gold"
self.num_ages=1
self.diameter=22.5
self.thickness=3.15
self.heads=True
self.num_pocket=10
def __del__(self):
print("Coin spent!")
def rust(self):
self.color="greenish"
def clean(self):
self.color="clean"
def flip(self):
heads_options=[True,False]
choice=random.choice(heads_options)
self.heads=choice
coin1=Pound()
print(coin1.value)
When I run it in the Python IDLE I have no issues whatsoever. But when I run it in ATOM (using the atom-python-run or the script package) it always shows me this output:
1.0
Coin spent!
The destructor seems to call itself. I have no clue how to solve the issues and I haven't found anything asked about this anywhere.
In this code, when interpreted as a script, the object goes out of scope right after print since after that, the program ends. __del__ is called in that case because at that point, "the instance is about to be destroyed" (docs).
When run from a REPL though (like IDLE), it doesn't go out of scope until you kill the REPL, so __del__ isn't called right away.
This is the expected behavior.
Also note, the docs don't prefer the word "destructor":
This is also called a finalizer or (improperly) a destructor.
I have an large script where i found out that lot of connections to a machine are left open and the reason was that for one of the class destructor was never getting called.
below is a simplified version of script manifesting the issue.
I tiered searching around and found out that it could be because of GC and weakref does help but in this case no help.
2 cases where i can see that the destructor is getting called are
If i call B_class object without passing A_class function
self.b = B_class("AA")
I call the make the B_class objects not global i.e not use self
b = B_class("AA",self.myprint)
b.do_something()
Both of these cases will cause further issues for my case. Last resort will be to close/del the objects at the end myself but i don't want to go that way.
can anybody suggest a better way out of this and help me understand this issue? Thanks in advance.
import weakref
class A_class:
def __init__(self,debug_level=1,version=None):
self.b = B_class("AA",self.myprint)
self.b.do_something()
def myprint(self, text):
print text
class B_class:
def __init__(self,ip,printfunc=None):
self.ip=ip
self.new_ip =ip
#self.printfunc = printfunc
self.printfunc = weakref.ref(printfunc)()
def __del__(self):
print("##B_Class Destructor called##")
def do_something(self,timeout=120):
self.myprint("B_Class ip=%s!!!" % self.new_ip)
def myprint(self,text):
if self.printfunc:
print ("ExtenalFUNC:%s" %text)
else:
print ("JustPrint:%s" %text)
def _main():
a = A_class()
if __name__ == '__main__':
_main()
You're not using the weakref.ref object properly. You're calling it immediately after it is created, which returns the referred-to object (the function passed in as printref).
Normally, you'd want to save the weak reference and only call it when you're going to use the reffered-to object (e.g. in myprint). However, that won't work for the bound method self.myprint you're getting passed in as printfunc, since the bound method object doesn't have any other references (every access to a method creates a new object).
If you're using Python 3.4 or later and you know that the object passed in will always be a bound method, you can use the WeakMethod class, rather than a regular ref. If you're not sure what kind of callable you're going to get, you might need to do some type checking to see if WeakMethod is required or not.
Use Python's "with" statement (http://www.python.org/dev/peps/pep-0343/).
It creates a syntactic scope and the __exit__ function which it creates is guaranteed to get called as soon as execution leaves the scope. You can also emulate "__enter__/__exit__" behavior by creating a generator with "contextmanager" decorator from the contextlib module (python 2.6+ or 2.5 using "from __future__ import with_statement" see PEP for examples).
Here's an example from the PEP:
import contextlib
#contextlib.contextmanger
def opening(filename):
f = open(filename) # IOError is untouched by GeneratorContext
try:
yield f
finally:
f.close() # Ditto for errors here (however unlikely)
and then in your main code, you write
with opening(blahblahblah) as f:
pass
# use f for something
# here you exited the with scope and f.close() got called
In your case, you'll want to use a different name (connecting or something) instead of "opening" and do socket connecting/disconnecting inside of your context manager.
self.printfunc = weakref.ref(printfunc)()
isn't actually using weakrefs to solve your problem; the line is effectively a noop. You create a weakref with weakref.ref(printfunc), but you follow it up with call parens, which converts back from weakref to a strong ref which you store (and the weakref object promptly disappears). Apparently it's not possible to store a weakref to the bound method itself (because the bound method is its own object created each time it's referenced on self, not a cached object whose lifetime is tied to self), so you have to get a bit hacky, unbinding the method so you can take a weakref on the object itself. Python 3.4 introduced WeakMethod to simplify this, but if you can't use that, then you're stuck doing it by hand.
Try changing it to (on Python 2.7, and you must import inspect):
# Must special case printfunc=None, since None is not weakref-able
if printfunc is None:
# Nothing provided
self.printobjref = self.printfuncref = None
elif inspect.ismethod(printfunc) and printfunc.im_self is not None:
# Handling bound method
self.printobjref = weakref.ref(printfunc.im_self)
self.printfuncref = weakref.ref(printfunc.im_func)
else:
self.printobjref = None
self.printfuncref = weakref.ref(printfunc)
and change myprint to:
def myprint(self,text):
if self.printfuncref is not None:
printfunc = self.printfuncref()
if printfunc is None:
self.printfuncref = self.printobjref = None # Ref died, so clear it to avoid rechecking later
elif self.printobjref is not None:
# Bound method not known to have disappeared
printobj = self.printobjref()
if printobj is not None:
print ("ExtenalFUNC:%s" %text) # To call it instead of just saying you have it, do printfunc(printobj, text)
return
self.printobjref = self.printfuncref = None # Ref died, so clear it to avoid rechecking later
else:
print ("ExtenalFUNC:%s" %text) # To call it instead of just saying you have it, do printfunc(text)
return
print ("JustPrint:%s" %text)
Yeah, it's ugly. You could factor out the ugly if you like (borrowing the implementation of WeakMethod from Python 3.4's source code would make sense, but names would have to change; __self__ is im_self in Py2, __func__ is im_func), but it's unpleasant even so. It's definitely not thread safe if the weakrefs could actually go dark, since the checks and clears of the weakref members aren't protected.
I'm trying to understand how the Python garbage collector functions and if there is anything I can do to control when an object is collected. I wrote this test:
>>> class Test:
... def __del__(self):
... print 'Delete ' + str(self)
...
>>> def fun():
... return Test()
...
>>> fun()
<__main__.Test instance at 0x0000000002989E48>
>>> fun()
Delete <__main__.Test instance at 0x0000000002989E48>
<__main__.Test instance at 0x00000000023E2488>
>>> fun()
Delete <__main__.Test instance at 0x00000000023E2488>
<__main__.Test instance at 0x0000000002989C48>
As you can see, the Test instance, although I do not keep an instance to it, is not deleted until the next time I call fun. Is this simply an accident (could it have been deleted at any other point) or is there a specific reason why it is deleted only when I call fun again? Is there anything I can do to ensure it gets deleted if I don't keep a reference to it?
The "contact" of the Python garbage collector (like all garbage collectors) is that it will release an object sometime after the last reachable reference to it disappears.
Because CPython uses reference counting, as an implementation detail it will release most garbage objects (specifically non-cyclic objects) immediately after the last reachable reference to them disappears. This is not a guarantee of the Python language, and is not true of other Python implementations like PyPy, Jython, IronPython, so relying on it is generally considered to be poor practice.
In your case, what you're observing with the object being collected after the function is called again has little to do with the behaviour of the garbage collector, but is rather due to the way the interactive interpreter shell works.
When you evaluate an expression in the interactive prompt, the resulting value is automatically saved in the variable _, so you can get it back if you discover that you still want it only after you've seen it printed. So after your fun() calls, there is still a reference to the return value. Then when you evaluate another expression (anything else, it doesn't have to involve fun again), _ is overwritten with the new value, allowing the old one to be garbage collected.
This only happens for expressions directly entered at the interactive prompt, so it won't delay collection of objects within functions or when your Python code is imported or run as a script.
Try explicitly calling del on the returned value:
returned_value = fun()
del returned_value
But finalizers like __del__ can be problematic; as you have already seen, one issue is that when they get called is not deterministic. Also, it is possible within a finalizer to reinstantiate a deleted object, such as sticking a reference to it in a global list.
If you need to release resources (besides just raw memory) - things like unlocking locks, closing files, or releasing database connections, use a context manager, and bound its life span using the with statement. Many of these resource are already context managers. For example, a threading.Lock can be locked and unlocked implicitly using with:
# "with" statement will call the __enter__ method of self.lock,
# which will block until self.lock can be locked
with self.lock:
# do thread-synchronized stuff here
# self.lock is automatically released here - at then end of
# the "with" block, the lock's __exit__ method is called, which
# releases the lock. This will get called even if the block is
# exited by a raised exception
I have a threaded program in Python that works fine except that __del__ does not get called once the thread is running:
class tt(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.stop_event = threading.Event()
def __del__(self):
print "Deleting"
self.stop_event.set()
time.sleep(5)
def run(self):
self.stop_event.clear()
while not self.stop_event.isSet():
self.do_stuff()
threading.Thread.__init__(self)
def stop(self):
self.stop_event.set()
For example if I do
tc = aa()
del tc
It works fine (I get the deleting message pause etc). However if I do:
tc = aa()
tc.start()
del tc
Then the __del__ does not get run (Note it does execute __del__ when I do tc.start(); tc.stop(); del tc.
I'm running this in ipython
The __del__() method is executed when an instance is garbage collected, not when you call del on a name that happens to point to the instance. The latter will only delete the name (which might in some cases result in the instance pointed to by the name being gc'ed).
A threading.Thread instance will never be garbage collected while the thread is still running, since the instance is still in use in this case. And ditching a name that happens to point to the instance certainly won't stop the thread.
See the documentation:
del x doesn’t directly call x.__del__() — the former decrements the reference count for x by one, and the latter is only called when x‘s reference count reaches zero.
In your case, the thread is still running and there is still some (internal) reference to it. Therefore it has a reference count greater than one, and still greater than zero when you delete it from your module namespace. Since it is not garbage-collected, __del__ is not called.