I am trying this approach to delete an object in python. I read the documentation of Python stating that garbage collector will automatically delete the object that is not referenced.
def check():
class newOb():
def __init__(self,value):
self.value = value
print self.value
return None
class ob:
ins = {}
def cr(self,someuniqueid,value) :
newV = newOb(value)
ob.ins[someuniqueid] = newV ## saving this object refernce to the ob class ins dictionary
return newV
#### Accessing Object ###
someuniqueid = 12
c = ob()
d = c.cr(someuniqueid,123)
print d.value ## will print 123
# now deleting the associated object
del c.ins[someuniqueid]
check()
At the last step, I am removing the object reference from the memory
is using above procedure will delete the object from memory
If not then what is wrong with code and how to correct it
You would need to do del d as well, since d is also holding a reference to the same object. Calling del will only decrement the reference count and remove the particular reference from usage, but the actual in memory object is not garbage collected until the reference count hits 0.
I don't know what do you mean by writing:
If not then what is wrong with code and how to correct it
When you use del statement you delete a reference to an object. It will use up memory untill garbage collector is invoked. Remember that this can be a time-consuming process and not necessary if the process has enough memory to continue executing.
Generally speaking Python does not perform C++-like destructor bahaviour.
A quote from "Expert Python Programming":
The approach of such a memory manager is roughly based on a simple
statement: If a given object is not referenced anymore, it is removed.
In other words, all local references in a function are removed after
the interpreter:
• Leaves the function
• Makes sure the object is not being used anymore.
Under normal conditions, the collector will do a
nice job. But a del call can be used to help the garbage collector by
manually removing the references to an object manually.
So you don't manage memory by hand. You can help garbage collector, but it's better to leave memory managment behind the scenes.
Related
In my program I create an un-ending amount of class instances. the amount depends on how long the program is running. However I don't need the instances at all after a certain code being run. How could i remove them completely from memory?
Simple example code:
class Player:
def __init__(self, color):
self.color = color
for n in range(1000):
p = Player('black')
Would del p in this case completely remove that instance?
Python will remove them from memory for you when they are no longer referred to. If you have Player instances that refer to other Player instances (ex: p.teammates = [list of Players]) you could end up with circular references that may prevent them from being garbage collected. In this case you should consider the weakref module.
for example:
>>>sam = Player('blue')
>>>rob = Player('green')
>>>sam.team = [sam, rob]
>>>rob.team = [sam, rob]
>>> #sam and rob may not be deleted because they contain
>>> #references to eachother so the reference count cannot reach 0
>>>del sam #del is a way to manually dereference an object in an interactive prompt. Otherwise the interpreter cannot know you won't use it again unlike when the entire code is known at the beginning.
>>>print(rob.team[0].color) #this prints 'blue' proving that sam hasn't been deleted yet
blue
so how do we fix it?
>>>sam = Player('blue')
>>>rob = Player('green')
>>>sam.team = [weakref.ref(sam), weakref.ref(rob)]
>>>rob.team = [weakref.ref(sam), weakref.ref(rob)]
>>> #now sam and rob can be deleted, but we've changed the contents of `p.team` a bit:
>>> #if they both still exist:
>>>rob.team[0]() is sam #calling a `ref` object returns the object it refers to if it still exists
True
>>>del sam
>>>rob.team[0]() #calling a `ref` object that has been deleted returns `None`
None
>>>rob.team[0]().color #sam no longer exists so we can't get his color
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'color'
In this instance, del p would only remove the reference to the Player object, so that it can later be picked up by the garbage collector.
However, this also happens when it goes out-of-scope.
In most everyday Python, there is no need to use explicit del statements.
There is no way in Python to delete an instance. Instead you can delete references to the instance, and once they are all gone, the object is reclaimed.
I want to remove all objects, which refer to one memory location. How to do it if I do not know their all name?
class Foo(object):
pass
class Bar(object):
pass
a = Foo()
b = a
c = Bar()
c.a_ref = a
c.b_ref = b
for item in a, b, c.a_ref, c.b_ref:
print(id(item))
""" Out:
140035270075472
140035270075472
140035270075472
140035270075472
"""
UPD:
Ok. I want to remove link or port1.link and don't worry about all the other ports(port2.link) linked to it, they also have to disappear.
class LINK(object):
pass
class PORT(object):
def __init__(self, link=None):
self.link = link
def __repr__(self):
return str(id(self.link))
link = LINK()
port1 = PORT()
port2 = PORT()
port1.link = port2.link = link
print(id(link), port1, port2)
# (140586778512720, 140586778512720, 140586778512720)
del link
print(port1, port2)
# (140586778512720, 140586778512720)
# want: (None, None) ???
You can't explicitly free memory in Python. The garbage collector is responsible to do that for you.
What you can do is guarantee that all the references to your objects are cleaned. The garbage collector works with epochs (like e1, e2 and e3) and only the objects in e3 will be cleaned in the next iteration.
Objects only go from epoch eX to eY (where Y > X) if there is no reference to them. So your objects start in e1; if you "clean" all references, in the next iteration of garbage collector they will be moved to e2; in the next one they will be moved to e3 and finally the memory will be released.
You can change the periodicity that garbage collector is called, but I would not recommend that. Or you can force the calling of garbage collector using gc.collect() but as I said, your objects have to go through all epochs before they are cleaned (that's why the calling of gc.collector() didn't work for you). Just to clarify, the garbage collector is called once a threshold (number of objects in memory tracked by the gc) is achieved.
If you want to do a diagnoses of memory leaks in Python you can also use the objgraph library which is really good. It builds graphs with all links and references between objects and let you identify cycles. Cycles are the main reason for the objects not to be released by the garbage collector when you think that all references are cleaned. Here is a good tutorial http://chase-seibert.github.io/blog/2013/08/03/diagnosing-memory-leaks-python.html
See the weakref module. It maintains a reference to an object without preventing it from being garbage collected if all its strong references are removed. Example:
import weakref
class LINK(object):
pass
class PORT(object):
def __init__(self, link=None):
if link is not None:
self.link = weakref.ref(link)
else:
self.link = None
def __repr__(self):
if self.link is None:
return 'None'
o = self.link()
return 'None' if o is None else str(id(o))
link = LINK()
port1 = PORT()
port2 = PORT()
port1.link = port2.link = weakref.ref(link)
print(id(link), port1, port2)
del link
print(port1, port2)
Output:
70741240 70741240 70741240
None None
Note, you may still have to call gc.collect() in some situations before the weak references report None.
You can't explicitly free memory. What you need to do is to make sure you don't keep references to objects. They will then be garbage collected, freeing the memory.
BTW, you can call gc.collect() to force a gc operation.
UPDATE:
You CAN NOT delete all the references to one object through only deleting one of its references, since you don't know who else is still using this object. And if so, there will be another question, how to prevent others to delete my object without acknowledging me.
I think best solution to your question is to delete attribute link separately.
del link
del port1.link
del port2.link
Then to get None after deleting, you should do this:
...
def __repr__(self):
repr_string = getattr(self, 'link', None)
return str(id(self.link)) if repr_string else None
...
Your real problem, to me, sounds like you have a graph where PORTs are nodes and LINKs are edges. And your requirement is that when you delete an edge, the two nodes that the edge connects should also be deleted. This can be done explicitly by overriding the __del__ method of the node so that deleting it deletes the edges which is connects to.
Python abstracts away the "memory" and using the fact that id gives you the memory location (which is really an implementation detail of CPython) is a flaky way of doing this.
Is there some way in python to manually free memory allocated for a string while some other variable still references it? I would like to free a string passed to a function inside of the function.
def use_string(s):
do_something_with_s(s)
#can I free memory used for s here?
do_other_things_that_dont_need_s()
my_string = "hi there."*100000
use_string(my_string)
do_some_other_things_that_dont_need_my_string()
As I understand, both s (in the context of use_string) and my_string will be referencing the same immutable string object, which can only be freed when no variables point to it. Is this ture?
Yes, that's what the del keyword is for.
Whether it actually makes sense to do this in a garbage collected language is another matter.
Using del s in the function will remove the reference in that scope.
If it's the last reference to the string object, it may then be freed. (In the example here, it won't, since the global my_string still refers to it.)
Yes. Python's garbage collector will free the memory when there are no more references to it.
If you want more details: http://www.digi.com/wiki/developer/index.php/Python_Garbage_Collection
Given your example, you can either
my_string = "hi there."*100000
use_string(my_string)
del my_string
which allows the garbage collector to release the memory initially allocated for my_string, or
use_string( "hi there." * 100000 )
in which case the only reference to the string was the function-local variable, and the memory will be freed once the function returns.
Under no circumstances should you (even if you were able to) deallocate the memory inside the function, since you have no idea if other code held a reference to that memory and might try to use it.
Since Python objects are passed as a reference passed by value (ie you can't change the reference, but you can change the contents of the referenced object unless the object is immutable), you cannot change the reference a caller directly passes into the function from inside the function.
If you need to change/free the string, you need to wrap it in a class so you can change the only reference to the string which is inside the object. I'd recommend taking a deep breath before obfuscating the code like this, but when you really need it, you really need it.
This example is designed only to in minimal code show the concept, not to show how to do it in a beautiful way :)
def use_string(wrap):
print wrap.s # Prints 'olle'
wrap.s = None # Resets the reference
print wrap.s # Prints 'None'
class wrapper(object):
pass
a = wrapper()
a.s = "olle"
print a.s # Prints 'olle'
use_string(a)
print a.s # Prints 'None'
I am working with classes in Python for the first time and I need to loop through my class attributes and delete certain instances under certain conditions. The problem is that I cannot find any examples of deleting instances of certain attributes. To be a little more specific, my class is phone_bills and one of my attributes is minutes_used and the instance would be minutes used of a specific month. Well, sometimes I need to delete that one month or one instance.
I am starting to wonder if working with classes is wrong for this particular project.
Here is some of my code (where i make the class and then at the bottom where i try to deltete an instance.
class MeterBill:
'components of MeterBill'
def __init__(self,IDC,Name,StartD,End_D,Mdays,Cons): #Name,StartD,End_D,Mdays,Cons):
self.IDC = IDC #idc
self.Name= Name #name
self.StartD = StartD #startd
self.End_D = End_D #end_d
self.Mdays = Mdays #modays
self.Cons = Cons #cons
def __repr__(self):
return repr((self.IDC,self.Name,self.StartD,self.End_D,self.Mdays,self.Cons))
#there is some other code here to read in the data then
e=len(bills); startlen=e;s=0
row=0; daysthresh=38; count=0
while count < startlen:
for row in range(s,e):
count = 1+ count
if bills[row-1].Mdays < daysthresh and bills[row-1].IDC==bills[row].IDC:
print bills[row-1],#row-1,meter[row-1]
bills[row].Mdays = bills[row-1].Mdays+bills[row].Mdays
bills[row].Cons = bills[row-1].Cons+bills[row].Cons
bills[row].StartD=bills[row-1].StartD
#del mybills.minutes_used
#bills=MeterBill()
del bills[row-1].Cons
the last 3 lines is me trying to delte an instance of my class at row-1 (using code from Peter Downs post). I want to delete this one line. I want to delete 1 single instance of each attribute that i defined.
so if I could get that del bill[row-1].cons to work then i would do it for all the other attributes at row-1.
Note you have to scroll to the right ot see my if statement.
I am starting to wonder if working with classes is wrong for this particular project.
No, certainly not, no worries :)
Lets say we have the following class:
class PhoneBills(object):
def __init__(self, minutes_used):
self.minutes_used = minutes_used
Then you can delete the attribute minutes_used simply by:
mybills = PhoneBills()
del mybills.minutes_used
Which would remove the attribute from your object mybills. All lookups would result in an exception. I'm not sure that this is what you want. Probably just setting minutes_used to 0 or None would be a better approach?
Using an object in this case isn't a bad idea, as others have pointed out. However, you have to think about the problem just a little bit differently in order to get the best use of these objects.
If you have a phone_bills object, then we assume its only responsibility is to manage a phone bill. It could be for a single month, it could be for an entire year - there's enough context in the object name that it could do both.
If it's managing a month-to-month bill, then what's required is, at the end of every month, the minutes used is recalculated, meaning that the value for the used minutes at this current point is reset, not deleted.
Now, unless your assignment specifically calls for you to delete the end-of-month total, then you're best served with resetting the value. The way to do this with Python objects is simple:
phone_bills.minutes_used = 0
Deleting means dereferencing the object, till its referencecounter reaches 0, so the garabage collector may free the allocated space for that particular object and effectivly destroying/deleting it.
What you want to do is set the appropriate attribute to None. By doing so, you reduce the refcounter by 1.
To illustrate what I mean, try the following:
import sys
ref = 'a Random object'
print sys.getrefcount(ref), id(ref)
newlist = [ref]
print sys.getrefcount(newlist[0]), id(newlist[0])
ref = None
print sys.getrefcount(newlist[0]), id(newlist[0])
newlist[0] = None
after the last line you have no reference to the underlying object, the refounter reaches 0 and the object gets collected by the garbage collector.
You may also use the del statement to express your intend clearly in your sourecode. e.g.: del june but then you also remove the identifier june from your namespace. Don't shoot the messanger! ;-)
I am starting to wonder if working with classes is wrong for this
particular project.
I believe that they may be unnecessary.
Instead of a class, try using dictionaries.
You could write a helper function to populate them, and it's very easy to remove a key from a dictionary.
The only reason to use a class is if you need instance methods - is that true?
Event then, you could rewrite them as regular functions.
def createPhoneBill(minutes_used):
return {
"minutes_used":minutes_used,
# you could put more attributes here, just add them as arguments to the function
}
As an added bonus, default values are much easier. Also, you get to use the dictionary.get(attr_name, default_value) function now, too.
Here's what deletion would look like:
Deleting an attribute:
mybills = createPhoneBill(5000):
if mybills["minutes_used"] > 2000:
del mybills["minutes_used"]
Deleting an 'instance':
mybills = createPhoneBill(5000):
if mybills["minutes_used"] > 3000:
del mybills
In Python, you don't delete objects--you simply remove any references towards them and allow the garbage collector to reclaim the memory they're holding.
Assigning phone_bills_obj.minutes_used = None would cause the garbage collector to remove the object referenced by phone_bills_ojb.minutes_used, in case the same object isn't being referenced anywhere else.
There are several threads on Python garbage collection in SO, and after reading about five, plus some doc on line, i am still not sure as to how garbage collection works and how i should manage objects which i am not using. In fact somewhere i read one should not do anything about collecting garbage, others tell one should del objects, while others again explain de-referencing an object is enough for Python to collect it as garbage.
So, at the risk of creating a duplicate, i will ask the question again, but differently, hoping to get more comprehensive and clearer information.
In my case i want to make a small simulation with objects representing people. Several instances of the Person() class will be created. It should exist for some time until it virtually "dies" while other instances will be created.
Now how do i make this Person() instance "die" (assuming many many of these instances will be created and i don't want these instances to hang out like ghosts)?
There are several ways i can reference an object:
john = Person('john')
or
people = []
people.append(Person('john'))
or
people = {}
people['john'] = Person('john')
What is the best way to keep my program clean, freeing resources optimally? And what is the best way then to reference my object so i can control the deletion of the object?
Maybe this also can help:
>>> # Create a simple object with a verbose __del__ to track gc.
>>> class C:
... def __del__(self):
... print "delete object"
...
>>> c = C()
>>> # Delete the object c successfully.
>>> del c
delete object
>>> # Deletion of an object when it go out of the scope where it was defined.
>>> def f():
... c = C()
...
>>> f()
delete object
>>> c = C()
>>> # Create another reference of the object.
>>> b = c
>>> # The object wasn't destructed the call of del only decremented the object reference.
>>> del c
>>> # Now the reference counter of the object reach 0 so the __del__ was called.
>>> del b
delete object
>>> # Create now a list that hold all the objects.
>>> l = [C(), C()]
>>> del l
delete object
delete object
>>> # Create an object that have a cyclic reference.
>>> class C:
... def __init__(self):
... self.x = self
... def __del__(self):
... print "delete object"
...
>>> c = C()
>>> # Run the garbage collector to collect object.
>>> gc.collect()
9
>>> # the gc.garbage contain object that the gc found unreachable and could not be freed.
>>> gc.garbage
[<__main__.C instance at 0x7ff588d84368>]
>>> # Break the cyclic reference.
>>> c.x = None
>>> # And now we can collect this object.
>>> del c
delete object
>>> # Create another object with cyclic reference.
>>> c = C()
>>> # When closing the interactive python interpreter the object will be collected.
delete object
Refrences : del method ; gc module ; weakref module
None of this really has anything to do with garbage collection.
Python's main method of memory management uses reference counting.
In all cases above, Python keeps a count of all the references to the object, and when there are none left, the object is deleted (similar to std::shared_pointer in C++).
References get decreased when
the object holding them is either explicitly deleted (via del)
or goes out of scope (see also here (esp. ex. 8)).
In your case, this applies to either the john object, or either of the people containers. They go out of scope at the end of the function that created them (assuming they are not returned to the calling function). The vast majority of the time, you can just let them go out of scope - it's only when you create really heavy objects or collections - say inside a big loop - that you might want to consider explicitly using del.
Garbage collection really only comes into play when there are reference cycles
- for instance, when an object refers to itself. Like:
a = []
a.append(a)
Again, this happens automatically, and you shouldn't need to do anything special.
I find that most programs create and dispose of objects quite naturally, so I never normally worry about it.
Some examples:
person = Person('john')
person = Person('james')
# Whoops! 'john' has died!
people = []
people.append(Person('john'))
# ...
# All 'Persons' live in people
people = []
# Now all 'Persons' are dead (including the list that referenced them)
class House():
def setOwner(self, person):
self.owner = person
house.setOwner(people[0])
# Now a House refers to a Person
people = []
# Now all 'Persons' are dead, except the one that house.owner refers to.
What I assume you are after is this:
people = {}
people['john'] = Person('john')
def removePerson(personName):
del people[personName]
removePerson('john')
In this case people is the master list and you can control when a Person gets added and removed from the list (its a dictionary).
You may have to think through the concept of a person being created and then dying very thoroughly: Once created how does the person first interact with the simulation. Upon death, how should you untangle the references? (Its ok for a person to refer to other stuff, its things like House in my example that would keep a person alive. You could have other objects hold on to just the name of the person).
Previous answers are correct but here is what is recommended according to python 3.7 Document:
"Python does automatic memory management (reference counting for most objects and garbage collection to eliminate cycles). The memory is freed shortly after the last reference to it has been eliminated."
If you really have to do this because of memory management issues you are experiencing with a particular case then import the gc library and just do this
del self.someInstanceOfyourClass
gc.collect()
here is a simple example https://github.com/nanoseconds/PythonTest/blob/master/test.py