We Keep Coding

Deleting objects in memory using Python

I am trying this approach to delete an object in python. I read the documentation of Python stating that garbage collector will automatically delete the object that is not referenced.
def check():
class newOb():
def __init__(self,value):
self.value = value
print self.value
return None
class ob:
ins = {}
def cr(self,someuniqueid,value) :
newV = newOb(value)
ob.ins[someuniqueid] = newV ## saving this object refernce to the ob class ins dictionary
return newV
#### Accessing Object ###
someuniqueid = 12
c = ob()
d = c.cr(someuniqueid,123)
print d.value ## will print 123
# now deleting the associated object
del c.ins[someuniqueid]
check()
At the last step, I am removing the object reference from the memory
is using above procedure will delete the object from memory
If not then what is wrong with code and how to correct it

You would need to do del d as well, since d is also holding a reference to the same object. Calling del will only decrement the reference count and remove the particular reference from usage, but the actual in memory object is not garbage collected until the reference count hits 0.

I don't know what do you mean by writing:
If not then what is wrong with code and how to correct it
When you use del statement you delete a reference to an object. It will use up memory untill garbage collector is invoked. Remember that this can be a time-consuming process and not necessary if the process has enough memory to continue executing.
Generally speaking Python does not perform C++-like destructor bahaviour.
A quote from "Expert Python Programming":
The approach of such a memory manager is roughly based on a simple
statement: If a given object is not referenced anymore, it is removed.
In other words, all local references in a function are removed after
the interpreter:
• Leaves the function
• Makes sure the object is not being used anymore.
Under normal conditions, the collector will do a
nice job. But a del call can be used to help the garbage collector by
manually removing the references to an object manually.
So you don't manage memory by hand. You can help garbage collector, but it's better to leave memory managment behind the scenes.

How do I access class variables without changing them in python?

I'm new to programming so sorry for the basic question. I am trying to write a search algorithm for a class, and I thought creating a class for each search node would be helpful.
class Node(object):
def __init__(self, path_to_node, search_depth, current_state):
self.path_to_node = path_to_node
self.search_depth = search_depth
self.current_state = current_state
...
With some functions too. I am now trying to define a function outside of the class to create children nodes of a node and add them to a queue. node.current_state is a list
def bfs_expand(node, queuey, test_states):
# Node Queue List -> Queue List
# If legal move and not already in test states create and put children nodes
# into the queue and their state into test_states. Return queue and test states
# Copy original path, depth, and state to separate variables
original_path = node.path_to_node
original_depth = node.search_depth
original_state = node.current_state
# Check if up is legal, if so add new node to queue and state to test state
if node.is_legal_move('Up'):
up_state = original_state
a = up_state.index(0)
b = a - 3
up_state[a], up_state[b] = up_state[b], up_state[a]
if up_state not in test_states:
test_states.append(up_state)
up_node = Node(original_path + ['Up'], original_depth + 1, up_state)
queuey.put(up_node)
print(test_states)
print(original_state)
I then try to proceed through down, left and right with similar if statements, but they are messed up because the original_state has changed. When I print the original state after that up statement, it returns the up_state created in the if statement. I realize (well, I think) that this is because original_state, and therefore up_state, are actually calling node.current_state and do not store the list in a separate variable. How should I get the variable from a node to manipulate independently? Should I not even be using a class for something like this, maybe a dictionary? I don't need code written for me but a conceptual nudge would be greatly appreciated!

You should use copy.deepcopy if you want to avoid modifying the original
original_path = copy.deepcopy(node.path_to_node)
original_depth = copy.deepcopy(node.search_depth)
original_state = copy.deepcopy(node.current_state)
Or essentially whichever object you want to use as a "working copy" should be a deep copy of the original if you don't want to modify the original version of it.

Expanding a bit on #CoryKramer's answer: In Python, objects have reference semantics, which means that saying
a = b
where a and b are objects, makes both a and b references to the same object, meaning that changing a property on a will change that same property on b as well. In order to actually get a new object with the same properties as the old one, you should use copy.deepcopy as already stated. However, be careful when using that function. If your object contains a reference cycle (i.e.: It contains a reference to an object which contains a reference to itself), copy.deepcopy will lead to an infinite loop.
For this reason, there is also copy.copy, which does not follow object references contained in the object to copy.

How to remove memory location that is referenced objects in python?

I want to remove all objects, which refer to one memory location. How to do it if I do not know their all name?
class Foo(object):
pass
class Bar(object):
pass
a = Foo()
b = a
c = Bar()
c.a_ref = a
c.b_ref = b
for item in a, b, c.a_ref, c.b_ref:
print(id(item))
""" Out:
140035270075472
140035270075472
140035270075472
140035270075472
"""
UPD:
Ok. I want to remove link or port1.link and don't worry about all the other ports(port2.link) linked to it, they also have to disappear.
class LINK(object):
pass
class PORT(object):
def __init__(self, link=None):
self.link = link
def __repr__(self):
return str(id(self.link))
link = LINK()
port1 = PORT()
port2 = PORT()
port1.link = port2.link = link
print(id(link), port1, port2)
# (140586778512720, 140586778512720, 140586778512720)
del link
print(port1, port2)
# (140586778512720, 140586778512720)
# want: (None, None) ???

You can't explicitly free memory in Python. The garbage collector is responsible to do that for you.
What you can do is guarantee that all the references to your objects are cleaned. The garbage collector works with epochs (like e1, e2 and e3) and only the objects in e3 will be cleaned in the next iteration.
Objects only go from epoch eX to eY (where Y > X) if there is no reference to them. So your objects start in e1; if you "clean" all references, in the next iteration of garbage collector they will be moved to e2; in the next one they will be moved to e3 and finally the memory will be released.
You can change the periodicity that garbage collector is called, but I would not recommend that. Or you can force the calling of garbage collector using gc.collect() but as I said, your objects have to go through all epochs before they are cleaned (that's why the calling of gc.collector() didn't work for you). Just to clarify, the garbage collector is called once a threshold (number of objects in memory tracked by the gc) is achieved.
If you want to do a diagnoses of memory leaks in Python you can also use the objgraph library which is really good. It builds graphs with all links and references between objects and let you identify cycles. Cycles are the main reason for the objects not to be released by the garbage collector when you think that all references are cleaned. Here is a good tutorial http://chase-seibert.github.io/blog/2013/08/03/diagnosing-memory-leaks-python.html

See the weakref module. It maintains a reference to an object without preventing it from being garbage collected if all its strong references are removed. Example:
import weakref
class LINK(object):
pass
class PORT(object):
def __init__(self, link=None):
if link is not None:
self.link = weakref.ref(link)
else:
self.link = None
def __repr__(self):
if self.link is None:
return 'None'
o = self.link()
return 'None' if o is None else str(id(o))
link = LINK()
port1 = PORT()
port2 = PORT()
port1.link = port2.link = weakref.ref(link)
print(id(link), port1, port2)
del link
print(port1, port2)
Output:
70741240 70741240 70741240
None None
Note, you may still have to call gc.collect() in some situations before the weak references report None.

You can't explicitly free memory. What you need to do is to make sure you don't keep references to objects. They will then be garbage collected, freeing the memory.
BTW, you can call gc.collect() to force a gc operation.
UPDATE:
You CAN NOT delete all the references to one object through only deleting one of its references, since you don't know who else is still using this object. And if so, there will be another question, how to prevent others to delete my object without acknowledging me.
I think best solution to your question is to delete attribute link separately.
del link
del port1.link
del port2.link
Then to get None after deleting, you should do this:
...
def __repr__(self):
repr_string = getattr(self, 'link', None)
return str(id(self.link)) if repr_string else None
...

Your real problem, to me, sounds like you have a graph where PORTs are nodes and LINKs are edges. And your requirement is that when you delete an edge, the two nodes that the edge connects should also be deleted. This can be done explicitly by overriding the __del__ method of the node so that deleting it deletes the edges which is connects to.
Python abstracts away the "memory" and using the fact that id gives you the memory location (which is really an implementation detail of CPython) is a flaky way of doing this.

Recursion and Helper Function

Sorry if this is a general question but I am a beginner in Python and many times when I see other people code using recursion, they create a helper function for the main function and then call that helper function which itself is recursive.
This seems a bit different from the simplest cases of recursion for example (sum of lists, factorial) where the function only calls itself.
Can someone explain this technique more carefully perhaps with examples?
Much appreciated.
Example 1: (Reversing linked list using recursion)
def revert_list(self):
self.head = self._revert_helper(self.head)
def _revert_helper(self, node):
temp = None
if node.forward == None:
return node
else:
temp = self._revert_helper(node.forward)
node.forward.forward = node
node.forward = None
return temp
Example 2: (Binary Search Tree)
def __contains__(self, key):
return self._bstSearch(self._root, key)
# Returns the value associated with the key.
def valueOf(self, key):
node = self._bstSearch(self._root, key)
assert node is not None, "Invalid may key."
return node.value
# Helper method that recursively searches the tree for a target key:
# returns a reference to the Node. This allows
# us to use the same helper method to implement
# both the contains and valueOf() methods of the Map class.
def _bstSearch(self, subtree, target):
if subtree is None: # base case
return None
elif target < subtree.key: # target is left of the subtree root
return self._bstSearch(subtree.left)
elif target > subtree.key: # target is right of the subtree root
return self.bstSearch(subtree.right)
else: # base case
return subtree

This is actually used more often in other languages, because python can usually emulate that behavior with optional arguments. The idea is that the recursion gets a number of initial arguments, that the user doesn't need to provide, which help keep track of the problem.
def sum(lst):
return sumhelper(lst, 0)
def sumhelper(lst, acc):
if lst:
acc += lst[0]
return sumhelper(lst[1:], acc)
return acc
Here it's used to set a starting parameter to 0, so the user doesn't have to provide it. However, in python you can emulate it by making acc optional:
def sum(lst, acc=0):
if lst:
acc += lst[0]
return sum(lst[1:], acc)
return acc

Usually when I do this, it is because the recursive function is tricky or annoying to call, so I have a wrapper that is more convenient. For example, imagine a maze solver function. The recursive function needs a data structure to keep track of visited spots inside the maze, but for convenience to the caller I just want the caller to need to pass in a maze to solve. You can maybe handle this with a default variable in Python.
The other major reason I have done this is for speed. The recursive function is very trusting, and assumes its arguments are all valid; it just goes full speed ahead with the recursion. Then the wrapper function carefully checks all the arguments before making the first call to the recursive function. As a trivial example, factorial:
def _fact(n):
if n == 0: # still need to handle the basis case
return 1
return n*_fact(n-1)
def fact(n):
n0 = int(n)
if n0 != n:
raise ValueError("argument must make sense as an int")
if n < 0:
raise ValueError("negative numbers not allowed")
return _fact(n)
I have edited this from the original, and now it's actually a pretty reasonable example. We coerce the argument to an integer ("duck typing") but we require that the != operator not indicate it to have changed in value by this coercion; if converting it to int changes the value (for example, a float value that had a fractional part truncated) we reject the argument. Likewise, we check for negative and reject the argument. Then the actual recursive function is very trusting and contains no checks at all.
I could give less vague answers if you posted an example you have seen of code that inspired this question.
EDIT: Okay, discussion of your examples.
Example 1: (Reversing linked list using recursion)
Pretty simple: the "helper" function is a general recursive function that will work on any node in the class that has a linked list. Then the wrapper is a method function that knows how to find self.head, the head of the list. This "helper" is a class member function, but it could also be a simple function in a general data-structures stuff library. (This makes more sense in Python than in languages like C, because a function like this could work with any linked list that is a class with a member called forward as its "next pointer" value. So you really could write this once and then use it with multiple classes that implement linked lists.)
Example 2: (Binary Search Tree)
The actual recursive function returns None if no node can be found with the specified key. Then there are two wrappers: one that implements __contains__(), which works just fine if it returns None; and valueOf(), which raises an exception if the key is not found. As the comment notes, two wrappers lets us solve two different problems with a single recursive function.
Also, just as with the first example, the two wrappers kick off the search in a specific location: self._root, the root of the tree. The actual recursive function can be started anywhere inside a tree.
If __contains__() were implemented with a default argument of a node to search, and the default was set to some unique value, it could check for the special value and start at the root in that case. Then when __contains__() is called normally, the unique value would be passed in, and the recursive function could know that it needs to look at the special location self._root. (You can't just pass in self._root as the default value, because the default value is set at compile time, and the class instance can change after that, so it wouldn't work right.)
class UniqueValue:
pass
def __contains__(self, key, subtree=UniqueValue):
if subtree is UniqueValue:
subtree = self._root
if subtree is None: # base case
return None
elif key < subtree.key: # target is left of the subtree root
return self.__contains__(key, subtree.left)
elif key > subtree.key: # target is right of the subtree root
return self.__contains__(key, subtree.right)
else: # base case
return subtree
Note that while I said it could be implemented as I show here, I didn't say I prefer it. Actually I prefer the two wrappers version. This is a little bit tricky, and it wastes time on every recursive call checking to see if subtree is UniqueValue. More complex and wastes time... not a win! Just write the two wrappers, which start it off in the right place. Simple.

From my experience (and my experience only), I use this style of coding when
The recursion is only useful in the larger function (not very recommended, but I have some bad habits)
There needs to be preparation done for the function, but only once (instead of a flag or other switch)
One way I use it is for logging purposes, while avoiding re-logging levels
def _factorial(x):
return 1 if x == 0 else x*_factorial(x)
#log #assuming some logging decorator "log"
def factorial(x):
return _factorial(x)
Otherwise, log would be called for each recursive level of the factorial function, something I may not desire.
Another usage would be to resolve default arguments.
def some_function(x = None):
x = x or set() #or whatever else
#some stuff
return some_function()
Would check if x is falsey for every iteration, while what I actually need is a decorator, or as an alternative:
def some_function(x = None):
return _some_function(x if x else set())
where _some_function is the helper function.
Specifically with 2, it allows for some freedom of abstraction. If for some reason you didn't want to use a bstsearch, you could just swap it for some other function in __contains__ (and you'd also be able to reuse code in different places)

Deleting certain instances of a class attribute

I am working with classes in Python for the first time and I need to loop through my class attributes and delete certain instances under certain conditions. The problem is that I cannot find any examples of deleting instances of certain attributes. To be a little more specific, my class is phone_bills and one of my attributes is minutes_used and the instance would be minutes used of a specific month. Well, sometimes I need to delete that one month or one instance.
I am starting to wonder if working with classes is wrong for this particular project.
Here is some of my code (where i make the class and then at the bottom where i try to deltete an instance.
class MeterBill:
'components of MeterBill'
def __init__(self,IDC,Name,StartD,End_D,Mdays,Cons): #Name,StartD,End_D,Mdays,Cons):
self.IDC = IDC #idc
self.Name= Name #name
self.StartD = StartD #startd
self.End_D = End_D #end_d
self.Mdays = Mdays #modays
self.Cons = Cons #cons
def __repr__(self):
return repr((self.IDC,self.Name,self.StartD,self.End_D,self.Mdays,self.Cons))
#there is some other code here to read in the data then
e=len(bills); startlen=e;s=0
row=0; daysthresh=38; count=0
while count < startlen:
for row in range(s,e):
count = 1+ count
if bills[row-1].Mdays < daysthresh and bills[row-1].IDC==bills[row].IDC:
print bills[row-1],#row-1,meter[row-1]
bills[row].Mdays = bills[row-1].Mdays+bills[row].Mdays
bills[row].Cons = bills[row-1].Cons+bills[row].Cons
bills[row].StartD=bills[row-1].StartD
#del mybills.minutes_used
#bills=MeterBill()
del bills[row-1].Cons
the last 3 lines is me trying to delte an instance of my class at row-1 (using code from Peter Downs post). I want to delete this one line. I want to delete 1 single instance of each attribute that i defined.
so if I could get that del bill[row-1].cons to work then i would do it for all the other attributes at row-1.
Note you have to scroll to the right ot see my if statement.

I am starting to wonder if working with classes is wrong for this particular project.
No, certainly not, no worries :)
Lets say we have the following class:
class PhoneBills(object):
def __init__(self, minutes_used):
self.minutes_used = minutes_used
Then you can delete the attribute minutes_used simply by:
mybills = PhoneBills()
del mybills.minutes_used
Which would remove the attribute from your object mybills. All lookups would result in an exception. I'm not sure that this is what you want. Probably just setting minutes_used to 0 or None would be a better approach?

Using an object in this case isn't a bad idea, as others have pointed out. However, you have to think about the problem just a little bit differently in order to get the best use of these objects.
If you have a phone_bills object, then we assume its only responsibility is to manage a phone bill. It could be for a single month, it could be for an entire year - there's enough context in the object name that it could do both.
If it's managing a month-to-month bill, then what's required is, at the end of every month, the minutes used is recalculated, meaning that the value for the used minutes at this current point is reset, not deleted.
Now, unless your assignment specifically calls for you to delete the end-of-month total, then you're best served with resetting the value. The way to do this with Python objects is simple:
phone_bills.minutes_used = 0

Deleting means dereferencing the object, till its referencecounter reaches 0, so the garabage collector may free the allocated space for that particular object and effectivly destroying/deleting it.
What you want to do is set the appropriate attribute to None. By doing so, you reduce the refcounter by 1.
To illustrate what I mean, try the following:
import sys
ref = 'a Random object'
print sys.getrefcount(ref), id(ref)
newlist = [ref]
print sys.getrefcount(newlist[0]), id(newlist[0])
ref = None
print sys.getrefcount(newlist[0]), id(newlist[0])
newlist[0] = None
after the last line you have no reference to the underlying object, the refounter reaches 0 and the object gets collected by the garbage collector.
You may also use the del statement to express your intend clearly in your sourecode. e.g.: del june but then you also remove the identifier june from your namespace. Don't shoot the messanger! ;-)

I am starting to wonder if working with classes is wrong for this
particular project.
I believe that they may be unnecessary.
Instead of a class, try using dictionaries.
You could write a helper function to populate them, and it's very easy to remove a key from a dictionary.
The only reason to use a class is if you need instance methods - is that true?
Event then, you could rewrite them as regular functions.
def createPhoneBill(minutes_used):
return {
"minutes_used":minutes_used,
# you could put more attributes here, just add them as arguments to the function
}
As an added bonus, default values are much easier. Also, you get to use the dictionary.get(attr_name, default_value) function now, too.
Here's what deletion would look like:
Deleting an attribute:
mybills = createPhoneBill(5000):
if mybills["minutes_used"] > 2000:
del mybills["minutes_used"]
Deleting an 'instance':
mybills = createPhoneBill(5000):
if mybills["minutes_used"] > 3000:
del mybills

In Python, you don't delete objects--you simply remove any references towards them and allow the garbage collector to reclaim the memory they're holding.
Assigning phone_bills_obj.minutes_used = None would cause the garbage collector to remove the object referenced by phone_bills_ojb.minutes_used, in case the same object isn't being referenced anywhere else.