We Keep Coding

How to calculate hash of a python class object

In python3, I have a class. Like below:
class Foo:
def __init__(self):
self.x = 3
def fcn(self, val):
self.x += val
Then I instantiate objects of that class, like so:
new_obj = Foo()
new_obj2 = Foo()
Now when I hash these objects, I get different hash values. I need them to return the same hash, as they are the same objects (in theory).
Any idea how I can do this?

Thank you to all who answered. You're right that instantiating a new instance of the same class object is not actually the same, as it occupies a different place in memory. What I ended up doing is similar to what #nosklo suggested.
I created a 'get_hashables' function that returned a dictionary with all the properties of the class that would constitute a unique class object, like so:
def get_hashables(self):
return {'data': self.data, 'result': self.result}
Then my main method would take these 'hashable' variables, and hash them to produce the hash itself.

class Foo:
def __init__(self):
self.x = 3
def fcn(self, val):
self.x += val
def __hash__(self):
return hash(self.x)
This will calculate the hash using self.x; That means the hash will be the same when self.x is the same. You can return anything from __hash__, but to prevent consistency bugs you should return the same hash if the objects compare equal. More about that in the docs.

They are not the same object. The expression Foo() invokes the class constructor, Foo.__init__, which returns a new, unique instance of the object on each call. Your two calls return two independent objects, residing in different memory locations, each containing its own, private instance of the x attribute.
You might want to read up on Python class and instance theory.

Must all attributes of an object be in the objects __hash__ function?

Using python 3.6
I have read the docs.
When implementing __hash__ Must all attributes of an object be in the objects __hash__ function?
So would this example be ok?
class Foo(object):
def __init__(self, value):
self.value = value
self.not_important = 'irrelevant'
def __hash__(self):
return hash((self.value,))
def __eq__(self, other):
if self.value == other.value:
return True
return False
foo.not_important is modified while being a key in a dictionary
>>> foo = Foo(1)
>>> d = {foo:'foo is in d'}
>>> foo.not_important = 'something else'
>>> d[foo]
'foo is in d'
>>> bar = Foo(1)
>>> foo == bar
True
>>> d[bar]
'foo is in d'
But foo.not_important isn't used by it's __hash__ implementation. Is this perfectly okay? Or can this go horribly wrong?

Answering the literal question, it's okay to leave out attributes not considered by __eq__. In fact, you can leave out all the attributes and just return 0, though it'll kill your dict efficiency, so don't do that.
Answering the implied question, it's okay to mutate an object while it's a dict key, as long as you don't mutate it in ways that affect __eq__ or __hash__. For example, the default __hash__ implementation doesn't consider any attributes of the object it's hashing at all - it's based on object identity. With the default __hash__ and __eq__, an object is only equal to itself, and you can mutate such an object all you want while it's a dict key without breaking dicts.

No. As the documentation states:
object.__hash__(self)
Called by built-in function hash() and for operations on members of hashed collections including set, frozenset, and dict. __hash__() should return an integer. The only required property is that objects which compare equal have the same hash value; it is advised to mix together the hash values of the components of the object that also play a part in comparison of objects by packing them into a tuple and hashing the tuple.
In your example, both __eq__ and __hash__ only use self.value. This satisfies the only requirement.

Python : Argument based Singleton

I'm following this link and trying to make a singleton class. But, taking arguments (passed while initiating a class) into account so that the same object is returned if the arguments are same.
So, instead of storing class name/class reference as a dict key, I want to store passed arguments as keys in dict. But, there could be unhashable arguments also (like dict, set itself).
What is the best way to store class arguments and class objects mapping? So that I can return an object corresponding to the arguments.
Thanks anyways.
EDIT-1 :
A little more explanation. Let's say there is class as follows
class A:
__metaclass__ == Singleton
def __init__(arg1, arg2):
pass
Now, A(1,2) should always return the same object. But, it should be different from A(3,4)
I think, the arguments very much define the functioning of a class. Let's say if the class is to make redis connections. I might want to create 2 singletons objects with diff redis hosts as parameters, but the underlying class/code could be common.

As theheadofabroom and me already mentioned in the comments, there are some odds when relying on non-hashable values for instance caching or memoization. Therefore, if you still want to do exactly that, the following example does not hide the memoization in the __new__ or __init__ method. (A self-memoizing class would be hazardous because the memoization criterion can be fooled by code that you don't control).
Instead, I provide the function memoize which returns a memoizing factory function for a class. Since there is no generic way to tell from non-hashable arguments, if they will result in an instance that is equivalent to an already existing isntance, the memoization semantics have to be provided explicitly. This is achieved by passing the keyfunc function to memoize. keyfunc takes the same arguments as the class' __init__ method and returns a hashable key, whose equality relation (__eq__) determines memoization.
The proper use of the memoization is in the responsibility of the using code (providing a sensible keyfunc and using the factory), since the class to be memoized is not modified and can still be instantiated normally.
def memoize(cls, keyfunc):
memoized_instances = {}
def factory(*args, **kwargs):
key = keyfunc(*args, **kwargs)
if key in memoized_instances:
return memoized_instances[key]
instance = cls(*args, **kwargs)
memoized_instances[key] = instance
return instance
return factory
class MemoTest1(object):
def __init__(self, value):
self.value = value
factory1 = memoize(MemoTest1, lambda value : value)
class MemoTest2(MemoTest1):
def __init__(self, value, foo):
MemoTest1.__init__(self, value)
self.foo = foo
factory2 = memoize(MemoTest2, lambda value, foo : (value, frozenset(foo)))
m11 = factory1('test')
m12 = factory1('test')
assert m11 is m12
m21 = factory2('test', [1, 2])
lst = [1, 2]
m22 = factory2('test', lst)
lst.append(3)
m23 = factory2('test', lst)
assert m21 is m22
assert m21 is not m23
I only included MemoTest2 as a sublclass of MemoTest1 to show that there is no magic involved in using regular class inheritance.

Python - __eq__ method not being called

I have a set of objects, and am interested in getting a specific object from the set. After some research, I decided to use the solution provided here: http://code.activestate.com/recipes/499299/
The problem is that it doesn't appear to be working.
I have two classes defined as such:
class Foo(object):
def __init__(self, a, b, c):
self.a = a
self.b = b
self.c = c
def __key(self):
return (self.a, self.b, self.c)
def __eq__(self, other):
return self.__key() == other.__key()
def __hash__(self):
return hash(self.__key())
class Bar(Foo):
def __init__(self, a, b, c, d, e):
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
Note: equality of these two classes should only be defined on the attributes a, b, c.
The wrapper _CaptureEq in http://code.activestate.com/recipes/499299/ also defines its own __eq__ method. The problem is that this method never gets called (I think). Consider,
bar_1 = Bar(1,2,3,4,5)
bar_2 = Bar(1,2,3,10,11)
summary = set((bar_1,))
assert(bar_1 == bar_2)
bar_equiv = get_equivalent(summary, bar_2)
bar_equiv.d should equal 4 and likewise bar_equiv .e should equal 5, but they are not. Like I mentioned, it looks like the __CaptureEq __eq__ method does not get called when the statement bar_2 in summary is executed.
Is there some reason why the __CaptureEq __eq__ method is not being called? Hopefully this is not too obscure of a question.

Brandon's answer is informative, but incorrect. There are actually two problems, one with
the recipe relying on _CaptureEq being written as an old-style class (so it won't work properly if you try it on Python 3 with a hash-based container), and one with your own Foo.__eq__ definition claiming definitively that the two objects are not equal when it should be saying "I don't know, ask the other object if we're equal".
The recipe problem is trivial to fix: just define __hash__ on the comparison wrapper class:
class _CaptureEq:
'Object wrapper that remembers "other" for successful equality tests.'
def __init__(self, obj):
self.obj = obj
self.match = obj
# If running on Python 3, this will be a new-style class, and
# new-style classes must delegate hash explicitly in order to populate
# the underlying special method slot correctly.
# On Python 2, it will be an old-style class, so the explicit delegation
# isn't needed (__getattr__ will cover it), but it also won't do any harm.
def __hash__(self):
return hash(self.obj)
def __eq__(self, other):
result = (self.obj == other)
if result:
self.match = other
return result
def __getattr__(self, name): # support anything else needed by __contains__
return getattr(self.obj, name)
The problem with your own __eq__ definition is also easy to fix: return NotImplemented when appropriate so you aren't claiming to provide a definitive answer for comparisons with unknown objects:
class Foo(object):
def __init__(self, a, b, c):
self.a = a
self.b = b
self.c = c
def __key(self):
return (self.a, self.b, self.c)
def __eq__(self, other):
if not isinstance(other, Foo):
# Don't recognise "other", so let *it* decide if we're equal
return NotImplemented
return self.__key() == other.__key()
def __hash__(self):
return hash(self.__key())
With those two fixes, you will find that Raymond's get_equivalent recipe works exactly as it should:
>>> from capture_eq import *
>>> bar_1 = Bar(1,2,3,4,5)
>>> bar_2 = Bar(1,2,3,10,11)
>>> summary = set((bar_1,))
>>> assert(bar_1 == bar_2)
>>> bar_equiv = get_equivalent(summary, bar_2)
>>> bar_equiv.d
4
>>> bar_equiv.e
5
Update: Clarified that the explicit __hash__ override is only needed in order to correctly handle the Python 3 case.

The problem is that the set compares two objects the “wrong way around” for this pattern to intercept the call to __eq__(). The recipe from 2006 evidently was written against containers that, when asked if x was present, went through the candidate y values already present in the container doing:
x == y
comparisons, in which case an __eq__() on x could do special actions during the search. But the set object is doing the comparison the other way around:
y == x
for each y in the set. Therefore this pattern might simply not be usable in this form when your data type is a set. You can confirm this by instrumenting Foo.__eq__() like this:
def __eq__(self, other):
print '__eq__: I am', self.d, self.e, 'and he is', other.d, other.e
return self.__key() == other.__key()
You will then see a message like:
__eq__: I am 4 5 and he is 10 11
confirming that the equality comparison is posing the equality question to the object already in the set — which is, alas, not the object wrapped with Hettinger's _CaptureEq object.
Update:
And I forgot to suggest a way forward: have you thought about using a dictionary? Since you have an idea here of a key that is a subset of the data inside the object, you might find that splitting out the idea of the key from the idea of the object itself might alleviate the need to attempt this kind of convoluted object interception. Just write a new function that, given an object and your dictionary, computes the key and looks in the dictionary and returns the object already in the dictionary if the key is present else inserts the new object at the key.
Update 2: well, look at that — Nick's answer uses a NotImplemented in one direction to force the set to do the comparison in the other direction. Give the guy a few +1's!

There are two issues here. The first is that:
t = _CaptureEq(item)
if t in container:
return t.match
return default
Doesn't do what you think. In particular, t will never be in container, since _CaptureEq doesn't define __hash__. This becomes more obvious in Python 3, since it will point this out to you rather than providing a default __hash__. The code for _CaptureEq seems to believe that providing an __getattr__ will solve this - it won't, since Python's special method lookups are not guaranteed to go through all the same steps as normal attribute lookups - this is the same reason __hash__ (and various others) need to be defined on a class and can't be monkeypatched onto an instance. So, the most direct way around this is to define _CaptureEq.__hash__ like so:
def __hash__(self):
return hash(self.obj)
But that still isn't guaranteed to work, because of the second issue: set lookup is not guaranteed to test equality. sets are based on hashtables, and only do an equality test if there's more than one item in a hash bucket. You can't (and don't want to) force items that hash differently into the same bucket, since that's all an implementation detail of set. The easiest way around this issue, and to neatly sidestep the first one, is to use a list instead:
summary = [bar_1]
assert(bar_1 == bar_2)
bar_equiv = get_equivalent(summary, bar_2)
assert(bar_equiv is bar_1)

python - readable list of objects

This is probably a kinda commonly asked question but I could do with help on this. I have a list of class objects and I'm trying to figure out how to make it print an item from that class but rather than desplaying in the;
<__main__.evolutions instance at 0x01B8EA08>
but instead to show a selected attribute of a chosen object of the class. Can anyone help with that?

If you want to just display a particular attribute of each class instance, you can do
print([obj.attr for obj in my_list_of_objs])
Which will print out the attr attribute of each object in the list my_list_of_objs. Alternatively, you can define the __str__() method for your class, which specifies how to convert your objects into strings:
class evolutions:
def __str__(self):
# return string representation of self
print(my_list_of_objs) # each object is now printed out according to its __str__() method

You'll want to override your class's "to string" method:
class Foo:
def __str__(self):
return "String representation of me"

Checkout the __str__() and __repr__() methods.
See http://docs.python.org/reference/datamodel.html#object.__repr__

You need to override either the __str__, or __repr__ methods of your object[s]

My preference is to define a __repr__ function that can reconstruct the object (whenever possible). Unless you have a __str__ as well, both repr() and str() will call this method.
So for example
class Foo(object):
def __init__(self, a, b):
self.a = a
self.b = b
def __repr__(self):
return 'Foo(%r, %r)' % (self.a, self.b)
Doing it this way, you have a readable string version, and as a bonus it can be eval'ed to get a copy of the original object.
x = Foo(5, 1 + 1)
y = eval(str(x))
print y
-> Foo(5, 2)