I want to dynamically query which objects from a class I would like to retrieve. getattr seems like what I want, and it performs fine for top-level objects in the class. However, I'd like to also specify sub-elements.
class MyObj(object):
def __init__(self):
self.d = {'a':1, 'b':2}
self.c = 3
myobj = MyObj()
val = getattr(myobj, "c")
print val # Correctly prints 3
val = getattr(myobj, "d['a']") # Seemingly incorrectly formatted query
print val # Throws an AttributeError
How can I get the object's dictionary elements via a string?
The reason you're getting an error is that getattr(myobj, "d['a']") looks for an attribute named d['a'] on the object, and there isn't one. Your attribute is named d and it's a dictionary. Once you have a reference to the dictionary, then you can access items in it.
mydict = getattr(myobj, "d")
val = mydict["a"]
Or as others have shown, you can combine this in one step (I showed it as two to better illustrate what is actually happening):
val = getattr(myobj, "d")["a"]
Your question implies that you think that items of a dictionary in an object are "sub-elements" of the object. An item in a dictionary, however, is a different thing from an attribute of an object. (getattr() wouldn't work with something like o.a either, though; it just gets one attribute of one object. If that's an object too and you want to get one of its attributes, that's another getattr().)
You can pretty easily write a function that walks an attribute path (given in a string) and attempts to resolve each name either as a dictionary key or an attribute:
def resolve(obj, attrspec):
for attr in attrspec.split("."):
try:
obj = obj[attr]
except (TypeError, KeyError):
obj = getattr(obj, attr)
return obj
The basic idea here is that you take a path and for each component of the path, try to find either an item in a dictionary-like container or an attribute on an object. When you get to the end of the path, return what you've got. Your example would be resolve(myobj, "d.a")
You simply use square brackets to get the dictionary's element:
val = getattr(myobj, "d")["a"]
That'll set val to 1.
If you need the dictionary item to be dynamic as well, you'll need to call get on the result of getattr:
value = getattr(myobj, 'd').get('a')
Thanks to Kindall's answer, I found the following works well for dict keys that are stings.
class Obj2(object):
def __init__(self):
self.d = {'a':'A', 'b':'B', 'c': {'three': 3, 'twothree': (2,3)}}
self.c = 4
class MyObj(object):
def __init__(self):
self.d = {'a':1, 'b':2, 'c': {'two': 2, 'onetwo': (1,2)}}
self.c = 3
self.obj2 = Obj2()
def resolve(self, obj, attrspec):
attrssplit = attrspec.split(".")
attr = attrssplit[0]
try:
obj = obj[attr]
except (TypeError, KeyError):
obj = getattr(obj, attr)
if len(attrssplit) > 1:
attrspec = attrspec.partition(".")[2] # right part of the string.
return self.resolve(obj, attrspec) # Recurse
return obj
def __getattr__(self, name):
return self.resolve(self, name)
# Test
myobj = MyObj()
print getattr(myobj, "c")
print getattr(myobj, "d.a")
print getattr(myobj, "d.c.two")
print getattr(myobj, "obj2.d.a")
print getattr(myobj, "obj2.d.c.twothree")
Related
I have a complex data structure, essentially a dict where the keys are hashed instances. If I only know the hashed value that equals the key, how can I get back the instance? I can do it by brute force, but it seems I should be able to get the key/instance in O(1) time.
class Test:
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
self.arr = ["extra"]
def __str__(self):
return self.foo + self.bar
def __hash__(self):
return hash(str(self))
def __eq__(self, other):
return hash(self) == hash(other)
my_thing = Test("FOO", "BAR")
my_dict = dict()
my_dict[my_thing] = 1
for k, v in my_dict.iteritems():
if k == "FOOBAR":
print k.arr
Edit: I want to be able to get the mutable data in the instance (in this case the array). So that if I only know the hash of "FOOBAR" I would like to be able to get ["extra"], without having to traverse the entire dictionary matching keys (the for loop at the bottom)
The dict should map a key (foo, bar) to the data you want to retrieve. Here is one way you could implement this:
class Test(object):
...
def key(self):
return (self.foo, self.bar)
my_thing = Test("FOO", "BAR")
my_dict = {}
my_dict[my_thing.key()] = my_thing
print my_dict[("FOO", "BAR")].arr
Note that I modified your key function to avoid collisions like:
Test("FOO", "BAR") == Test("FOOB", "AR")
It sounds like you have the string "FOOBAR" (for example) and you want to retrieve a key k in your dict for which str(k) == "FOOBAR".
One way to do this is to just reconstruct a new Test object that will have the same string representation and use that for your lookup:
my_thing = my_dict(Test("FOO", "BAR"))
But this is inefficient and creates a dependency between object creation and your string representation.
If this string representation has its own intrinsic value as a key, you can just maintain your dict (or another dict) keyed by string instead:
my_index = dict()
my_index[str(my_thing)] = my_thing
That way, you can lookup your value given the string only:
print my_index["FOOBAR"].arr
I am using a technique discussed here before, to turn a dictionary into an object, so that I can access the elements of the dictionary with the dot (.) notion, as instance variables.
This is what I am doing:
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
So now, I can do:
print k.apple
and the result is:
1
This works, however the issues start if I try to add some other methods to the "Struct" class. For example lets say that I am adding a simple method that just creates an variable:
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
def testMe(self):
self.myVariable = 67
If I do:
k.testMe()
My dictionary object is broken, "myVariable" is inserted as a key with the value "67". So If I do:
print k.__dict__
I am getting:
{'apple': '1', 'house': '3', 'myVariable': 67, 'car': '4', 'banana': '2', 'hippopotamus': '5'}
Is there a way to fix this? I kind of understand what is happening, but not sure If I need to entirely change my approach and build a class with internal methods to handle the dictionary object or is there a simpler way to fix this problem?
Here is the original link:
Convert Python dict to object?
Thanks.
For your needs, don't store you variables in __dict__. Use your own dictionary instead, and override .__getattr__ (for print k.apple) and __setattr__ (for k.apple=2):
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
_dict = {}
def __init__(self, **entries):
self._dict = entries
def __getattr__(self, name):
try:
return self._dict[name]
except KeyError:
raise AttributeError(
"'{}' object has no attribute or key '{}'".format(
self.__class__.__name__, name))
def __setattr__(self, name, value):
if name in self._dict:
self._dict[name] = value
else:
self.__dict__[name] = value
def testMe(self):
self.myVariable = 67
def FormattedDump(self):
return str(self._dict)
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
print k.apple
print k.FormattedDump()
k.testMe()
k.apple = '2'
print k.FormattedDump()
In the alternative, if your FormattedDump() routine is bothering you, you could just fix it:
# Initial dictionary
myData = {'apple':'1', 'banana':'2', 'house':'3', 'car':'4', 'hippopotamus':'5'}
# Create the container class
class Struct:
def __init__(self, **entries):
self.__dict__.update(entries)
self.public_names = entries.keys()
def testMe(self):
self.myVariable = 67
def GetPublicDict(self):
return {key:getattr(self, key) for key in self.public_names}
def FormattedDump(self):
return str(self.GetPublicDict())
# Finally create the instance and bind the dictionary to it
k = Struct(**myData)
print k.apple
print k.FormattedDump()
k.testMe()
k.apple = '2'
print k.FormattedDump()
My class has a dict, for example:
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
Then I want to use the dict's key with MyClass instance to access the dict, for example:
ob = MyClass()
v = ob.a # Here I expect ob.a returns 'v1'
I know this should be implemented by __getattr__, but I'm new to Python, I don't exactly know how to implement it.
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
return self.data[attr]
>>> ob = MyClass()
>>> v = ob.a
>>> v
'v1'
Be careful when implementing __setattr__ though, you will need to make a few modifications:
class MyClass(object):
def __init__(self):
# prevents infinite recursion from self.data = {'a': 'v1', 'b': 'v2'}
# as now we have __setattr__, which will call __getattr__ when the line
# self.data[k] tries to access self.data, won't find it in the instance
# dictionary and return self.data[k] will in turn call __getattr__
# for the same reason and so on.... so we manually set data initially
super(MyClass, self).__setattr__('data', {'a': 'v1', 'b': 'v2'})
def __setattr__(self, k, v):
self.data[k] = v
def __getattr__(self, k):
# we don't need a special call to super here because getattr is only
# called when an attribute is NOT found in the instance's dictionary
try:
return self.data[k]
except KeyError:
raise AttributeError
>>> ob = MyClass()
>>> ob.c = 1
>>> ob.c
1
If you don't need to set attributes just use a namedtuple
eg.
>>> from collections import namedtuple
>>> MyClass = namedtuple("MyClass", ["a", "b"])
>>> ob = MyClass(a=1, b=2)
>>> ob.a
1
If you want the default arguments you can just write a wrapper class around it:
class MyClass(namedtuple("MyClass", ["a", "b"])):
def __new__(cls, a="v1", b="v2"):
return super(MyClass, cls).__new__(cls, a, b)
or maybe it looks nicer as a function:
def MyClass(a="v1", b="v2", cls=namedtuple("MyClass", ["a", "b"])):
return cls(a, b)
>>> ob = MyClass()
>>> ob.a
'v1'
Late to the party, but found two really good resources that explain this better (IMHO).
As explained here, you should use self.__dict__ to access fields from within __getattr__, in order to avoid infinite recursion. The example provided is:
def __getattr__(self, attrName):
if not self.__dict__.has_key(attrName):
value = self.fetchAttr(attrName) # computes the value
self.__dict__[attrName] = value
return self.__dict__[attrName]
Note: in the second line (above), a more Pythonic way would be (has_key apparently was even removed in Python 3):
if attrName not in self.__dict__:
The other resource explains that the __getattr__ is invoked only when the attribute is not found in the object, and that hasattr always returns True if there is an implementation for __getattr__. It provides the following example, to demonstrate:
class Test(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
def __getattr__(self, name):
return 123456
t = Test()
print 'object variables: %r' % t.__dict__.keys()
#=> object variables: ['a', 'b']
print t.a
#=> a
print t.b
#=> b
print t.c
#=> 123456
print getattr(t, 'd')
#=> 123456
print hasattr(t, 'x')
#=> True
class A(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
try:
return self.data[attr]
except Exception:
return "not found"
>>>a = A()
>>>print a.a
v1
>>>print a.c
not found
I like to take this therefore.
I took it from somewhere, but I don't remember where.
class A(dict):
def __init__(self, *a, **k):
super(A, self).__init__(*a, **k)
self.__dict__ = self
This makes the __dict__ of the object the same as itself, so that attribute and item access map to the same dict:
a = A()
a['a'] = 2
a.b = 5
print a.a, a['b'] # prints 2 5
I figured out an extension to #glglgl's answer that handles nested dictionaries and dictionaries insides lists that are in the original dictionary:
class d(dict):
def __init__(self, *a, **k):
super(d, self).__init__(*a, **k)
self.__dict__ = self
for k in self.__dict__:
if isinstance(self.__dict__[k], dict):
self.__dict__[k] = d(self.__dict__[k])
elif isinstance(self.__dict__[k], list):
for i in range(len(self.__dict__[k])):
if isinstance(self.__dict__[k][i], dict):
self.__dict__[k][i] = d(self.__dict__[k][i])
A simple approach to solving your __getattr__()/__setattr__() infinite recursion woes
Implementing one or the other of these magic methods can usually be easy. But when overriding them both, it becomes trickier. This post's examples apply mostly to this more difficult case.
When implementing both these magic methods, it's not uncommon to get stuck figuring out a strategy to get around recursion in the __init__() constructor of classes. This is because variables need to be initialized for the object, but every attempt to read or write those variables go through __get/set/attr__(), which could have more unset variables in them, incurring more futile recursive calls.
Up front, a key point to remember is that __getattr__() only gets called by the runtime if the attribute can't be found on the object already. The trouble is to get attributes defined without tripping these functions recursively.
Another point is __setattr__() will get called no matter what. That's an important distinction between the two functions, which is why implementing both attribute methods can be tricky.
This is one basic pattern that solves the problem.
class AnObjectProxy:
_initialized = False # *Class* variable 'constant'.
def __init__(self):
self._any_var = "Able to access instance vars like usual."
self._initialized = True # *instance* variable.
def __getattr__(self, item):
if self._initialized:
pass # Provide the caller attributes in whatever ways interest you.
else:
try:
return self.__dict__[item] # Transparent access to instance vars.
except KeyError:
raise AttributeError(item)
def __setattr__(self, key, value):
if self._initialized:
pass # Provide caller ways to set attributes in whatever ways.
else:
self.__dict__[key] = value # Transparent access.
While the class is initializing and creating it's instance vars, the code in both attribute functions permits access to the object's attributes via the __dict__ dictionary transparently - your code in __init__() can create and access instance attributes normally. When the attribute methods are called, they only access self.__dict__ which is already defined, thus avoiding recursive calls.
In the case of self._any_var, once it's assigned, __get/set/attr__() won't be called to find it again.
Stripped of extra code, these are the two pieces that are most important.
... def __getattr__(self, item):
... try:
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __setattr__(self, key, value):
... self.__dict__[key] = value
Solutions can build around these lines accessing the __dict__ dictionary. To implement an object proxy, two modes were implemented: initialization and post-initialization in the code before this - a more detailed example of the same is below.
There are other examples in answers that may have differing levels of effectiveness in dealing with all aspects of recursion. One effective approach is accessing __dict__ directly in __init__() and other places that need early access to instance vars. This works but can be a little verbose. For instance,
self.__dict__['_any_var'] = "Setting..."
would work in __init__().
My posts tend to get a little long-winded.. after this point is just extra. You should already have the idea with the examples above.
A drawback to some other approaches can be seen with debuggers in IDE's. They can be overzealous in their use of introspection and produce warning and error recovery messages as you're stepping through code. You can see this happening even with solutions that work fine standalone. When I say all aspects of recursion, this is what I'm talking about.
The examples in this post only use a single class variable to support 2-modes of operation, which is very maintainable.
But please NOTE: the proxy class required two modes of operation to set up and proxy for an internal object. You don't have to have two modes of operation.
You could simply incorporate the code to access the __dict__ as in these examples in whatever ways suit you.
If your requirements don't include two modes of operation, you may not need to declare any class variables at all. Just take the basic pattern and customize it.
Here's a closer to real-world (but by no means complete) example of a 2-mode proxy that follows the pattern:
>>> class AnObjectProxy:
... _initialized = False # This class var is important. It is always False.
... # The instances will override this with their own,
... # set to True.
... def __init__(self, obj):
... # Because __getattr__ and __setattr__ access __dict__, we can
... # Initialize instance vars without infinite recursion, and
... # refer to them normally.
... self._obj = obj
... self._foo = 123
... self._bar = 567
...
... # This instance var overrides the class var.
... self._initialized = True
...
... def __setattr__(self, key, value):
... if self._initialized:
... setattr(self._obj, key, value) # Proxying call to wrapped obj.
... else:
... # this block facilitates setting vars in __init__().
... self.__dict__[key] = value
...
... def __getattr__(self, item):
... if self._initialized:
... attr = getattr(self._obj, item) # Proxying.
... return attr
... else:
... try:
... # this block facilitates getting vars in __init__().
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __call__(self, *args, **kwargs):
... return self._obj(*args, **kwargs)
...
... def __dir__(self):
... return dir(self._obj) + list(self.__dict__.keys())
The 2-mode proxy only needs a bit of "bootstrapping" to access vars in its own scope at initialization before any of its vars are set. After initialization, the proxy has no reason to create more vars for itself, so it will fare fine by deferring all attribute calls to it's wrapped object.
Any attribute the proxy itself owns will still be accessible to itself and other callers since the magic attribute functions only get called if an attribute can't be found immediately on the object.
Hopefully this approach can be of benefit to anyone who appreciates a direct approach to resolving their __get/set/attr__() __init__() frustrations.
You can initialize your class dictionary through the constructor:
def __init__(self,**data):
And call it as follows:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
All of the instance attributes being accessed (read) in __setattr__, need to be declared using its parent (super) method, only once:
super().__setattr__('NewVarName1', InitialValue)
Or
super().__setattr__('data', dict())
Thereafter, they can be accessed or assigned to in the usual manner:
self.data = data
And instance attributes not being accessed in __setattr__, can be declared in the usual manner:
self.x = 1
The overridden __setattr__ method must now call the parent method inside itself, for new variables to be declared:
super().__setattr__(key,value)
A complete class would look as follows:
class MyClass(object):
def __init__(self, **data):
# The variable self.data is used by method __setattr__
# inside this class, so we will need to declare it
# using the parent __setattr__ method:
super().__setattr__('data', dict())
self.data = data
# These declarations will jump to
# super().__setattr__('data', dict())
# inside method __setattr__ of this class:
self.x = 1
self.y = 2
def __getattr__(self, name):
# This will callback will never be called for instance variables
# that have beed declared before being accessed.
if name in self.data:
# Return a valid dictionary item:
return self.data[name]
else:
# So when an instance variable is being accessed, and
# it has not been declared before, nor is it contained
# in dictionary 'data', an attribute exception needs to
# be raised.
raise AttributeError
def __setattr__(self, key, value):
if key in self.data:
# Assign valid dictionary items here:
self.data[key] = value
else:
# Assign anything else as an instance attribute:
super().__setattr__(key,value)
Test:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
f.a = 'c'
f.d = 'e'
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
print("f.d = ", f.d)
print("f.x = ", f.x)
print("f.y = ", f.y)
# Should raise attributed Error
print("f.g = ", f.g)
Output:
f.a = v1
f.b = v2
f.data = {'a': 'v1', 'b': 'v2'}
f.a = c
f.b = v2
f.data = {'a': 'c', 'b': 'v2'}
f.d = e
f.x = 1
f.y = 2
Traceback (most recent call last):
File "MyClass.py", line 49, in <module>
print("f.g = ", f.g)
File "MyClass.py", line 25, in __getattr__
raise AttributeError
AttributeError
I think this implement is cooler
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self,key):
return self.data.get(key,None)
I have a dictionary-like object which store descriptors inside:
class MyDict(object):
def __init__(self):
dict.__init__(self)
def __getitem__(self, key):
v = dict.__getitem__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v
class MyDescriptor(object):
def __init__(self, value, attrib={}):
self.__value = value
self.attrib= attrib
def __get__(self, instance, owner):
return self.__value
def __set__(self, instance, value):
self.__value = value
I want to be able to do the following:
d = MyDict()
d['1'] = MyDescriptor("123", {"name": "val"})
print d['1'] # prints "123"
print d['1'].attrib["name"] # prints "val"
My classes don't work. Could you please help me?
Your solution looks unnecessarily complicated to solve your problem, unless there's more to it that is shown. Why not simply do this:
class MyObject(object):
def __init__(value, attrib=None):
self.__value = value
self.attrib = {} if attrib is None else attrib
def __str__(self):
return __value
d = {}
d['1'] = MyObject("123", {"name": "val"})
print d['1'] # prints "123"
print d['1'].attrib["name"] # prints "val"
As for why your code doesn't work, there are a few obvious problems.
From your calls in various special methods of __dict__, it appears that MyDict is meant to subclass dict, so the definition should be:
class MyDict(dict):
...
While not incorrect, it is better practice to use super rather than referring to the
base class directly, so dict.__init__(self) would become super(MyDict, self).__init__() and dict.__getitem__(self, key) becomes super(MyDict, dict).__getitem__(key).
Your call to get sill work, but doesn't match the method specification. You should
call it as v.__get__(self, MyDict). However, the way you are using it actually makes __get__ redundant, and I think that this usage it where the main problem lies.
In class MyDescriptor, early binding will give you unexpected results for attrib. See my example above for a better way for declaring it.
I suspect that instead of a description, what you actually want is an object which looks like a string (for some definition of "looks like"), but has an attribute attrib. To do this, there is no need to try to create a descriptor, which is intended for a different use case altogether. My example above gives a class which satisfies the requirement of an object which "looks like" a string, where "looks like" means it prints a string, but here is another which may be more like what you want:
class MyString(str):
def __init__(self, value, attrib=None):
super(MyString, self).__init__(value)
self.attrib = {} if attrib is None else attrib
I'm not sure if this solves your use case, but in terms of achieving the results you've stated you could simply remove your MyDict class and use an ordinary dict:
d = {}
Then, add a __str__ method to the MyDescriptor class returning self.__value and you'll achieve the results you've described.
>>> d['1'] = MyDescriptor("123", {"name": "val"})
>>> d['1']
123
>>> d['1'].attrib["name"]
val
Sorry for the confusing title.
I would like to do the following: (Similar to defstruct in Lisp)
def mkstruct(structname, field_dict):
# create a function called "structname" and get/set functions
# called "structname_get(s, field_name)" and "structname_set(s, field_name, value)"
# Create a struct "lstnode"
mkstruct("lstnode", {ndkey : 0, nxt: None})
# Make a new struct
node = lstnode()
node_set(node, "ndkey", 5)
v = node_get(node, "ndkey") # v should be 5
This can be done in C with a macro define. The reason I am not using a class is because The "struct" I am creating will be "tied" to a database (just a text file in some format in this case). And I don't want to take up any memory associated with an object - I will represent the struct as a number (an object ID if you will)
This should be a step in the direction of what you want:
def mkstruct(name, attrs):
def init(self):
self.id = # not sure how you want to get the id
def getattr(self, attr):
if attr not in attrs:
raise AttributeError(attr)
# put your database lookup statement here
def setattr(self, attr, value):
if attr not in attrs:
raise AttributeError(attr)
# put your database update statement here
return type(
name,
(object,),
__init__=init,
__getattr__=getattr,
__setattr__=setattr)
lstnode = mkstruct("lstnode", ("ndkey", "nxt"))
Looks to me that what you're looking for is already provided by the type built-in:
def mkstruct(structname, field_dict):
return type(structname, (object,), field_dict)
lstnode = mkstruct("lstnode", {'ndkey' : 0, 'nxt': None})
node = lstnode()
node.ndkey = 5
v = node.ndkey
If you need just the keys in field_dict to be members of the structure, you can add '__slots__' to field_dict.
Note: This doesn't implement any setter or getter, but as pointed out already by the comments, this is not really needed when using classes.
It looks like that this isn't easy to do in python - after some research. The only way to add a inner function to the global namespace is to modify the globals() dict, which is rather awkward.
>>> def mkfunc(funcname):
... def func():
... print "my name is %s" % funcname
... func.__name__ = funcname
... return func
...
>>> mkfunc("abc")
<function abc at 0xb773ae64>
>>> globals()["abc"] = mkfunc("abc")
>>> abc()
my name is abc
As for my own problem, I am content to do the following:
def mkstruct(fields):
def maker(args):
# validate #args against #fields
oid = db_insert_row(fields)
return oid
def getter(oid, fieldname):
rec = db_retrieve(oid)
return rec[fieldname]
def setter(oid, fieldname, value):
db_update(oid, fieldname, value)
return (maker, getter, setter,)
lstnode, lstnode_get, lstnode_set = mkstruct({nodekey: 0, nxt: None})
n = lstnode(nodekey=5)