I have inherited a project with many large classes constituent of nothing but class objects (integers, strings, etc). I'd like to be able to check if an attribute is present without needed to define a list of attributes manually.
Is it possible to make a python class iterable itself using the standard syntax? That is, I'd like to be able to iterate over all of a class's attributes using for attr in Foo: (or even if attr in Foo) without needing to create an instance of the class first. I think I can do this by defining __iter__, but so far I haven't quite managed what I'm looking for.
I've achieved some of what I want by adding an __iter__ method like so:
class Foo:
bar = "bar"
baz = 1
#staticmethod
def __iter__():
return iter([attr for attr in dir(Foo) if attr[:2] != "__"])
However, this does not quite accomplish what I'm looking for:
>>> for x in Foo:
... print(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'classobj' object is not iterable
Even so, this works:
>>> for x in Foo.__iter__():
... print(x)
bar
baz
Add the __iter__ to the metaclass instead of the class itself (assuming Python 2.x):
class Foo(object):
bar = "bar"
baz = 1
class __metaclass__(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
For Python 3.x, use
class MetaFoo(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
class Foo(metaclass=MetaFoo):
bar = "bar"
baz = 1
this is how we make a class object iterable. provide the class with a iter and a next() method, then you can iterate over class attributes or their values.you can leave the next() method if you want to, or you can define next() and raise StopIteration on some condition.
e.g:
class Book(object):
def __init__(self,title,author):
self.title = title
self.author = author
def __iter__(self):
for each in self.__dict__.values():
yield each
>>> book = Book('The Mill on the Floss','George Eliot')
>>> for each in book: each
...
'George Eliot'
'The Mill on the Floss'
this class iterates over attribute value of class Book.
A class object can be made iterable by providing it with a getitem method too.
e.g:
class BenTen(object):
def __init__(self, bentenlist):
self.bentenlist = bentenlist
def __getitem__(self,index):
if index <5:
return self.bentenlist[index]
else:
raise IndexError('this is high enough')
>>> bt_obj = BenTen([x for x in range(15)])
>>>for each in bt_obj:each
...
0
1
2
3
4
now when the object of BenTen class is used in a for-in loop, getitem is called with succesively higher index value, till it raises IndexError.
You can iterate over the class's unhidden attributes with for attr in (elem for elem in dir(Foo) if elem[:2] != '__').
A less horrible way to spell that is:
def class_iter(Class):
return (elem for elem in dir(Class) if elem[:2] != '__')
then
for attr in class_iter(Foo):
pass
class MetaItetaror(type):
def __iter__(cls):
return iter(
filter(
lambda k: not k[0].startswith('__'),
cls.__dict__.iteritems()
)
)
class Klass:
__metaclass__ = MetaItetaror
iterable_attr_names = {'x', 'y', 'z'}
x = 5
y = 6
z = 7
for v in Klass:
print v
An instance of enum.Enum happens to be iterable, and while it is not a general solution, it is a reasonable option for some use cases:
from enum import Enum
class Foo(Enum):
bar = "qux"
baz = 123
>>> print(*Foo)
Foo.bar Foo.baz
names = [m.name for m in Foo]
>>> print(*names)
bar baz
values = [m.value for m in Foo]
print(*values)
>>> qux 123
As with .__dict__, the order of iteration using this Enum based approach is the same as the order of definition.
You can make class members iterable within just a single line.
Despite the easy and compact code there are two mayor features included, additionally:
Type checking allows using additional class members not to be iterated.
The technique is also working if (public) class methods are defined. The proposals above using the "__" string checking filtering method propably fail in such cases.
# How to make class members iterable in a single line within Python (O. Simon, 14.4.2022)
# Includes type checking to allow additional class members not to be iterated
class SampleVector():
def __init__(self, x, y, name):
self.x = x
self.y = y
self.name = name
def __iter__(self):
return [value for value in self.__dict__.values() if isinstance(value, int) or isinstance(value, float)].__iter__()
if __name__ == '__main__':
v = SampleVector(4, 5, "myVector")
print (f"The content of sample vector '{v.name}' is:\n")
for m in v:
print(m)
This solution is fairly close and inspired by answer 12 from Hans Ginzel and Vijay Shanker.
I would like to get the auto-completion on a dictionary or some similar object in order to access objects. Here what I currently have:
class Foo:
def DoFoo(self):
pass
list = {"x": Foo(), "y": Foo()}
Now I would like to be able to do:
list.x.DoFoo()
How can I populate a list with objects that I can later access with the IPython auto-completion?
Do I need some kind of "class factory" as I saw here?
An instance of a custom class can work as a container for attributes you set on it. This can start trivially simple:
class MyContainer:
pass
a = MyContainer()
a.x = Foo()
a.y = Foo()
a.x.DoFoo()
You can make it a little more complicated to get the ability to pass in the attributes you want as part of the constructor (using the same semantics as the dict constructor):
def MyContainer:
def __init__(self, vals={}, **kwvals):
self.__dict__.update(vals, **kwvals)
a = MyContainer({"x":Foo()}, y=Foo()) # there are multiple ways to pass the args
a.x.DoFoo()
If you don't need to add or remove values after you create a container of this sort, you can instead use the standard library's collections.namedtuple class factory. You pass it a class name and a sequence of attribute names and it will return you a new class.
from collections import namedtuple
A = namedtuple("A", ["x", "y"])
a = A(Foo(), y=Foo()) # can use positional or keyword arguments
a.x.DoFoo() # this still works!
a.x = "something else" # but this will raise an error
You need to override __getattr__:
class Foo(object):
attr_a = 'default_attr_a_value'
attr_b = 'default_attr_b_value'
def __init__(self, *args, **kwargs):
if len(args) >= 2:
self.attr_a = args[0]
self.attr_b = args[1]
if len(args) == 1:
self.attr_a = args[0]
# holds the "emulated" attributes
self._properties = kwargs
def __getattr__(self, name):
if name in self._properties:
return self._properties[name]
else:
raise AttributeError
def __hasattr__(self, name):
return name in self._properties
bar = Foo('modified_value_of_attr_a', **{'aaa':1,'bbb':2})
print bar.attr_a
print bar.attr_b
print bar.aaa
try:
print bar.sss
except AttributeError:
print 'Attribute error raised!'
list_attrs = ['attr_a', 'aaa', 'sss']
for a in list_attrs:
if hasattr(bar, a):
print 'bar, instance of Foo, has an attribute named "%s"' % a
else:
print 'bar, instance of Foo, doesn\'t have an attribute named "%s"' % a
More to read
I've seen a class in python which does something of this kind:
obj = Class( name = "somename" )
obj.somename = something
Namely the class initialisation created a member called as the argument string.
I cannot manage to reproduce this behaviour. Any ideas?
I managed to do something similar using globals()["name"] = value. But in this case the created object is son of the module not of the "Class" object. And it's callable as module.somename instead of obj.somename.
You can easily create a dynamically named member by using setattr:
>>> class Foo(object):
... def __init__(self, name):
... setattr(self, name, 42)
...
>>> f = Foo('bar')
>>> f.bar
42
>>> f.bar = 'hello'
>>> f.bar
'hello'
Note however, that just setting any attribute on an instance of a class is possible for any regular class (that doesn't define __slots__):
>>> class Qux(object):
... pass
...
>>> q = Qux()
>>> q.foobar = 'hello'
>>> q.foobar
'hello'
You can either create a dynamically named fixed value property like this:
class MyClass():
def __init__(self, attr):
self.__dict__[attr] = 'some_value'
Or create as many properties as the calling function wants:
class MyClass():
def __init__(self, *argv, **kwargs):
for key,value in kwargs.items():
self.__dict__[key] = value
My class has a dict, for example:
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
Then I want to use the dict's key with MyClass instance to access the dict, for example:
ob = MyClass()
v = ob.a # Here I expect ob.a returns 'v1'
I know this should be implemented by __getattr__, but I'm new to Python, I don't exactly know how to implement it.
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
return self.data[attr]
>>> ob = MyClass()
>>> v = ob.a
>>> v
'v1'
Be careful when implementing __setattr__ though, you will need to make a few modifications:
class MyClass(object):
def __init__(self):
# prevents infinite recursion from self.data = {'a': 'v1', 'b': 'v2'}
# as now we have __setattr__, which will call __getattr__ when the line
# self.data[k] tries to access self.data, won't find it in the instance
# dictionary and return self.data[k] will in turn call __getattr__
# for the same reason and so on.... so we manually set data initially
super(MyClass, self).__setattr__('data', {'a': 'v1', 'b': 'v2'})
def __setattr__(self, k, v):
self.data[k] = v
def __getattr__(self, k):
# we don't need a special call to super here because getattr is only
# called when an attribute is NOT found in the instance's dictionary
try:
return self.data[k]
except KeyError:
raise AttributeError
>>> ob = MyClass()
>>> ob.c = 1
>>> ob.c
1
If you don't need to set attributes just use a namedtuple
eg.
>>> from collections import namedtuple
>>> MyClass = namedtuple("MyClass", ["a", "b"])
>>> ob = MyClass(a=1, b=2)
>>> ob.a
1
If you want the default arguments you can just write a wrapper class around it:
class MyClass(namedtuple("MyClass", ["a", "b"])):
def __new__(cls, a="v1", b="v2"):
return super(MyClass, cls).__new__(cls, a, b)
or maybe it looks nicer as a function:
def MyClass(a="v1", b="v2", cls=namedtuple("MyClass", ["a", "b"])):
return cls(a, b)
>>> ob = MyClass()
>>> ob.a
'v1'
Late to the party, but found two really good resources that explain this better (IMHO).
As explained here, you should use self.__dict__ to access fields from within __getattr__, in order to avoid infinite recursion. The example provided is:
def __getattr__(self, attrName):
if not self.__dict__.has_key(attrName):
value = self.fetchAttr(attrName) # computes the value
self.__dict__[attrName] = value
return self.__dict__[attrName]
Note: in the second line (above), a more Pythonic way would be (has_key apparently was even removed in Python 3):
if attrName not in self.__dict__:
The other resource explains that the __getattr__ is invoked only when the attribute is not found in the object, and that hasattr always returns True if there is an implementation for __getattr__. It provides the following example, to demonstrate:
class Test(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
def __getattr__(self, name):
return 123456
t = Test()
print 'object variables: %r' % t.__dict__.keys()
#=> object variables: ['a', 'b']
print t.a
#=> a
print t.b
#=> b
print t.c
#=> 123456
print getattr(t, 'd')
#=> 123456
print hasattr(t, 'x')
#=> True
class A(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
try:
return self.data[attr]
except Exception:
return "not found"
>>>a = A()
>>>print a.a
v1
>>>print a.c
not found
I like to take this therefore.
I took it from somewhere, but I don't remember where.
class A(dict):
def __init__(self, *a, **k):
super(A, self).__init__(*a, **k)
self.__dict__ = self
This makes the __dict__ of the object the same as itself, so that attribute and item access map to the same dict:
a = A()
a['a'] = 2
a.b = 5
print a.a, a['b'] # prints 2 5
I figured out an extension to #glglgl's answer that handles nested dictionaries and dictionaries insides lists that are in the original dictionary:
class d(dict):
def __init__(self, *a, **k):
super(d, self).__init__(*a, **k)
self.__dict__ = self
for k in self.__dict__:
if isinstance(self.__dict__[k], dict):
self.__dict__[k] = d(self.__dict__[k])
elif isinstance(self.__dict__[k], list):
for i in range(len(self.__dict__[k])):
if isinstance(self.__dict__[k][i], dict):
self.__dict__[k][i] = d(self.__dict__[k][i])
A simple approach to solving your __getattr__()/__setattr__() infinite recursion woes
Implementing one or the other of these magic methods can usually be easy. But when overriding them both, it becomes trickier. This post's examples apply mostly to this more difficult case.
When implementing both these magic methods, it's not uncommon to get stuck figuring out a strategy to get around recursion in the __init__() constructor of classes. This is because variables need to be initialized for the object, but every attempt to read or write those variables go through __get/set/attr__(), which could have more unset variables in them, incurring more futile recursive calls.
Up front, a key point to remember is that __getattr__() only gets called by the runtime if the attribute can't be found on the object already. The trouble is to get attributes defined without tripping these functions recursively.
Another point is __setattr__() will get called no matter what. That's an important distinction between the two functions, which is why implementing both attribute methods can be tricky.
This is one basic pattern that solves the problem.
class AnObjectProxy:
_initialized = False # *Class* variable 'constant'.
def __init__(self):
self._any_var = "Able to access instance vars like usual."
self._initialized = True # *instance* variable.
def __getattr__(self, item):
if self._initialized:
pass # Provide the caller attributes in whatever ways interest you.
else:
try:
return self.__dict__[item] # Transparent access to instance vars.
except KeyError:
raise AttributeError(item)
def __setattr__(self, key, value):
if self._initialized:
pass # Provide caller ways to set attributes in whatever ways.
else:
self.__dict__[key] = value # Transparent access.
While the class is initializing and creating it's instance vars, the code in both attribute functions permits access to the object's attributes via the __dict__ dictionary transparently - your code in __init__() can create and access instance attributes normally. When the attribute methods are called, they only access self.__dict__ which is already defined, thus avoiding recursive calls.
In the case of self._any_var, once it's assigned, __get/set/attr__() won't be called to find it again.
Stripped of extra code, these are the two pieces that are most important.
... def __getattr__(self, item):
... try:
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __setattr__(self, key, value):
... self.__dict__[key] = value
Solutions can build around these lines accessing the __dict__ dictionary. To implement an object proxy, two modes were implemented: initialization and post-initialization in the code before this - a more detailed example of the same is below.
There are other examples in answers that may have differing levels of effectiveness in dealing with all aspects of recursion. One effective approach is accessing __dict__ directly in __init__() and other places that need early access to instance vars. This works but can be a little verbose. For instance,
self.__dict__['_any_var'] = "Setting..."
would work in __init__().
My posts tend to get a little long-winded.. after this point is just extra. You should already have the idea with the examples above.
A drawback to some other approaches can be seen with debuggers in IDE's. They can be overzealous in their use of introspection and produce warning and error recovery messages as you're stepping through code. You can see this happening even with solutions that work fine standalone. When I say all aspects of recursion, this is what I'm talking about.
The examples in this post only use a single class variable to support 2-modes of operation, which is very maintainable.
But please NOTE: the proxy class required two modes of operation to set up and proxy for an internal object. You don't have to have two modes of operation.
You could simply incorporate the code to access the __dict__ as in these examples in whatever ways suit you.
If your requirements don't include two modes of operation, you may not need to declare any class variables at all. Just take the basic pattern and customize it.
Here's a closer to real-world (but by no means complete) example of a 2-mode proxy that follows the pattern:
>>> class AnObjectProxy:
... _initialized = False # This class var is important. It is always False.
... # The instances will override this with their own,
... # set to True.
... def __init__(self, obj):
... # Because __getattr__ and __setattr__ access __dict__, we can
... # Initialize instance vars without infinite recursion, and
... # refer to them normally.
... self._obj = obj
... self._foo = 123
... self._bar = 567
...
... # This instance var overrides the class var.
... self._initialized = True
...
... def __setattr__(self, key, value):
... if self._initialized:
... setattr(self._obj, key, value) # Proxying call to wrapped obj.
... else:
... # this block facilitates setting vars in __init__().
... self.__dict__[key] = value
...
... def __getattr__(self, item):
... if self._initialized:
... attr = getattr(self._obj, item) # Proxying.
... return attr
... else:
... try:
... # this block facilitates getting vars in __init__().
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __call__(self, *args, **kwargs):
... return self._obj(*args, **kwargs)
...
... def __dir__(self):
... return dir(self._obj) + list(self.__dict__.keys())
The 2-mode proxy only needs a bit of "bootstrapping" to access vars in its own scope at initialization before any of its vars are set. After initialization, the proxy has no reason to create more vars for itself, so it will fare fine by deferring all attribute calls to it's wrapped object.
Any attribute the proxy itself owns will still be accessible to itself and other callers since the magic attribute functions only get called if an attribute can't be found immediately on the object.
Hopefully this approach can be of benefit to anyone who appreciates a direct approach to resolving their __get/set/attr__() __init__() frustrations.
You can initialize your class dictionary through the constructor:
def __init__(self,**data):
And call it as follows:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
All of the instance attributes being accessed (read) in __setattr__, need to be declared using its parent (super) method, only once:
super().__setattr__('NewVarName1', InitialValue)
Or
super().__setattr__('data', dict())
Thereafter, they can be accessed or assigned to in the usual manner:
self.data = data
And instance attributes not being accessed in __setattr__, can be declared in the usual manner:
self.x = 1
The overridden __setattr__ method must now call the parent method inside itself, for new variables to be declared:
super().__setattr__(key,value)
A complete class would look as follows:
class MyClass(object):
def __init__(self, **data):
# The variable self.data is used by method __setattr__
# inside this class, so we will need to declare it
# using the parent __setattr__ method:
super().__setattr__('data', dict())
self.data = data
# These declarations will jump to
# super().__setattr__('data', dict())
# inside method __setattr__ of this class:
self.x = 1
self.y = 2
def __getattr__(self, name):
# This will callback will never be called for instance variables
# that have beed declared before being accessed.
if name in self.data:
# Return a valid dictionary item:
return self.data[name]
else:
# So when an instance variable is being accessed, and
# it has not been declared before, nor is it contained
# in dictionary 'data', an attribute exception needs to
# be raised.
raise AttributeError
def __setattr__(self, key, value):
if key in self.data:
# Assign valid dictionary items here:
self.data[key] = value
else:
# Assign anything else as an instance attribute:
super().__setattr__(key,value)
Test:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
f.a = 'c'
f.d = 'e'
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
print("f.d = ", f.d)
print("f.x = ", f.x)
print("f.y = ", f.y)
# Should raise attributed Error
print("f.g = ", f.g)
Output:
f.a = v1
f.b = v2
f.data = {'a': 'v1', 'b': 'v2'}
f.a = c
f.b = v2
f.data = {'a': 'c', 'b': 'v2'}
f.d = e
f.x = 1
f.y = 2
Traceback (most recent call last):
File "MyClass.py", line 49, in <module>
print("f.g = ", f.g)
File "MyClass.py", line 25, in __getattr__
raise AttributeError
AttributeError
I think this implement is cooler
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self,key):
return self.data.get(key,None)
how do i create object-instances on runtime in python?
say i have 2 classes:
class MyClassA(object):
def __init__(self, prop):
self.prop = prop
self.name = "CLASS A"
def println(self):
print self.name
class MyClassB(object):
def __init__(self, prop):
self.prop = prop
self.name = "CLASS B"
def println(self):
print self.name
and a dict
{('a': MyClassA), ('b': MyClassB)}
how can i create dynamic an instance of one of my two classes, depending of i choose 'a' or 'b'.
kind of this:
somefunc(str):
if 'a': return new MyClassA
if 'b': return new MyClassB
to get "CLASS B" on calling: somefunc('a').println
but in a more elegant and dynamic way (say i add more classes to the dict on runtime)
You might create a dispatcher, which is a dictionary with your keys mapping to classes.
dispatch = {
"a": MyClassA,
"b": MyClassB,
}
instance = dispatch[which_one]() # Notice the second pair of parens here!
You create a class instance by calling the class. Your class dict {('a': MyClassA), ('b': MyClassB)} returns classes; so you need only call the class:
classes['a']()
But I get the sense you want something more specific. Here's a subclass of dict that, when called with a key, looks up the associated item and calls it:
>>> class ClassMap(dict):
... def __call__(self, key, *args, **kwargs):
... return self.__getitem__(key)(*args, **kwargs)
...
>>> c = ClassMap()
>>> c['a'] = A
>>> c['b'] = B
>>> c('a')
<__main__.A object at 0x1004cc7d0>