I'm building a python3 object Foo that inherit from list. Actually, it's a grammatical tagger for language analysis. I need the following behavior:
object Foo must be iterable
some attributes (strings or integers) are items inside the iterable.
some attributes are set in the Foo.__init__()
some are set later, inside a Foo.method()
attributes must be encapsulated so that any modification affect items inside the list
And I must save memory use, so i expect that :
Foo.foo0 is Foo[0] (not only Foo.attr[0] == Foo[0])
some other attributes do not need to be inside the "list".
I've found a way to do it, but I suspect there must be a cleaner and more pythonic way. Also,any improvement in performance and memory use is welcome. Here is my code :
class Foo(list):
def __init__(self, foo1, foo2):
# there is 13 attributes
# setters raise IndexError without this hack :
[self.append(x) for x in [None]*13]
# first pack of attribute
self.foo1 = foo1
self.foo2 = foo2
# ... etc
# getter and setter for init attributes
#property
def foo1(self): return self[0]
#foo1.setter
def foo1(self,value): self[0] = value
# same thing for foo2, foo3... etc
## getter and setter for other attributes
#property
def bar1(self): return self[10]
#bar1.setter
def bar1(self,value): self[10] = value
# same thing for bar2, bar3... etc
def method(self, value1, value2):
self.bar1 = value1
self.bar2 = value2
# ... etc
# init object
f= Foo('some string', 'some other string')
# later :
f.method('data', 'more data')
# expected behavior
assert f.foo1 is f[0]
assert f.bar1 is f[10]
I'm not sure I'm doing it right.. particularly with this hack:
[self.append(x) for x in [None]*13]
Can you think of a better way ?
EDIT :
After some benchmark, I definitely saved performance thanks to MartijnPieters comment :
self.extend(None for _ in range(13))
Related
I have inherited a project with many large classes constituent of nothing but class objects (integers, strings, etc). I'd like to be able to check if an attribute is present without needed to define a list of attributes manually.
Is it possible to make a python class iterable itself using the standard syntax? That is, I'd like to be able to iterate over all of a class's attributes using for attr in Foo: (or even if attr in Foo) without needing to create an instance of the class first. I think I can do this by defining __iter__, but so far I haven't quite managed what I'm looking for.
I've achieved some of what I want by adding an __iter__ method like so:
class Foo:
bar = "bar"
baz = 1
#staticmethod
def __iter__():
return iter([attr for attr in dir(Foo) if attr[:2] != "__"])
However, this does not quite accomplish what I'm looking for:
>>> for x in Foo:
... print(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'classobj' object is not iterable
Even so, this works:
>>> for x in Foo.__iter__():
... print(x)
bar
baz
Add the __iter__ to the metaclass instead of the class itself (assuming Python 2.x):
class Foo(object):
bar = "bar"
baz = 1
class __metaclass__(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
For Python 3.x, use
class MetaFoo(type):
def __iter__(self):
for attr in dir(self):
if not attr.startswith("__"):
yield attr
class Foo(metaclass=MetaFoo):
bar = "bar"
baz = 1
this is how we make a class object iterable. provide the class with a iter and a next() method, then you can iterate over class attributes or their values.you can leave the next() method if you want to, or you can define next() and raise StopIteration on some condition.
e.g:
class Book(object):
def __init__(self,title,author):
self.title = title
self.author = author
def __iter__(self):
for each in self.__dict__.values():
yield each
>>> book = Book('The Mill on the Floss','George Eliot')
>>> for each in book: each
...
'George Eliot'
'The Mill on the Floss'
this class iterates over attribute value of class Book.
A class object can be made iterable by providing it with a getitem method too.
e.g:
class BenTen(object):
def __init__(self, bentenlist):
self.bentenlist = bentenlist
def __getitem__(self,index):
if index <5:
return self.bentenlist[index]
else:
raise IndexError('this is high enough')
>>> bt_obj = BenTen([x for x in range(15)])
>>>for each in bt_obj:each
...
0
1
2
3
4
now when the object of BenTen class is used in a for-in loop, getitem is called with succesively higher index value, till it raises IndexError.
You can iterate over the class's unhidden attributes with for attr in (elem for elem in dir(Foo) if elem[:2] != '__').
A less horrible way to spell that is:
def class_iter(Class):
return (elem for elem in dir(Class) if elem[:2] != '__')
then
for attr in class_iter(Foo):
pass
class MetaItetaror(type):
def __iter__(cls):
return iter(
filter(
lambda k: not k[0].startswith('__'),
cls.__dict__.iteritems()
)
)
class Klass:
__metaclass__ = MetaItetaror
iterable_attr_names = {'x', 'y', 'z'}
x = 5
y = 6
z = 7
for v in Klass:
print v
An instance of enum.Enum happens to be iterable, and while it is not a general solution, it is a reasonable option for some use cases:
from enum import Enum
class Foo(Enum):
bar = "qux"
baz = 123
>>> print(*Foo)
Foo.bar Foo.baz
names = [m.name for m in Foo]
>>> print(*names)
bar baz
values = [m.value for m in Foo]
print(*values)
>>> qux 123
As with .__dict__, the order of iteration using this Enum based approach is the same as the order of definition.
You can make class members iterable within just a single line.
Despite the easy and compact code there are two mayor features included, additionally:
Type checking allows using additional class members not to be iterated.
The technique is also working if (public) class methods are defined. The proposals above using the "__" string checking filtering method propably fail in such cases.
# How to make class members iterable in a single line within Python (O. Simon, 14.4.2022)
# Includes type checking to allow additional class members not to be iterated
class SampleVector():
def __init__(self, x, y, name):
self.x = x
self.y = y
self.name = name
def __iter__(self):
return [value for value in self.__dict__.values() if isinstance(value, int) or isinstance(value, float)].__iter__()
if __name__ == '__main__':
v = SampleVector(4, 5, "myVector")
print (f"The content of sample vector '{v.name}' is:\n")
for m in v:
print(m)
This solution is fairly close and inspired by answer 12 from Hans Ginzel and Vijay Shanker.
I'm trying to build a class that returns a dictionary when you call it. For instance this code:
class foobar():
def __init__(self):
self.Dictionary = {}
self.DictAddition()
def DictAddition(self):
self.Dictionary["Foo"] = "Bar"
def __repr__(self):
return repr([self.Dictionary])
When I call the class in my script like so will output class 'foobar.foobar'
Object = getattr(foobar, foobar)
Data = Object()
print(type(Data))
All tho I can print Data and it will print as expected a Dictionary, but I can't loop through the dictionary as it gives a TypeError, object is not iterable. Is there a way I can really return a type Dictionary from a class?
with kind regards,
So you want an object that behaves just like a dictionary, except for some special behavior that occurs during object creation? Sounds like an excellent time to use inheritance.
class foobar(dict):
def __init__(self):
super(foobar, self).__init__()
self["Foo"] = "Bar"
data = foobar()
print data
for item in data:
print "Item:", item
Result:
{'Foo': 'Bar'}
Item: Foo
Now, printing and iteration and everything else a dict can do, can also be done with your foobar class.
I must say I don't really understand what you are trying to do here: just making repr print a dictionary doesn't make your class one. But if you want to enable iteration for a class, you will need to override the __iter__ method.
Does there exist special class in python to create empty objects? I tried object(), but it didn't allow me to add fields.
I want to use it like this:
obj = EmptyObject()
obj.foo = 'far'
obj.bar = 'boo'
Should I each time(in several independent scripts) define new class like this?
class EmptyObject:
pass
I use python2.7
types.SimpleNamespace was introduced with Python 3.3 to serve this exact purpose. The documentation also shows a simple way to implement it yourself in Python, so you can add it to your pre-Python 3.3 setup and use it as if it was there (note that the actual implementation is done in C):
class SimpleNamespace (object):
def __init__ (self, **kwargs):
self.__dict__.update(kwargs)
def __repr__ (self):
keys = sorted(self.__dict__)
items = ("{}={!r}".format(k, self.__dict__[k]) for k in keys)
return "{}({})".format(type(self).__name__, ", ".join(items))
def __eq__ (self, other):
return self.__dict__ == other.__dict__
But of course, if you don’t need its few features, a simple class Empty: pass does just the same.
If you are looking for a place holder object to which you can add arbitrary static members, then the closest I got is an empty lambda function.
obj = lambda: None # Dummy function
obj.foo = 'far'
obj.bar = 'boo'
print obj.foo, obj.bar
# far boo
Remember: obj is not an object of a class, object doesn't mean class instance, because in Python classes and functions are objects at runtime just like class instances
There is no types.SimpleNamespace in Python 2.7, you could use collections.namedtuple() for immutable objects instead:
>>> from collections import namedtuple
>>> FooBar = namedtuple('FooBar', 'foo bar')
>>> FooBar('bar', 'foo')
FooBar(foo='bar', bar='foo')
Or argparse.Namespace:
>>> from argparse import Namespace
>>> o = Namespace(foo='bar')
>>> o.bar = 'foo'
>>> o
Namespace(bar='foo', foo='bar')
See also, How can I create an object and add attributes to it?
You can create a new type dynamically with the fields you want it to have using the type function, like this:
x = type('empty', (object,), {'foo': 'bar'})
x.bar = 3
print(x.foo)
This is not entirely what you want though, since it will have a custom type, not an empty type.
I have a class where I add some attributes dynamically and at some point I want to restore the class to it's pristine condition without the added attributes.
The situation:
class Foo(object):
pass
Foo.x = 1
# <insert python magic here>
o = Foo() # o should not have any of the previously added attributes
print o.x # Should raise exception
My initial thought was to create a copy of the original class:
class _Foo(object):
pass
Foo = _Foo
Foo.x = 1
Foo = _Foo # Clear added attributes
o = Foo()
print o.x # Should raise exception
But since Foo is just a reference to _Foo any attributes get added to the original _Foo as well. I also tried
Foo = copy.deepcopy(_Foo)
in case that would help but apparently it does not.
clarification:
The user should not need to care about how the class is implemented. It should, therefore, have the same features of a "normally defined" class, i.e. introspection, built-in help, subclassing, etc. This pretty much rules out anything based on __getattr__
I agree with Glenn that this is a horribly broken idea. Anyways, here how you'd do it with a decorator. Thanks to Glenn's post as well for reminding me that you can delete items from a class's dictionary, just not directly. Here's the code.
def resetable(cls):
cls._resetable_cache_ = cls.__dict__.copy()
return cls
def reset(cls):
cache = cls._resetable_cache_ # raises AttributeError on class without decorator
for key in [key for key in cls.__dict__ if key not in cache]:
delattr(cls, key)
for key, value in cache.items(): # reset the items to original values
try:
setattr(cls, key, value)
except AttributeError:
pass
I'm torn on whether to reset the values by catching attempts to update non-assignable attributes with a try as I've shown or building a list of such attributes. I'll leave it up to you.
And here's a use:
#resetable # use resetable on a class that you want to do this with
class Foo(object):
pass
Foo.x = 1
print Foo.x
reset(Foo)
o = Foo()
print o.x # raises AttributeError as expected
You can use inspect and maintain an original list of members and than delete all members that are not in the original list
import inspect
orig_members = []
for name, ref in inspect.getmembers(o):
orig_members.append(name)
...
Now, when you need to restore back to original
for name, ref in inspect.getmembers(o):
if name in orig_members:
pass
else:
#delete ref here
You have to record the original state and restore it explicitly. If the value existed before you changed it, restore that value; otherwise delete the value you set.
class Foo(object):
pass
try:
original_value = getattr(Foo, 'x')
originally_existed = True
except AttributeError:
originally_existed = False
Foo.x = 1
if originally_existed:
Foo.x = original_value
else:
del Foo.x
o = Foo() # o should not have any of the previously added attributes
print o.x # Should raise exception
You probably don't want to be doing this. There are valid cases for monkey patching, but you generally don't want to try to monkey unpatch. For example, if two independent bits of code monkey patch the same class, one of them trying to reverse the action without being aware of the other is likely to break things. For an example of a case where this is actually useful, see https://stackoverflow.com/questions/3829742#3829849.
The simplest way I found was this:
def foo_maker():
class Foo(object):
pass
return Foo
Foo = foo_maker()
Foo.x = 1
Foo = foo_maker() # Foo is now clean again
o = Foo() # Does not have any of the previously added attributes
print o.x # Raises exception
edit: As pointed out in comments, does not actually reset class but has the same effect in practice.
In your second example you're making a reference to the class rather than an instance.
Foo = _Foo # Reference
If you instead made an instance copy, what you want to do is exactly the way it will work. You can modify the instance all you want and 'revert' it by creating a new instance.
Foo = _Foo()
#!/usr/bin/python
class FooClass(object):
pass
FooInstance = FooClass() # Create an instance
FooInstance.x = 100 # Modify the instance
print dir(FooClass) # Verify FooClass doesn't have an 'x' attribute
FooInstance = FooClass() # Creates a new instance
print FooInstance.x # Exception
I don't know if you can accept an additional module file for class, if you can:
my_class.py
class Foo(object):
pass
You main script:
import my_class
Foo = my_class.Foo
Foo.x = 1
p = Foo()
print p.x # Printing '1'
# Some code....
reload(my_class) # reload to reset
Foo = my_class.Foo
o = Foo()
print p.x # Printing '1'
print o.__class__ == p.__class__ # Printing 'False'
print o.x # Raising exception
I am not sure if there is any side-effect. It seems to do what OP wants, though this is really unusal.
I don't fully understand why you need this, but I'll have a go. Ordinary inheritance probably won't do because you want to 'reset' to the old state. How about a proxy pattern?
class FooProxy(object):
def __init__(self, f):
self.f = foo
self.magic = {}
def set_magic(self, k, v):
self.magic[k] = v
def get_magic(self, k):
return self.magic.get(k)
def __getattr__(self, k):
return getattr(self.f, k)
def __setattr__(self, k, v):
setattr(self.f, k, v)
f = Foo()
p = FooProxy(f)
p.set_magic('m_bla', 123)
use f for ordinary, 'original' access, use p for proxied access, it should behave mostly like Foo. Re-proxy f with new configuration if you need to
I don't understand what you are trying to do, but keep in mind that you don't have to add attributes to the class in order to make it look like you added attributes to the class.
You can give the class a __getattr__ method that will be invoked for any missing attribute. Where it gets the value from is up to you:
class MyTrickyClass(object):
self.magic_prefix = "m_"
self.other_attribute_source = SomeOtherObject()
def __getattr__(self, name):
if name.startswith(self.magic_prefix):
stripped = name[len(self.magic_prefix):]
return getattr(self.other_attribute_source, stripped)
raise AttributeError
m = MyTrickyClass()
assert hasattr(m, "m_other")
MyTrickyClass.magic_prefix = "f_"
assert hasattr(m, "f_other")
If all the stuff you added starts with a given distinctive prefix, you could search the object's __dict__ for members with that prefix, and delete them, when it's time to restore.
To create a deep copy of a class you can use the new.classobj function
class Foo:
pass
import new, copy
FooSaved = new.classobj(Foo.__name__, Foo.__bases__, copy.deepcopy(Foo.__dict__))
# ...play with original class Foo...
# revert changes
Foo = FooSaved
UPD: module new is deprecated. Instead you should use types.ClassType with the same args
I need to "flatten" objects into nested dicts of the object's properties. The objects I want to do this with are generally just containers for basic types or other objects which act in a similar way. For example:
class foo(object):
bar = None
baz = None
class spam(object):
eggs = []
x = spam()
y = foo()
y.bar = True
y.baz = u"boz"
x.eggs.append(y)
What I need to "flatten" this to is:
{ 'eggs': [ { 'bar': True, 'baz': u'boz' } ] }
Is there anything in the stdlib which can do this for me? If not, would I have to test isinstance against all known base-types to ensure I don't try to convert an object which can't be converted (eg: bool)?
Edit:
These are objects are being returned to my code from an external library and therefore I have no control over them. I could use them as-is in my methods, but it would be easier (safer?) to convert them to dicts - especially for unit testing.
Code: You may need to handle other iterable types though:
def flatten(obj):
if obj is None:
return None
elif hasattr(obj, '__dict__') and obj.__dict__:
return dict([(k, flatten(v)) for (k, v) in obj.__dict__.items()])
elif isinstance(obj, (dict,)):
return dict([(k, flatten(v)) for (k, v) in obj.items()])
elif isinstance(obj, (list,)):
return [flatten(x) for x in obj]
elif isinstance(obj, (tuple,)):
return tuple([flatten(x) for x in obj])
else:
return obj
Bug?
In your code instead of:
class spam(object):
eggs = []
x = spam()
x.eggs.add(...)
please do:
class spam(object):
eggs = None #// if you need this line at all though
x = spam()
x.eggs = []
x.eggs.add(...)
If you do not, then all instances of spam will share the same eggs list.
No, there is nothing in the standardlib. Yes, you would have to somehow test that the types are basic types like str, unicode, bool, int, float, long...
You could probably make a registry of methods to "serialize" different types, but that would only be useful if you have some types that should not have all it's attributes serialized, for example, or if you also need to flatten class attributes, etc.
Almost every object has a dictionary (called __dict__), with all its methods and members.
With some type checking, you can then write a function that filters out only the members you are interested in.
It is not a big task, but as chrispy said, it could worth to try looking at your problem from a completely different perspective.
Well, I'm not very proud of this, but is possible to do the following:
Create a super class that has the serialization method and loop through its properties.
At runtime extend your classes using bases at runtime
Execute the class from the new super class. It should be able to access the dict data from the children and work.
Here is an example:
class foo(object):
def __init__(self):
self.bar = None
self.baz = None
class spam(object):
delf __init__(self):
self.eggs = []
class Serializable():
def serialize(self):
result = {}
for property in self.__dict__.keys():
result[property] = self.__dict__[property]
return result
foo.__bases__ += (Serializable,)
spam.__bases__ += (Serializable,)
x = spam()
y = foo()
y.bar = True
y.baz = u"boz"
x.eggs.append(y)
y.serialize()
Things to point out. If you do not set the var is init the dict willnot work 'cause it is accessing the instance variables not the class variables (I suppose you meant instance ones). Second, make sure Serializable DOES NOT inherit from object, it is does you will have a
TypeError: Error when calling the metaclass bases
Cannot create a consistent method resolution
Hope it helps!
Edit: If you are just copying the dict use the deepcopy module, this is just an example :P