What is the best approach to take if you want to dynamically create and reference nested attributes?
I was writing a simple Flickr client, and wanted to match the documented API as closely as possible, without actually defining every method. For instance, to make a request to Flickr's flickr.people.getInfo API method:
flickr = Client()
data = flickr.people.getInfo(user_id='xxx')
In this case flickr.people.getInfo directly maps to the corresponding method in their API documentation. When called, people and getInfo are created as they are looked up, then the proper request to make is determined by the path to getInfo, which is people.getInfo. This is the approach I used:
class Attr(object):
def __init__(self, client, name, parent):
self._client = client
self._name = name
self._parent = parent
def __getattr__(self, name):
attr = Attr(self._client, name, self)
setattr(self, name, attr)
return attr
def _get_path(self, path=None):
if path:
path = '.'.join((self._name, path))
else:
path = self._name
if isinstance(self._parent, Attr):
return self._parent._get_path(path)
return path
def __call__(self, *args, **kwargs):
return self._client.execute_method(self._get_path(), *args, **kwargs)
class Client(object):
def __getattr__(self, name):
attr = Attr(self, name, None)
setattr(self, name, attr)
return attr
def execute_method(self, method, *args, **kwargs):
print method, args, kwargs
This works, but I'm curious if my approach to deal with nested attribute assignment/lookup can be improved, or if there are any errors lurking in wait, unbeknownst to me. In particular, I'm curious if there is a better way to figure out the "path" to a given attribute. For example, if I call Client().x.y.z(), x, y, z do not exist, and will be created one by one (as __getattr__ looks up a single attribute at a time). By the time z is called, I need to be able to discern that the path to z is x.y.z.
Thanks to Thomas K for pointing out that flipy already does this (and seems like a nice library for interacting with flickr). A cleaner approach:
class Method(object):
def __init__(self, client, method_name):
self.client = client
self.method_name = method_name
def __getattr__(self, key):
return Method(self.client, '.'.join((self.method_name, key)))
def __call__(self, **kwargs):
print self.method_name, kwargs
class Client(object):
def __getattr__(self, key):
return Method(self, key)
Et voilà:
>>> c = Client()
>>> c.some.method(x=1, y=2)
some.method {'y': 2, 'x': 1}
Related
I'm trying to make a class like Proxy that I indirectly access all methods of my objects, for example like:
class Radio():
def __init__(self):
self._channel = "channel"
def get_channel(self):
return self._channel
def set_channel(self, value):
self._channel = value
class Proxy:
def __init__(self, obj):
self.obj = obj
# rest of the code
radio = Radio()
radio_proxy = Proxy(radio)
print(radio_proxy.get_channel())
so this works exactly as print(radio.get_channel()) !!! but I'm actually stuck how to do this, I know that it's somehow I should use getattr and stuff but I don't really know how to use them
You're almost there:
class Proxy:
def __init__(self, obj):
self.obj = obj
def __getattr__(self, attr):
return getattr(self.obj, attr)
This doesn't handle "dunder methods", ie. you can't do:
a = Proxy(5)
a + 37
but for regular methods (and attributes) it should be fine.
This question is not in general about the observer pattern. It is focused on the use of decorators in that pattern. The question is based on the answer of a similar question.
#!/usr/bin/env python3
class Observable:
"""
The object that need to be observed. Alternative names are 'Subject'.
In the most cases it is a data object.
"""
def __init__(self):
self._observers = []
def register_observer(self, callback):
self._observers.append(callback)
return callback
def _broadcast_observers(self, *args, **kwargs):
for callback in self._observers:
callback(*args, **kwargs)
class TheData(Observable):
"""
Example of a data class just for demonstration.
"""
def __init__(self, data):
Observable.__init__(self)
self._data = data
#property
def data(self):
return self._data
#data.setter
def data(self, data):
self._data = data
self._broadcast_observers()
class TheGUIElement:
"""
Example of a gui class (Widget) just for demonstration.
e. g. it could be a text field in GUI.
"""
def __init__(self, data):
self._data = data
#data.register_observer(self._data_updated)
self._redraw()
def _redraw(self):
print('in _redraw(): ' + data.data)
#Observable.register_observer
def _data_updated(self, **kwargs):
"""
This is the callback that is called by the Observable if the
data changed.
"""
print('in _data_updated() - kwargs: {}'.format(kwargs))
self._redraw()
if __name__ == '__main__':
data = TheData('DATA')
gui = TheGUIElement(data)
data.data = 'SECOND DATA'
This code doesn't work because of this error.
Traceback (most recent call last):
File "./o.py", line 42, in <module>
class TheGUIElement:
File "./o.py", line 55, in TheGUIElement
#Observable.register_observer
TypeError: register_observer() missing 1 required positional argument: 'callback'
It is unclear to me how to use a decorator for to register the observers (e.g. TheGUIElement).
To register the callback, you need to have an actual object. In your code, how is #Observable.register_observer supposed to find which instance is should register on?
Please drop that Observable thing that's a javaism, cumbersome in python.
Look at this.
#!/usr/bin/env python
class SomeData(object):
def __init__(self, value):
self.callbacks = []
self.foo = value
def register(self, callback):
self.callbacks.append(callback)
return callback
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(self, *args, **kwargs)
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.register(my_gui.redraw)
# Try changing it. Note my_data is dumb for now, notify manually.
my_data.foo = 10
my_data.notify("foo", 10)
I intentionally removed automatic notifications to illustrate registration by itself.
Let's add it back. But there is no point using that Observable class. Let's make it lighter, simply defining an event class.
#!/usr/bin/env python3
class Event(object):
def __init__(self):
self.callbacks = []
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(*args, **kwargs)
def register(self, callback):
self.callbacks.append(callback)
return callback
class SomeData(object):
def __init__(self, foo):
self.changed = Event()
self._foo = foo
#property
def foo(self):
return self._foo
#foo.setter
def foo(self, value):
self._foo = value
self.changed.notify(self, 'foo', value)
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.changed.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.changed.register(my_gui.redraw)
# Try changing it.
my_data.foo = 10
As you probably noted now, the decorator syntax is useful in those circumstances:
You have a single registry. Either a singleton or the class itself class are first-order objects, and most are singletons.
You dynamically define the function and register it as you go.
Now, those manual getters/setters you have are cumbersome as well, if you have many why not factor them out?
#!/usr/bin/env python3
class Event(object):
def __init__(self):
self.callbacks = []
def notify(self, *args, **kwargs):
for callback in self.callbacks:
callback(*args, **kwargs)
def register(self, callback):
self.callbacks.append(callback)
return callback
#classmethod
def watched_property(cls, event_name, key):
actual_key = '_%s' % key
def getter(obj):
return getattr(obj, actual_key)
def setter(obj, value):
event = getattr(obj, event_name)
setattr(obj, actual_key, value)
event.notify(obj, key, value)
return property(fget=getter, fset=setter)
class SomeData(object):
foo = Event.watched_property('changed', 'foo')
def __init__(self, foo):
self.changed = Event()
self.foo = foo
class SomeGUI(object):
def redraw(self, obj, key, newvalue):
print('redrawing %s with value %s' % (self, newvalue))
if __name__ == '__main__':
my_data = SomeData(42)
# Register some function using decorator syntax
#my_data.changed.register
def print_it(obj, key, value):
print('Key %s changed to %s' % (key, value))
# Register the SomeGUI element
my_gui = SomeGUI()
my_data.changed.register(my_gui.redraw)
# Try changing it.
my_data.foo = 10
For reference, all three programs output the exact same thing:
$ python3 test.py
Key foo changed to 10
redrawing <__main__.SomeGUI object at 0x7f9a90d55fd0> with value 10
Even though the thread is kinda old (probably the problem is already solved), I would like to share a solution of mine to the "Decorated Observer Pattern" problem:
https://pypi.org/project/notifyr/
I created a package that implements decorators which add the observer-observed methods/attributes to python classes. I managed to use the package in a Django project too, but with a few adaptations (the .observers attribute is not persisted in the database, so I had to load the list of observers into it every time I expected to notify them).
Here is an implementation example:
Original Code:
class Dog(object):
def __init__(self, name):
self.name = name
def bark(self):
print('Woof')
def sleep(self):
print(self.name, 'is now asleep: ZZzzzzZzzZ...')
class Person(object):
def __init__(self, name):
self.name = name
def educate_dog(self, dog):
print(self.name + ':','Sleep,', dog.name)
dog.sleep()
Suppose we want a person to educate a dog every time the animal barks:
from notifyr.agents import observed, observer
from notifyr.functions import target
#observed
class Dog(object):
def __init__(self, name):
self.name = name
#target
def bark(self):
print('Woof')
def sleep(self):
print(self.name, 'is now asleep: ZZzzzzZzzZ...')
#observer('educate_dog')
class Person(object):
def __init__(self, name):
self.name = name
def educate_dog(self, dog):
print(self.name + ':','Sleep,', dog.name)
dog.sleep()
Given the decorated classes, it is possible to achieve the following result:
d = Dog('Tobby')
p = Person('Victor')
d.attach(p) # Victor is now observing Tobby
d.bark()
# Woof
# Victor: Sleep, Tobby
# Tobby is now asleep: ZZzzzzZzzZ...
The package is still very primitive, but it presents a working solution to this type of situation.
I was recently looking for something similar and here's what I came up with. It works by intercepting the __setattr__ method -- a useful stunt I plan on keeping in my pocket for later.
def watchableClass(cls):
"""
Class Decorator!
* If the class has a "dirty" member variable, then it will be
automatically set whenever any class value changes
* If the class has an "onChanged()" method, it will be called
automatically whenever any class value changes
* All this only takes place if the value is different from what it was
that is, if myObject.x is already 10 and you set myObject.x=10
nothing happens
* DOES NOT work with getter/setter functions. But then, you are
already in a function, so do what you want!
EXAMPLE:
#watchableClass
class MyClass:
def __init__(self):
self.dirty=False
def onChanged(self):
print('class has changed')
"""
if hasattr(cls,'__setattr__'):
cls.__setattr_unwatched__=cls.__setattr__
cls.__setattr__=_setObjValueWatchedCascade
else:
cls.__setattr__=_setObjValueWatched
return cls
def _setObjValueWatched(ob,k,v):
"""
called when an object value is set
"""
different=not k in ob.__dict__ or ob.__dict__[k]!=v
if different:
ob.__dict__[k]=v
if k not in ('dirty'):
if hasattr(ob,'dirty'):
ob.dirty=True
if hasattr(ob,'onChanged'):
ob.onChanged()
def _setObjValueWatchedCascade(ob,k,v):
"""
called when an object value is set
IF the class had its own __setattr__ member defined!
"""
different=not k in ob.__dict__ or ob.__dict__[k]!=v
ob.__setattr_unwatched__(k,v)
if different:
if k not in ('dirty'):
if hasattr(ob,'dirty'):
ob.dirty=True
if hasattr(ob,'onChanged'):
ob.onChanged()
I want to define a class containing read and write methods, which can be called as follows:
instance.read
instance.write
instance.device.read
instance.device.write
To not use interlaced classes, my idea was to overwrite the __getattr__ and __setattr__ methods and to check, if the given name is device to redirect the return to self. But I encountered a problem giving infinite recursions. The example code is as follows:
class MyTest(object):
def __init__(self, x):
self.x = x
def __setattr__(self, name, value):
if name=="device":
print "device test"
else:
setattr(self, name, value)
test = MyTest(1)
As in __init__ the code tried to create a new attribute x, it calls __setattr__, which again calls __setattr__ and so on. How do I need to change this code, that, in this case, a new attribute x of self is created, holding the value 1?
Or is there any better way to handle calls like instance.device.read to be 'mapped' to instance.read?
As there are always questions about the why: I need to create abstractions of xmlrpc calls, for which very easy methods like myxmlrpc.instance,device.read and similar can be created. I need to 'mock' this up to mimic such multi-dot-method calls.
You must call the parent class __setattr__ method:
class MyTest(object):
def __init__(self, x):
self.x = x
def __setattr__(self, name, value):
if name=="device":
print "device test"
else:
super(MyTest, self).__setattr__(name, value)
# in python3+ you can omit the arguments to super:
#super().__setattr__(name, value)
Regarding the best-practice, since you plan to use this via xml-rpc I think this is probably better done inside the _dispatch method.
A quick and dirty way is to simply do:
class My(object):
def __init__(self):
self.device = self
Or you can modify self.__dict__ from inside __setattr__():
class SomeClass(object):
def __setattr__(self, name, value):
print(name, value)
self.__dict__[name] = value
def __init__(self, attr1, attr2):
self.attr1 = attr1
self.attr2 = attr2
sc = SomeClass(attr1=1, attr2=2)
sc.attr1 = 3
You can also use object.
class TestClass:
def __init__(self):
self.data = 'data'
def __setattr__(self, name, value):
print("Attempt to edit the attribute %s" %(name))
object.__setattr__(self, name, value)
or you can just use #property:
class MyTest(object):
def __init__(self, x):
self.x = x
#property
def device(self):
return self
If you don't want to specify which attributes can or cannot be set, you can split the class to delay the get/set hooks until after initialization:
class MyTest(object):
def __init__(self, x):
self.x = x
self.__class__ = _MyTestWithHooks
class _MyTestWithHooks(MyTest):
def __setattr__(self, name, value):
...
def __getattr__(self, name):
...
if __name__ == '__main__':
a = MyTest(12)
...
As noted in the code you'll want to instantiate MyTest, since instantiating _MyTestWithHooks will result in the same infinite recursion problem as before.
I want to keep track of object instances of a given class/subclasses in Python given my particular framework I cannot use constructors. As an alternative I could use either decorators or Singletons objects.
Are there any other alternatives except using global variables ?
You could use a descriptor. That is, it's an object that can and will keep track of what things are where with the caveat that it will only do so if you've assigned a value to do it. Here's an example:
class RevealAccess(object):
"""
A data descriptor that sets and returns values
normally and prints a message logging their access.
"""
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print 'Retrieving', self.name
return self.val
def __set__(self, obj, val):
print 'Updating', self.name
self.val = val
This is taken out from the documentation. However, add the following from this SO Q&A:
_instances = []
def __new__(cls, *args, **kw):
instance = object.__new__(cls)
cls._instances.append(instance)
return instance
#classmethod
def get_instances(cls):
return self._instances
and you've achieved your goals:
class Foo(object):
desc = MyDesc('yo')
foo = Foo()
Foo.get_instances().length
I have a class that knows its existing instances. Sometimes I want the class constructor to return an existing object instead of creating a new one.
class X:
def __new__(cls, arg):
i = f(arg)
if i:
return X._registry[i]
else:
return object.__new__(cls)
# more stuff here (such as __init_, _registry, etc.)
Of course, if the first branch is executed, I don't need __init__, but it's invoked anyways. What's a good way to tell __init__ to do nothing?
I can probably just add some attribute to keep track of whether __init__ has run yet, but perhaps there's a better way?
In languages that support private constructors (C#, Dart, Scala, etc), factory methods provide a robust solution to this problem.
In Python, however, class constructors are always accessible, and so a user of your class may easily forget the factory method and call the constructor directly, producing duplicate copies of objects that should be unique.
A fool-proof solution to this problem can be achieved using a metaclass. The example below assumes that the zeroth constructor argument can be used to uniquely identify each instance:
class Unique(type):
def __call__(cls, *args, **kwargs):
if args[0] not in cls._cache:
self = cls.__new__(cls, *args, **kwargs)
cls.__init__(self, *args, **kwargs)
cls._cache[args[0]] = self
return cls._cache[args[0]]
def __init__(cls, name, bases, attributes):
super().__init__(name, bases, attributes)
cls._cache = {}
It can be used as follows:
class Country(metaclass=Unique):
def __init__(self, name: str, population: float, nationalDish: str):
self.name = name
self.population = population
self.nationalDish = nationalDish
placeA = Country("Netherlands", 16.8e6, "Stamppot")
placeB = Country("Yemen", 24.41e6, "Saltah")
placeC = Country("Netherlands", 11, "Children's tears")
print(placeA is placeB) # -> False
print(placeA is placeC) # -> True
print(placeC.nationalDish) # -> Stamppot
In summary, this approach is useful if you want to produce a set of unique objects at runtime (possibly using data in which entries may be repeated).
Use a factory, i.e.
_x_singleton = None
def XFactory():
global _x_singleton
if _x_singleton is None:
_x_singleton = X()
return _x_singleton
or use a "create" classmethod in your class that behaves the way you want it to,
class X(object):
instance = None
def __init__(self):
# ...
#classmethod
def create(cls):
if cls.instance is None:
cls.instance = cls()
return cls.instance
You might even consider making __init__ raise an exception if some condition isn't met (i.e. self.instance not being None)