I know Python will simply return the key list when I put a dict in for...in... syntax.
But what what happens to the dict?
When we use help(dict), we can not see __next()__ method in the method list. So if I want to make a derived class based on dict:
class MyDict(dict)
def __init__(self, *args, **kwargs):
super(MyDict, self).__init__(*args, **kwargs)
and return the value list with for...in...
d = Mydict({'a': 1, 'b': 2})
for value in d:
what should I do?
Naively, if all you want is for iteration over an instance of MyClass to yield the values instead of the keys, then in MyClass define:
def __iter__(self):
return self.itervalues()
In Python 3:
def __iter__(self):
return iter(self.values())
But beware! By doing this your class no longer implements the contract of collections.MutableMapping, even though issubclass(MyClass, collections.MutableMapping) is True. You might be better off not subclassing dict, if this is the behaviour you want, but instead have an attribute of type dict to hold the data, and implement only the functions and operators you need.
Related
I'm trying to clean up some logic and remove duplicate values in some code and am looking for a way to introduce some very simple lazy-loading to handle settings variables. Something that would work like this:
FOO = {'foo': 1}
BAR = {'test': FOO['foo'] }
# ...complex logic here which ultimately updates the value of Foo['foo']...
FOO['foo'] = 2
print(BAR['test']) # Outputs 1 but would like to get 2
Update:
My question may not have been clear based on the initial responses. I'm looking to replace the value being set for test in BAR with a lazy-loaded substitute. I know a way I can do this but it seems unnecessarily complex for what it is, I'm wondering if there's a simpler approach.
Update #2:
Okay, here's a solution that works. Is there any built-in type that can do this out of the box:
FOO = {'foo': 1}
import types
class LazyDict(dict):
def __getitem__(self, item):
value = super().__getitem__(item)
return value if not isinstance(value, types.LambdaType) else value()
BAR = LazyDict({ 'test': lambda: FOO['foo'] })
# ...complex logic here which ultimately updates the value of Foo['foo']...
FOO['foo'] = 2
print(BAR['test']) # Outputs 2
As I stated in the comment above, what you are seeking is some of the facilities of reactive programming paradigm. (not to be confounded with the JavaScript library which borrows its name from there).
It is possible to instrument objects in Python to do so - I think the minimum setup here would be a specialized target mapping, and a special object type you set as the values in it, that would fetch the target value.
Python can do this in more straightforward ways with direct attribute access (using the dot notation: myinstance.value) than by using the key-retrieving notation used in dictionaries mydata['value'] due to the fact a class is already a template to a certain data group, and class attributes can define mechanisms to access each instance's attribute value. That is called the "descriptor protocol" and is bound into the language model itself.
Nonetheless a minimalist Mapping based version can be implemented as such:
FOO = {'foo': 1}
from collections.abc import MutableMapping
class LazyValue:
def __init__(self, source, key):
self.source = source
self.key = key
def get(self):
return self.source[self.key]
def __repr__(self):
return f"<LazyValue {self.get()!r}>"
class LazyDict(MutableMapping):
def __init__(self, *args, **kw):
self.data = dict(*args, **kw)
def __getitem__(self, key):
value = self.data[key]
if isinstance(value, LazyValue):
value = value.get()
return value
def __setitem__(self, key, value):
self.data[key] = value
def __delitem__(key):
del self.data[key]
def __iter__(self):
return iter(self.data)
def __len__():
return len(self.data)
def __repr__():
return repr({key: value} for key, value in self.items())
BAR = LazyDict({'test': LazyValue(FOO, 'foo')})
# ...complex logic here which ultimately updates the value of Foo['foo']...
FOO['foo'] = 2
print(BAR['test']) # Outputs 2
The reason this much code is needed is that there are several ways to retrieve data from a dictionary or mapping (.values, .items, .get, .setdefault) and simply inheriting from dict and implementing __getitem__ would "leak" the special lazy object in any of the other methods. Going through this MutableMapping approach ensure a single point of reading of the value in the __getitem__ method - and the resulting instance can be used reliably anywhere a mapping is expected.
However, notice that if you are using normal classes and instances rather than dictionaries, this can be much simpler - you can just use plain Python "property" and have a getter that will fetch the value. The main factor you should ponder is whether your referenced data keys are fixed, and can be hard-coded when writting the source code, or if they are dynamic, and which keys will work as lazy-references are only known at runtime. In this last case, the custom mapping approach, as above, will be usually better:
FOO = {'foo': 1}
class LazyStuff:
def __init__(self, source):
self.source = source
#property
def test(self):
return self.source["foo"]
BAR = LazyStuff(FOO)
FOO["foo"] = 2
print(BAR.test)
Perceive that in this way you have to hardcode the key "foo" and "test" in the class body, but it is just plaincode, and no need for the intermediary "LazyValue" class. Also, if you need this data as a dictionary, you could add an .as_dict method to LazyStuff that would collect all attributes in the moment it were called and yield a snapshot of those values as a dictionary..
You can try using lambdas and calling the value on return. Like this:
FOO = {'foo': 1}
BAR = {'test': lambda: FOO['foo'] }
FOO['foo'] = 2
print(BAR['test']()) # Outputs 2
If you're only one level deep, you may wish to try ChainMap, E.g.,
>>> from collections import ChainMap
>>> defaults = {'foo': 42}
>>> myvalues = {}
>>> result = ChainMap(myvalues, defaults)
>>> result['foo']
42
>>> defaults['foo'] = 99
>>> result['foo']
99
I'm following this link and trying to make a singleton class. But, taking arguments (passed while initiating a class) into account so that the same object is returned if the arguments are same.
So, instead of storing class name/class reference as a dict key, I want to store passed arguments as keys in dict. But, there could be unhashable arguments also (like dict, set itself).
What is the best way to store class arguments and class objects mapping? So that I can return an object corresponding to the arguments.
Thanks anyways.
EDIT-1 :
A little more explanation. Let's say there is class as follows
class A:
__metaclass__ == Singleton
def __init__(arg1, arg2):
pass
Now, A(1,2) should always return the same object. But, it should be different from A(3,4)
I think, the arguments very much define the functioning of a class. Let's say if the class is to make redis connections. I might want to create 2 singletons objects with diff redis hosts as parameters, but the underlying class/code could be common.
As theheadofabroom and me already mentioned in the comments, there are some odds when relying on non-hashable values for instance caching or memoization. Therefore, if you still want to do exactly that, the following example does not hide the memoization in the __new__ or __init__ method. (A self-memoizing class would be hazardous because the memoization criterion can be fooled by code that you don't control).
Instead, I provide the function memoize which returns a memoizing factory function for a class. Since there is no generic way to tell from non-hashable arguments, if they will result in an instance that is equivalent to an already existing isntance, the memoization semantics have to be provided explicitly. This is achieved by passing the keyfunc function to memoize. keyfunc takes the same arguments as the class' __init__ method and returns a hashable key, whose equality relation (__eq__) determines memoization.
The proper use of the memoization is in the responsibility of the using code (providing a sensible keyfunc and using the factory), since the class to be memoized is not modified and can still be instantiated normally.
def memoize(cls, keyfunc):
memoized_instances = {}
def factory(*args, **kwargs):
key = keyfunc(*args, **kwargs)
if key in memoized_instances:
return memoized_instances[key]
instance = cls(*args, **kwargs)
memoized_instances[key] = instance
return instance
return factory
class MemoTest1(object):
def __init__(self, value):
self.value = value
factory1 = memoize(MemoTest1, lambda value : value)
class MemoTest2(MemoTest1):
def __init__(self, value, foo):
MemoTest1.__init__(self, value)
self.foo = foo
factory2 = memoize(MemoTest2, lambda value, foo : (value, frozenset(foo)))
m11 = factory1('test')
m12 = factory1('test')
assert m11 is m12
m21 = factory2('test', [1, 2])
lst = [1, 2]
m22 = factory2('test', lst)
lst.append(3)
m23 = factory2('test', lst)
assert m21 is m22
assert m21 is not m23
I only included MemoTest2 as a sublclass of MemoTest1 to show that there is no magic involved in using regular class inheritance.
Today I have learnt a Bunch class from the book < Python algorithm >
class Bunch(dict):
def __init__(self, *args, **kwds):
super(Bunch, self).__init__(*args, **kwds)
self.__dict__ = self
But I'm not quite understanding what does the third line do:
super(Bunch, self).__init__(*args, **kwds)
I know it will call the parent class, and do something like:
dict.__init__(self, *args, **kwds)
But I have no idea what does the __init__ function do to the dict.
How should I understand that?
You can think of dict.__init__ as the dictionary constructor. e.g. I can write:
d = dict(a=1, b=2)
or
d = dict([('a', 1), ('b', 2)])
Both create (initialize) the dictionary {'a': 1, 'b': 2}. Whenever I write those two statements, it calls dict.__init__ which is what does the job of adding the a and b items.
In the same way, with Bunch, it causes all of the keyword arguments (or an iterable of 2-sequences) to be turned into key-value pairs in the dictionary subclass (Bunch).
You don't need to know exactly what dict__init__ does. The way to think about it is that dict.__init__ does what is necessary to make the object "work like a dict". In general, when you subclass, you need to call the superclass __init__ to make the class "work like" its superclass, and the same is true when subclassing a builtin type like dict.
How can every set method in python's list object be overridden in a derived object such that every item in that list is of a specific class?
Consider
class Index(int):
pass
class IndexList(list):
def __init__(self, int_list):
for el in int_list:
self.append(Index(el))
def __setitem__(self, key, val):
super(IndexList, self).__setitem__(key, Index(val))
# override append insert etc...
Can this be done without directly overriding every single function that adds elements to the list? I expected simply overriding __setitem__ was enough.
Example, if append is not overridden.
ilist = IndexList([1,2])
ilist.append(3)
for i in ilist:
print(isinstance(i, Index)) # True, True, False
You'll have to implement the various directly; the underlying C implementation does not call __setitem__ for each and every change, as it is far more efficient to directly manipulate the (dynamically grown) C array.
Take a look at the collections abstract base classes, specifically at the MutableSequence ABC, to get an idea of what methods all can mutate your list, to maintain your type invariant you'd need to implement insert, append, extend and __iadd__.
Better still, you can use the collections.MutableSequence() class as an alternative base class to list; this is a pure-python implementation that does cast many of those methods as calls to a core set of methods; you'd only need to provide implementations for __len__, __getitem__, __setitem__, __delitem__ and insert; any method named in the Abstract Methods column of the table.
class IndexList(collections.MutableSequence):
def __init__(self, int_list):
self._list = []
for el in int_list:
self.append(Index(el))
def __len__(self): return len(self._list)
def __getitem__(self, item): return self._list[item]
def __delitem__(self, item): del self._list[item]
def __setitem__(self, index, value):
self._list.key[index] = Index(value)
def insert(self, index, value):
self._list.insert(index, Index(value))
I am dealing with classes whose attributes are sometimes list whose elements can be dictionaries or further nested objects with attributes etc. I would like to perform some slicing that with my grasp of python is only doable with what feels profoundly un-Pythonic.
My minimal code looks like this:
class X(object):
def __init__(self):
self.a = []
x=X()
x.a.append({'key1':'v1'})
x.a.append({'key1':'v2'})
x.a.append({'key1':'v3'})
# this works as desired
x.a[0]['key1'] # 'v1'
I would like to perform an access to a key in the nested dictionary but make that call for all elements of the list containing that dictionary. The standard python way of doing this would be a list comprehension a la:
[v['key1'] for v in x.a]
However, my minimal example doesn't quite convey the full extent of nesting in my real-world scenario: The attribute list a in class X might contain objects, whose attributes are objects, whose attributes are dictionaries whose keys I want to select on while iterating over the outer list.
# I would like something like
useful_list = x.a[:]['key1'] # TypeError: list indices must be integers, not str
# or even better
cool_list = where(x.a[:]['key1'] == 'v2') # same TypeError
If I start list comprehending for every interesting key it quickly doesn't look all that Pythonic. Is there a nice way of doing this or do I have to code 'getter' methods for all conceivable pairings of lists and dictionary keys?
UPDATE:
I have been reading about overloading lists. Apparently one can mess with the getitem method which is used for indeces for lists and keys for dict. Maybe a custom class that iterates over list members. This is starting to sound contrived...
So, you want to create an hierarchical structure, with an operation which means
different things for different types, and is defined recursively.
Polymorphism to the rescue.
You could override __getitem__ instead of my get_items below, but in your case it might be better to define a non-builtin operation to avoid risking ambiguity. It's up to you really.
class ItemsInterface(object):
def get_items(self, key):
raise NotImplementedError
class DictItems(ItemsInterface, dict):
def __init__(self, *args, **kwargs):
dict.__init__(self, *args, **kwargs)
def get_items(self, key):
res = self[key]
# apply recursively
try:
res = res.get_items(key)
except AttributeError:
pass
return res
class ListItems(ItemsInterface, list):
def __init__(self, *args, **kwargs):
list.__init__(self, *args, **kwargs)
def get_items(self, key):
return [ x.get_items(key) for x in self ]
x = ListItems()
x.append(DictItems({'key1':'v1'}))
x.append(DictItems({'key1':'v2'}))
x.append(DictItems({'key1':'v3'}))
y = DictItems({'key1':'v999'})
x.append(ListItems([ y ]))
x.get_items('key1')
=> ['v1', 'v2', 'v3', ['v999']]
Of course, this solution might not be exactly what you need (you didn't explain what it should do if the key is missing, etc.)
but you can easily modify it to suit your needs.
This solution also supports ListItems as values of the DictItems. the get_items operation is applied recursively.