Python dictionary from an object's fields

Python dictionary from an object's fields - python

Do you know if there is a built-in function to build a dictionary from an arbitrary object? I'd like to do something like this:
>>> class Foo:
... bar = 'hello'
... baz = 'world'
...
>>> f = Foo()
>>> props(f)
{ 'bar' : 'hello', 'baz' : 'world' }
NOTE: It should not include methods. Only fields.

Note that best practice in Python 2.7 is to use new-style classes (not needed with Python 3), i.e.
class Foo(object):
...
Also, there's a difference between an 'object' and a 'class'. To build a dictionary from an arbitrary object, it's sufficient to use __dict__. Usually, you'll declare your methods at class level and your attributes at instance level, so __dict__ should be fine. For example:
>>> class A(object):
... def __init__(self):
... self.b = 1
... self.c = 2
... def do_nothing(self):
... pass
...
>>> a = A()
>>> a.__dict__
{'c': 2, 'b': 1}
A better approach (suggested by robert in comments) is the builtin vars function:
>>> vars(a)
{'c': 2, 'b': 1}
Alternatively, depending on what you want to do, it might be nice to inherit from dict. Then your class is already a dictionary, and if you want you can override getattr and/or setattr to call through and set the dict. For example:
class Foo(dict):
def __init__(self):
pass
def __getattr__(self, attr):
return self[attr]
# etc...

Instead of x.__dict__, it's actually more pythonic to use vars(x).

The dir builtin will give you all the object's attributes, including special methods like __str__, __dict__ and a whole bunch of others which you probably don't want. But you can do something like:
>>> class Foo(object):
... bar = 'hello'
... baz = 'world'
...
>>> f = Foo()
>>> [name for name in dir(f) if not name.startswith('__')]
[ 'bar', 'baz' ]
>>> dict((name, getattr(f, name)) for name in dir(f) if not name.startswith('__'))
{ 'bar': 'hello', 'baz': 'world' }
So can extend this to only return data attributes and not methods, by defining your props function like this:
import inspect
def props(obj):
pr = {}
for name in dir(obj):
value = getattr(obj, name)
if not name.startswith('__') and not inspect.ismethod(value):
pr[name] = value
return pr

I've settled with a combination of both answers:
dict((key, value) for key, value in f.__dict__.iteritems()
if not callable(value) and not key.startswith('__'))

I thought I'd take some time to show you how you can translate an object to dict via dict(obj).
class A(object):
d = '4'
e = '5'
f = '6'
def __init__(self):
self.a = '1'
self.b = '2'
self.c = '3'
def __iter__(self):
# first start by grabbing the Class items
iters = dict((x,y) for x,y in A.__dict__.items() if x[:2] != '__')
# then update the class items with the instance items
iters.update(self.__dict__)
# now 'yield' through the items
for x,y in iters.items():
yield x,y
a = A()
print(dict(a))
# prints "{'a': '1', 'c': '3', 'b': '2', 'e': '5', 'd': '4', 'f': '6'}"
The key section of this code is the __iter__ function.
As the comments explain, the first thing we do is grab the Class items and prevent anything that starts with '__'.
Once you've created that dict, then you can use the update dict function and pass in the instance __dict__.
These will give you a complete class+instance dictionary of members. Now all that's left is to iterate over them and yield the returns.
Also, if you plan on using this a lot, you can create an #iterable class decorator.
def iterable(cls):
def iterfn(self):
iters = dict((x,y) for x,y in cls.__dict__.items() if x[:2] != '__')
iters.update(self.__dict__)
for x,y in iters.items():
yield x,y
cls.__iter__ = iterfn
return cls
#iterable
class B(object):
d = 'd'
e = 'e'
f = 'f'
def __init__(self):
self.a = 'a'
self.b = 'b'
self.c = 'c'
b = B()
print(dict(b))

A downside of using __dict__ is that it is shallow; it won't convert any subclasses to dictionaries.
If you're using Python3.5 or higher, you can use jsons:
>>> import jsons
>>> jsons.dump(f)
{'bar': 'hello', 'baz': 'world'}

To build a dictionary from an arbitrary object, it's sufficient to use __dict__.
This misses attributes that the object inherits from its class. For example,
class c(object):
x = 3
a = c()
hasattr(a, 'x') is true, but 'x' does not appear in a.__dict__

Python3.x
return dict((key, value) for key, value in f.__dict__.items() if not callable(value) and not key.startswith('__'))

Late answer but provided for completeness and the benefit of googlers:
def props(x):
return dict((key, getattr(x, key)) for key in dir(x) if key not in dir(x.__class__))
This will not show methods defined in the class, but it will still show fields including those assigned to lambdas or those which start with a double underscore.

vars() is great, but doesn't work for nested objects of objects
Convert nested object of objects to dict:
def to_dict(self):
return json.loads(json.dumps(self, default=lambda o: o.__dict__))

I think the easiest way is to create a getitem attribute for the class. If you need to write to the object, you can create a custom setattr . Here is an example for getitem:
class A(object):
def __init__(self):
self.b = 1
self.c = 2
def __getitem__(self, item):
return self.__dict__[item]
# Usage:
a = A()
a.__getitem__('b') # Outputs 1
a.__dict__ # Outputs {'c': 2, 'b': 1}
vars(a) # Outputs {'c': 2, 'b': 1}
dict generates the objects attributes into a dictionary and the dictionary object can be used to get the item you need.

In 2021, and for nested objects/dicts/json use pydantic BaseModel - will convert nested dicts and nested json objects to python objects and JSON and vice versa:
https://pydantic-docs.helpmanual.io/usage/models/
>>> class Foo(BaseModel):
... count: int
... size: float = None
...
>>>
>>> class Bar(BaseModel):
... apple = 'x'
... banana = 'y'
...
>>>
>>> class Spam(BaseModel):
... foo: Foo
... bars: List[Bar]
...
>>>
>>> m = Spam(foo={'count': 4}, bars=[{'apple': 'x1'}, {'apple': 'x2'}])
Object to dict
>>> print(m.dict())
{'foo': {'count': 4, 'size': None}, 'bars': [{'apple': 'x1', 'banana': 'y'}, {'apple': 'x2', 'banana': 'y'}]}
Object to JSON
>>> print(m.json())
{"foo": {"count": 4, "size": null}, "bars": [{"apple": "x1", "banana": "y"}, {"apple": "x2", "banana": "y"}]}
Dict to object
>>> spam = Spam.parse_obj({'foo': {'count': 4, 'size': None}, 'bars': [{'apple': 'x1', 'banana': 'y'}, {'apple': 'x2', 'banana': 'y2'}]})
>>> spam
Spam(foo=Foo(count=4, size=None), bars=[Bar(apple='x1', banana='y'), Bar(apple='x2', banana='y2')])
JSON to object
>>> spam = Spam.parse_raw('{"foo": {"count": 4, "size": null}, "bars": [{"apple": "x1", "banana": "y"}, {"apple": "x2", "banana": "y"}]}')
>>> spam
Spam(foo=Foo(count=4, size=None), bars=[Bar(apple='x1', banana='y'), Bar(apple='x2', banana='y')])

Dataclass(from Python 3.7) is another option which can be used for converting class properties to dict. asdict can be used along with dataclass objects
for the conversion.
Example:
#dataclass
class Point:
x: int
y: int
p = Point(10, 20)
asdict(p) # it returns {'x': 10, 'y': 20}

As mentioned in one of the comments above, vars currently isn't universal in that it doesn't work for objects with __slots__ instead of a normal __dict__. Moreover, some objecs (e.g., builtins like str or int) have neither a __dict__ nor __slots__.
For now, a more versatile solution could be this:
def instance_attributes(obj: Any) -> Dict[str, Any]:
"""Get a name-to-value dictionary of instance attributes of an arbitrary object."""
try:
return vars(obj)
except TypeError:
pass
# object doesn't have __dict__, try with __slots__
try:
slots = obj.__slots__
except AttributeError:
# doesn't have __dict__ nor __slots__, probably a builtin like str or int
return {}
# collect all slots attributes (some might not be present)
attrs = {}
for name in slots:
try:
attrs[name] = getattr(obj, name)
except AttributeError:
continue
return attrs
Example:
class Foo:
class_var = "spam"
class Bar:
class_var = "eggs"
__slots__ = ["a", "b"]
>>> foo = Foo()
>>> foo.a = 1
>>> foo.b = 2
>>> instance_attributes(foo)
{'a': 1, 'b': 2}
>>> bar = Bar()
>>> bar.a = 3
>>> instance_attributes(bar)
{'a': 3}
>>> instance_attributes("baz")
{}
Rant:
It's a pity that this isn't built into vars already. Many builtins in Python promise to be "the" solution to a problem but then there's always several special cases that aren't handled... And one just ends up having to write the code manually in any case.

If you want to list part of your attributes, override __dict__:
def __dict__(self):
d = {
'attr_1' : self.attr_1,
...
}
return d
# Call __dict__
d = instance.__dict__()
This helps a lot if your instance get some large block data and you want to push d to Redis like message queue.

PYTHON 3:
class DateTimeDecoder(json.JSONDecoder):
def __init__(self, *args, **kargs):
JSONDecoder.__init__(self, object_hook=self.dict_to_object,
*args, **kargs)
def dict_to_object(self, d):
if '__type__' not in d:
return d
type = d.pop('__type__')
try:
dateobj = datetime(**d)
return dateobj
except:
d['__type__'] = type
return d
def json_default_format(value):
try:
if isinstance(value, datetime):
return {
'__type__': 'datetime',
'year': value.year,
'month': value.month,
'day': value.day,
'hour': value.hour,
'minute': value.minute,
'second': value.second,
'microsecond': value.microsecond,
}
if isinstance(value, decimal.Decimal):
return float(value)
if isinstance(value, Enum):
return value.name
else:
return vars(value)
except Exception as e:
raise ValueError
Now you can use above code inside your own class :
class Foo():
def toJSON(self):
return json.loads(
json.dumps(self, sort_keys=True, indent=4, separators=(',', ': '), default=json_default_format), cls=DateTimeDecoder)
Foo().toJSON()

Try:
from pprint import pformat
a_dict = eval(pformat(an_obj))

Python 3.7+ in 2023
You can add the dataclass decorator to your class and define a custom JSON serializer, then json.dumps will work (and you can extend it to work with non-serializable attributes by providing a custom encoder to cls).
f=Foo()
json.dumps(f, cls=CustomJSONEncoder)
{"bar": "hello", "baz": "world", "modified": "2023-02-08T11:49:15.675837"}
A custom JSON serializer can be easily modified to make it compatible with any type that isn't natively JSON serializable.
from datetime import datetime
import dataclasses
import json
#dataclasses.dataclass # <<-- add this decorator
class Foo():
"""An example dataclass."""
bar: str = "hello"
baz: str = "world"
modified: datetime = Column(DateTime(timezone=True), default=datetime.utcnow)
class CustomJSONEncoder(json.JSONEncoder): # <<-- Add this custom encoder
"""Custom JSON encoder for the DB class."""
def default(self, o):
if dataclasses.is_dataclass(o): # this serializes anything dataclass can handle
return dataclasses.asdict(o)
if isinstance(o, datetime): # this adds support for datetime
return o.isoformat()
return super().default(o)
To further extend it for any non-serializable type, add another if statement to the custom encoder class that returns something serializable (e.g. str).

Related

How do I update the dictionary of a mapped trait, after I've already constructed it?

I need to update the dictionary of a mapped trait some time after initial trait creation.
How do I do this?
The following code:
from traits.api import (HasTraits, Trait)
class bar(HasTraits):
zap = Trait("None", {"None": None})
def __init__(self):
# In reality, determined programmatically at runtime.
add_dict_entries = {"One": 1}
new_dict = {"None": None}
new_dict.update(add_dict_entries)
self.zap = Trait("None", new_dict)
theBar = bar()
yields:
Traceback (most recent call last):
File "tst_mapped_trait.py", line 13, in <module>
theBar = bar()
File "tst_mapped_trait.py", line 11, in __init__
self.zap = Trait("None", new_dict)
File "C:\Users\dbanas\Anaconda3\envs\pybert-dev\lib\site-packages\traits\trait_handlers.py", line 236, in error
object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: The 'zap' trait of a bar instance must be 'None', but a value of <traits.traits.CTrait object at 0x00000000034AA9E8> <class 'traits.traits.CTrait'> was specified.```

Okay, the following code worked:
from traits.api import (HasTraits, Trait)
class bar(HasTraits):
zap = Trait("None", {"None": None})
def __init__(self):
# In reality, determined programmatically at runtime.
add_dict_entries = {"One": 1}
new_dict = {"None": None}
new_dict.update(add_dict_entries)
# self.zap = Trait("None", new_dict)
# self.zap.update(new_dict)
# self.trait_setq(zap=Trait("None", new_dict))
self.remove_trait("zap")
self.add_trait("zap", Trait("None", new_dict))
theBar = bar()
Note: The commented out lines are things I tried, which did not work.

I'm not sure I understand what you're after, but I can make a few recommendations:
Either is a good choice here if you allow both None and Dict.
Use dynamic initialization to create a value for a trait at runtime. It's preferred to using an __init__ method.
If you really need an __init__ method, you must call super inside of it for Traits to work properly, e.g. `super()init(*args, **kwargs)
Here's a version of your code that works and I think solves your problem.
from traits.api import (HasTraits, Either, Dict)
class bar(HasTraits):
zap = Either(None, Dict)
def _zap_default(self):
add_dict_entries = {"One": 1}
new_dict = {"None": None}
new_dict.update(add_dict_entries)
return new_dict
theBar = bar()
print(theBar.zap)
And here's some feedback on the code that didn't work. The line self.zap = Trait("None", new_dict) below doesn't work because it tries to create a Trait object but self.zap only accepts None or Dict. My recommendation is to use trait definitions only for typing, at the class-level. Within methods, use regular Python types.
from traits.api import (HasTraits, Trait)
class bar(HasTraits):
zap = Trait("None", {"None": None})
def __init__(self):
# In reality, determined programmatically at runtime.
add_dict_entries = {"One": 1}
new_dict = {"None": None}
new_dict.update(add_dict_entries)
self.zap = Trait("None", new_dict)
theBar = bar()

Here's second attempt at an answer given the original poster's comment
If you want the type of zap to be Dict and only Dict, then define it as such. You can also inline the initial value if it doesn't have to be computer at runtime:
>>> from traits.api import HasTraits, Dict
>>> class Bar(HasTraits):
... zap = Dict({5: 'e'})
...
>>> bar = Bar()
>>> bar.zap
{5: 'e'}
If it needs to be computed at runtime, then use dynamic initialization to initialize the value:
>>> class Bar(HasTraits):
... zap = Dict()
...
... def _zap_default(self):
... default = {}
... default[1] = 'a'
... return default
...
>>> bar_dynamic = Bar()
>>> bar_dynamic.zap
{1: 'a'}
Either way, the zap attribute on the Bar instance is a regular dictionary once the class has be instantiated (after bar = Bar()). You should use Trait types after instantiation, only regular Python objects. Traits is there to define and enforce types. The type() of the objects you're assigning to the typed traits (like zap here) are regular Python types.
Here's how you'd modify zap from outside of the class:
>>> bar.zap[2] = 'b'
>>> bar.zap
{5: 'e', 2: 'b'}
>>>
>>> bar_dynamic.zap[3] = 'c'
>>> bar_dynamic.zap
{1: 'a', 3: 'c'}
And now from inside the class, as a regular attribute on self:
>>> class Bar(HasTraits):
... zap = Dict()
...
... def _zap_default(self):
... default = {}
... default[1] = 'a'
... return default
...
... def add_pair(self, key, value):
... self.zap[key] = value
...
>>> bar_method = Bar()
>>> bar_method.zap
{1: 'a'}
>>> bar_method.add_pair(26, 'z')
>>> bar_method.zap
{1: 'a', 26: 'z'}

Making a method that is called when a class's member dictionary/list is changed [duplicate]

Rather than saving a duplicate of the dictionary and comparing the old with the new, alike this:
dict = { "apple":10, "pear":20 }
if ( dict_old != dict ):
do something
dict_old = dict
How is it possible to detect WHEN any element of a dictionary changes?

You could subclass dict and include some custom __setitem__ behavior:
class MyDict(dict):
def __setitem__(self, item, value):
print "You are changing the value of %s to %s!!"%(item, value)
super(MyDict, self).__setitem__(item, value)
Example usage:
In [58]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:class MyDict(dict):
: def __setitem__(self, item, value):
: print "You are changing the value of %s to %s!!"%(item, value)
: super(MyDict, self).__setitem__(item, value)
:--
In [59]: d = MyDict({"apple":10, "pear":20})
In [60]: d
Out[60]: {'apple': 10, 'pear': 20}
In [61]: d["pear"] = 15
You are changing the value of pear to 15!!
In [62]: d
Out[62]: {'apple': 10, 'pear': 15}
You would just change the print statement to involve whatever checking you need to perform when modifying.
If you are instead asking about how to check whether a particular variable name is modified, it's a much trickier problem, especially if the modification doesn't happen within the context of an object or a context manager that can specifically monitor it.
In that case, you could try to modify the dict that globals or locals points to (depending on the scope you want this to happen within) and switch it out for, e.g. an instance of something like MyDict above, except the __setitem__ you custom create could just check if the item that is being updated matches the variable name you want to check for. Then it would be like you have a background "watcher" that is keeping an eye out for changes to that variable name.
The is a very bad thing to do, though. For one, it would involve some severe mangling of locals and globals which is not usually very safe to do. But perhaps more importantly, this is much easier to achieve by creating some container class and creating the custom update / detection code there.

You could create an observer, which will monitor if the content of data has been changed.
The code below should be quite self-explanatory. It should work for nested dicts and lists.
"""Observer descriptor class allows to trigger out any arbitrary action, when the content of observed
data changes.
"""
import weakref
class Observer(object):
"""Observes attached data and trigger out given action if the content of data changes.
Observer is a descriptor, which means, it must be declared on the class definition level.
Example:
>>> def action(observer, instance, value):
... print 'Data has been modified: %s' % value
>>> class MyClass(object):
... important_data = Observer('init_value', callback=action)
>>> o = MyClass()
>>> o.important_data = 'new_value'
Data has been modified: new_value
Observer should work with any kind of built-in data types, but `dict` and `list` are strongly advice.
Example:
>>> class MyClass2(object):
... important_data = Observer({}, callback=action)
>>> o2 = MyClass2()
>>> o2.important_data['key1'] = {'item1': 'value1', 'item2': 'value2'}
Data has been modified: {'key1': {'item2': 'value2', 'item1': 'value1'}}
>>> o2.important_data['key1']['item1'] = range(5)
Data has been modified: {'key1': {'item2': 'value2', 'item1': [0, 1, 2, 3, 4]}}
>>> o2.important_data['key1']['item1'][0] = 'first'
Data has been modified: {'key1': {'item2': 'value2', 'item1': ['first', 1, 2, 3, 4]}}
Here is an example of using `Observer` as a base class.
Example:
>>> class AdvanceDescriptor(Observer):
... def action(self, instance, value):
... logger = instance.get_logger()
... logger.info(value)
...
... def __init__(self, additional_data=None, **kwargs):
... self.additional_data = additional_data
...
... super(AdvanceDescriptor, self).__init__(
... callback=AdvanceDescriptor.action,
... init_value={},
... additional_data=additional_data
... )
"""
def __init__(self, init_value=None, callback=None, **kwargs):
"""
Args:
init_value: initial value for data, if there is none
callback: callback function to evoke when the content of data will change; the signature of
the callback should be callback(observer, instance, value), where:
observer is an Observer object, with all additional data attached to it,
instance is an instance of the object, where the actual data lives,
value is the data itself.
**kwargs: additional arguments needed to make inheritance possible. See the example above, to get an
idea, how the proper inheritance should look like.
The main challenge here comes from the fact, that class constructor is used inside the class methods,
which is quite tricky, when you want to change the `__init__` function signature in derived classes.
"""
self.init_value = init_value
self.callback = callback
self.kwargs = kwargs
self.kwargs.update({
'callback': callback,
})
self._value = None
self._instance_to_name_mapping = {}
self._instance = None
self._parent_observer = None
self._value_parent = None
self._value_index = None
#property
def value(self):
"""Returns the content of attached data.
"""
return self._value
def _get_attr_name(self, instance):
"""To respect DRY methodology, we try to find out, what the original name of the descriptor is and
use it as instance variable to store actual data.
Args:
instance: instance of the object
Returns: (str): attribute name, where `Observer` will store the data
"""
if instance in self._instance_to_name_mapping:
return self._instance_to_name_mapping[instance]
for attr_name, attr_value in instance.__class__.__dict__.iteritems():
if attr_value is self:
self._instance_to_name_mapping[weakref.ref(instance)] = attr_name
return attr_name
def __get__(self, instance, owner):
attr_name = self._get_attr_name(instance)
attr_value = instance.__dict__.get(attr_name, self.init_value)
observer = self.__class__(**self.kwargs)
observer._value = attr_value
observer._instance = instance
return observer
def __set__(self, instance, value):
attr_name = self._get_attr_name(instance)
instance.__dict__[attr_name] = value
self._value = value
self._instance = instance
self.divulge()
def __getitem__(self, key):
observer = self.__class__(**self.kwargs)
observer._value = self._value[key]
observer._parent_observer = self
observer._value_parent = self._value
observer._value_index = key
return observer
def __setitem__(self, key, value):
self._value[key] = value
self.divulge()
def divulge(self):
"""Divulges that data content has been change calling callback.
"""
# we want to evoke the very first observer with complete set of data, not the nested one
if self._parent_observer:
self._parent_observer.divulge()
else:
if self.callback:
self.callback(self, self._instance, self._value)
def __getattr__(self, item):
"""Mock behaviour of data attach to `Observer`. If certain behaviour mutate attached data, additional
wrapper comes into play, evoking attached callback.
"""
def observe(o, f):
def wrapper(*args, **kwargs):
result = f(*args, **kwargs)
o.divulge()
return result
return wrapper
attr = getattr(self._value, item)
if item in (
['append', 'extend', 'insert', 'remove', 'pop', 'sort', 'reverse'] + # list methods
['clear', 'pop', 'update'] # dict methods
):
return observe(self, attr)
return attr
def action(self, instance, value):
print '>> log >', value, '<<'
class MyClass(object):
meta = Observer('', action)
mc1 = MyClass()
mc2 = MyClass()
mc1.meta = {
'a1': {
'a11': 'a11_val',
'a22': 'a22_val',
},
'b1': 'val_b1',
}
mc1.meta['a1']['a11'] = ['1', '2', '4']
mc1.meta['a1']['a11'].append('5')
mc1.meta.update({'new': 'new_value'})
mc2.meta = 'test'
mc2.meta = 'test2'
mc2.meta = range(10)
mc2.meta[5] = 'test3'
mc2.meta[9] = {
'a': 'va1',
'b': 'va2',
'c': 'va3',
'd': 'va4',
'e': 'va5',
}
mc2.meta[9]['a'] = 'val1_new'
class MyClass2(object):
pkg = Observer('', action)
mc3 = MyClass2()
mc3.pkg = 'test_myclass2'
print mc1.meta.value

To go a bit further than #EMS;
Subclass dict and additionally add a sentinal attribute to keep track of changes and a method to inform you if if anything has changed.
class MyDict(dict):
def __init__(self):
super(MyDict, self).__init__
self.sentinal = list()
def __setitem__(self, item, value):
self.sentinal.append(item)
super(MyDict, self).__setitem__(item, value)
def __getitem__(self, item):
self.sentinal.remove(item)
return super(MyDict, self).__getitem__(item)
def update(self, iterable):
super(MyDict, self).update(iterable)
self.sentinal.extend(k for k, v in iterable)
def items(self):
self.sentinal = list()
return super(MyDict, self).items()
def iteritems(self):
self.sentinal = list()
return super(MyDict, self).iteritems()
def item_changed(self):
return bool(self.sentinal), self.sentinal
>>> d = MyDict()
>>> d.update(((i, i*i) for i in xrange(5)))
>>> d
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
>>> d[1] = 'g'
>>> d.item_changed()
(True, [1])
>>> z = d[1]
>>> d.item_changed()
(False, [])
>>> d[3] = 'b'
>>> d[4] = 'foo'
>>> d
{0: 0, 1: 'g', 2: 4, 3: 'b', 4: 'foo'}
>>> d.item_changed()
(True, [3, 4])
>>> d.items()
[(0, 0), (1, 'g'), (2, 4), (3, 'b'), (4, 'foo')]
>>> d.item_changed()
(False, [])
>>> d.update([(0, 'bar'), (2, 'baz')])
>>> d
{0: 'bar', 1: 'g', 2: 'baz', 3: 'b', 4: 'foo'}
>>> d.item_changed()
(True, [0, 2])
>>> list(d.iteritems())
foo
[(0, 'bar'), (1, 'g'), (2, 'baz'), (3, 'b'), (4, 'foo')]
>>> d.item_changed()
(False, [])
>>>

The simplest solution I managed to solve my particular instance of this problem was to hash a string of the collective __repr__() of each object in the dictionary and compare hashes to see if any changes were made:
checksum = make_hash(d)
def make_hash(d):
check = ''
for key in d:
check += str(d[key])
return hash(check)
if checksum != make_hash(d):
print('Dictionary changed')

My jsonfile module detects changes of (nested) JSON compatible Python objects. Just subclass JSONFileRoot to adapt change detection for your needs.
>>> import jsonfile
>>> class DoSomething(jsonfile.JSONFileRoot):
... def on_change(self):
... print("do something")
...
>>> d = DoSomething({"apple": 10, "pear": 20})
>>> d.data["apple"] += 1
do something
>>> d.data
{'apple': 11, 'pear': 20}
>>> d.data["plum"] = 5
do something
>>> d.data
{'apple': 11, 'pear': 20, 'plum': 5}

No need to subclass, if you only want to detect a change in it:
dict1 == dict2
will sort you.

This is the approach I've used with several dictionaries, loaded in from excel spreadsheets, and stored as an objects data member. The is_dirty property can be checked anytime to find out if I've altered any of the dictionaries. This approach is similar to the one suggested by #jamesek using the str() to get its string representation but we may not print out all of an object's data (for objects stored in a dictionary).
#property
def is_dirty(self):
return self.data_hash != self.get_data_hash()
def get_data_hash(self):
data = [
self.dictionary1,
self.dictionary2,
self.dictionary3,
]
m = hashlib.sha256()
for item in data:
m.update(
pickle.dumps(item)
)
return m.digest()
def load(self):
# do stuff to load in the data to
# self.dictionary1, self.dictionary2 & self.dictionary3
# then get and store the data's current hashed value
self.data_hash = self.get_data_hash()

How to achieve the reverse of "attr.asdict(MyObject)" using Python module 'attrs'

In documentation of Python module attrs stated that there is a method to convert attributes’ class into dictionary representation:
Example:
>>> #attr.s
... class Coordinates(object):
... x = attr.ib()
... y = attr.ib()
...
>>> attr.asdict(Coordinates(x=1, y=2))
{'x': 1, 'y': 2}
How can I achive the oposite: get instance of the Coordinates from its valid dictionary representation without boilerplate and with a joy of the attrs module?

Apparently as easy as using dictionary unpacking (double star) operator in corresponding attrs class instantiation.
Example:
>>> Coordinates(**{'x': 1, 'y': 2})
Coordinates(x=1, y=2)

As a more universal solution, which works with attrs nested classes, enums or any other type annotated structures you can use https://github.com/Tinche/cattrs. It also supports structure/unstructure customization by defining the structure/unstructure hooks
Example:
import attr, cattr
#attr.s(slots=True, frozen=True) # It works with normal classes too.
class C:
a = attr.ib()
b = attr.ib()
instance = C(1, 'a')
cattr.unstructure(instance)
# {'a': 1, 'b': 'a'}
cattr.structure({'a': 1, 'b': 'a'}, C)
# C(a=1, b='a')

I do this in my web app to be able to serialize/de-serialize into JSON:
First I made a method on my classes that returns a more serializing friendly version:
def asdict(self, serializable=True):
if serializable:
as_dict['type'] = self.__class__.__name__
return as_dict
else:
return attr.asdict(self)
Then when I need to convert one of these dictionaries (JSON objects actually) back into a class instance:
obj_type = values.pop('type')
if obj_type in obj_list:
obj = getattr(sys.modules[__name__], obj_type)(**values)

json.dumps TypeError on Python dict

The following class is implemented to provide a generic object that can be passed through network as a json-encoded dictionary. I'm actually trying to json encode a dict (!) but it won't work.
I know it will work with custom encoder class, but I don't see why it's necessary when I'm just encoding a dict.
Can someone explain the TypeError or offer a way to encode this without subclassing JSONEncoder?
Here is the bad behavior.
>>> def tree(): return CustomDict(tree)
>>> d = tree()
>>> d['one']['test']['four'] = 19
>>> d.dict
{ 'one' : { 'test': {'four': 19}}}
>>> type(d.dict)
<type 'dict'>
>>> import json
>>> json.dumps(d.dict)
# stacktrace removed
TypeError: {'one': {'test': {'four': 19}}} is not JSON serializable
>>> normal_d = {'one': {'test': {'four': 19}}}
>>> type(normal_d)
<type 'dict'>
>>> json.dumps(normal_d)
"{'one': {'test': {'four': 19}}}"
>>> normal_d == d
True
I would love to be able to do the following
>>>> json.dumps(dict(d))
"{'one': {'test': {'four': 19}}}"
but I added the dict property to 'force it' (didn't work obviously). Now it's an even bigger mystery. Here is the code for the CustomDict class
class CustomDict(collections.MutableMapping):
"""
A defaultdict-like object that can also have properties and special methods
"""
def __init__(self, default_type=str, *args, **kwargs):
"""
instantiate as a default-dict (str if type not provided). Try to update
self with each arg, and then update self with kwargs.
#param default_type: the type of the default dict
#type default_type: type (or class)
"""
self._type = default_type
self._store = collections.defaultdict(default_type)
self._dict = {}
for arg in args:
if isinstance(arg, collections.MutableMapping):
self.update(arg)
self.update(kwargs)
#property
def dict(self):
return self._dict
def __contains__(self, key):
return key in self._store
def __len__(self):
return len(self._store)
def __iter__(self):
return iter(self._store)
def __getitem__(self, key):
self._dict[key] = self._store[key]
return self._store[key]
def __setitem__(self, key, val):
self._dict[key] = val
self._store[key] = val
def __delitem__(self, key):
del self._store[key]
def __str__(self):
return str(dict(self._store))

You want to make your type a subclass of dict, not of collections.MutableMapping, really.
Even better still, use collections.defaultdict directly instead, it already is a subclass of dict and can be used to implement your tree 'type' easily:
from collections import defaultdict
def Tree():
return defaultdict(Tree)
tree = Tree()
Demonstration:
>>> from collections import defaultdict
>>> def Tree():
... return defaultdict(Tree)
...
>>> tree = Tree()
>>> tree['one']['two'] = 'foobar'
>>> tree
defaultdict(<function Tree at 0x107f40e60>, {'one': defaultdict(<function Tree at 0x107f40e60>, {'two': 'foobar'})})
>>> import json
>>> json.dumps(tree)
'{"one": {"two": "foobar"}}'
If you must add your own methods and behaviour, then I'd subclass defaultdict and build upon that base:
class CustomDict(defaultdict):
pass
As this is still a subclass of dict, the json library will happily convert that to a JSON object without special handling.

Adding dynamic property to a python object

site = object()
mydict = {'name': 'My Site', 'location': 'Zhengjiang'}
for key, value in mydict.iteritems():
setattr(site, key, value)
print site.a # it doesn't work
The above code didn't work. Any suggestion?

The easiest way to populate one dict with another is the update() method, so if you extend object to ensure your object has a __dict__ you could try something like this:
>>> class Site(object):
... pass
...
>>> site = Site()
>>> site.__dict__.update(dict)
>>> site.a
Or possibly even:
>>> class Site(object):
... def __init__(self,dict):
... self.__dict__.update(dict)
...
>>> site = Site(dict)
>>> site.a

As docs say, object() returns featureless object, meaning it cannot have any attributes. It doesn't have __dict__.
What you could do is the following:
>>> site = type('A', (object,), {'a': 42})
>>> site.a
42

class site(object):
pass
for k,v in dict.iteritems():
setattr(site,k,v)
print site.a #it does works

We Keep Coding

Python is a programming language that lets you work quickly and integrate systems more effectively.

Python dictionary from an object's fields - python

Do you know if there is a built-in function to build a dictionary from an arbitrary object? I'd like to do something like this: >>> class Foo: ... bar = 'hello' ... baz = 'world' ... >>> f = Foo() >>> props(f) { 'bar' : 'hello', 'baz' : 'world' } NOTE: It should not include methods. Only fields.

Instead of x.dict, it's actually more pythonic to use vars(x).

I've settled with a combination of both answers: dict((key, value) for key, value in f.dict.iteritems() if not callable(value) and not key.startswith('__'))

A downside of using dict is that it is shallow; it won't convert any subclasses to dictionaries. If you're using Python3.5 or higher, you can use jsons: >>> import jsons >>> jsons.dump(f) {'bar': 'hello', 'baz': 'world'}

To build a dictionary from an arbitrary object, it's sufficient to use dict. This misses attributes that the object inherits from its class. For example, class c(object): x = 3 a = c() hasattr(a, 'x') is true, but 'x' does not appear in a.dict

Python3.x return dict((key, value) for key, value in f.dict.items() if not callable(value) and not key.startswith('__'))

vars() is great, but doesn't work for nested objects of objects Convert nested object of objects to dict: def to_dict(self): return json.loads(json.dumps(self, default=lambda o: o.dict))

Dataclass(from Python 3.7) is another option which can be used for converting class properties to dict. asdict can be used along with dataclass objects for the conversion. Example: #dataclass class Point: x: int y: int p = Point(10, 20) asdict(p) # it returns {'x': 10, 'y': 20}

If you want to list part of your attributes, override dict: def dict(self): d = { 'attr_1' : self.attr_1, ... } return d # Call dict d = instance.dict() This helps a lot if your instance get some large block data and you want to push d to Redis like message queue.

Try: from pprint import pformat a_dict = eval(pformat(an_obj))

Related

How do I update the dictionary of a mapped trait, after I've already constructed it?

Making a method that is called when a class's member dictionary/list is changed [duplicate]

How to achieve the reverse of "attr.asdict(MyObject)" using Python module 'attrs'

json.dumps TypeError on Python dict

Adding dynamic property to a python object

Categories

Resources

We Keep Coding

Python is a programming language that lets you work quickly and integrate systems more effectively.

Python dictionary from an object's fields - python

Do you know if there is a built-in function to build a dictionary from an arbitrary object? I'd like to do something like this: >>> class Foo: ... bar = 'hello' ... baz = 'world' ... >>> f = Foo() >>> props(f) { 'bar' : 'hello', 'baz' : 'world' } NOTE: It should not include methods. Only fields.

Instead of x.__dict__, it's actually more pythonic to use vars(x).

I've settled with a combination of both answers: dict((key, value) for key, value in f.__dict__.iteritems() if not callable(value) and not key.startswith('__'))

A downside of using __dict__ is that it is shallow; it won't convert any subclasses to dictionaries. If you're using Python3.5 or higher, you can use jsons: >>> import jsons >>> jsons.dump(f) {'bar': 'hello', 'baz': 'world'}

To build a dictionary from an arbitrary object, it's sufficient to use __dict__. This misses attributes that the object inherits from its class. For example, class c(object): x = 3 a = c() hasattr(a, 'x') is true, but 'x' does not appear in a.__dict__

Python3.x return dict((key, value) for key, value in f.__dict__.items() if not callable(value) and not key.startswith('__'))

vars() is great, but doesn't work for nested objects of objects Convert nested object of objects to dict: def to_dict(self): return json.loads(json.dumps(self, default=lambda o: o.__dict__))

Dataclass(from Python 3.7) is another option which can be used for converting class properties to dict. asdict can be used along with dataclass objects for the conversion. Example: #dataclass class Point: x: int y: int p = Point(10, 20) asdict(p) # it returns {'x': 10, 'y': 20}

If you want to list part of your attributes, override __dict__: def __dict__(self): d = { 'attr_1' : self.attr_1, ... } return d # Call __dict__ d = instance.__dict__() This helps a lot if your instance get some large block data and you want to push d to Redis like message queue.

Try: from pprint import pformat a_dict = eval(pformat(an_obj))

Related

How do I update the dictionary of a mapped trait, after I've already constructed it?

Making a method that is called when a class's member dictionary/list is changed [duplicate]

How to achieve the reverse of "attr.asdict(MyObject)" using Python module 'attrs'

json.dumps TypeError on Python dict

Adding dynamic property to a python object

Categories

Resources

Instead of x.dict, it's actually more pythonic to use vars(x).

I've settled with a combination of both answers: dict((key, value) for key, value in f.dict.iteritems() if not callable(value) and not key.startswith('__'))

A downside of using dict is that it is shallow; it won't convert any subclasses to dictionaries. If you're using Python3.5 or higher, you can use jsons: >>> import jsons >>> jsons.dump(f) {'bar': 'hello', 'baz': 'world'}

To build a dictionary from an arbitrary object, it's sufficient to use dict. This misses attributes that the object inherits from its class. For example, class c(object): x = 3 a = c() hasattr(a, 'x') is true, but 'x' does not appear in a.dict

Python3.x return dict((key, value) for key, value in f.dict.items() if not callable(value) and not key.startswith('__'))

vars() is great, but doesn't work for nested objects of objects Convert nested object of objects to dict: def to_dict(self): return json.loads(json.dumps(self, default=lambda o: o.dict))

If you want to list part of your attributes, override dict: def dict(self): d = { 'attr_1' : self.attr_1, ... } return d # Call dict d = instance.dict() This helps a lot if your instance get some large block data and you want to push d to Redis like message queue.