We Keep Coding

Dynamically accessing nested dictionary keys?

Is there some simple way to access nested dictionary key when at first you don't know which key you will be accessing?
For example:
dct = {'label': 'A', 'config': {'value': 'val1'}}
In this dictionary I will need to access either label key or value key inside another dict that is accessible through config key.
It depends on state.
For example if we have variable called label, so if:
label = True
if label:
key = 'label'
in this case its kind of easy:
dct[key]
Now if label is false and I need to access nested dictionary, how can I dynamically specify it so I would not need to use ifs on every iterated item (I mean check everytime if label is used instead of value, because I will know that before starting iteration on dictionary full of dct dictionaries)?
like:
label = False
if label:
key = 'label'
else:
key = 'config..?' # it should be something like ['config']['value']

Expanding on #Barun's work, and possibly helping answer #Bhavani's question re setting values in a nested dictionary, here is a generalised solution to dynamically accessing or setting nested dictionary keys. Python 3.7.
from typing import List
class DynamicAccessNestedDict:
"""Dynamically get/set nested dictionary keys of 'data' dict"""
def __init__(self, data: dict):
self.data = data
def getval(self, keys: List):
data = self.data
for k in keys:
data = data[k]
return data
def setval(self, keys: List, val) -> None:
data = self.data
lastkey = keys[-1]
for k in keys[:-1]: # when assigning drill down to *second* last key
data = data[k]
data[lastkey] = val
You just wrap your dictionary in an instance of this class, then get and set by passing a list of keys.
dct = {'label': 'A', 'config': {'value': 'val1'}}
d = DynamicAccessNestedDict(dct)
assert d.getval(["label"]) == "A"
assert d.getval(["config", "value"]) == "val1"
# Set some new values
d.setval(["label"], "B")
d.setval(["config", "value"], "val2")
assert d.getval(["label"]) == "B"
assert d.getval(["config", "value"]) == "val2"

If you know the key to be traversed, you can try out the following. This would work for any level of nested dicts.
dct = {'label': 'A', 'config': {'value': 'val1'}}
label = True
key = ('label',)
if not label:
key = ('config', 'value')
ret = dct
for k in key:
ret = ret[k]
print ret

from functools import reduce
import operator
#db is dictionary which need to update
#keys is a list need to depth of keys in a order
#value targeted value to update in dictionary
class Dict_update:
def set_by_path(self, db, keys, value):
"""Set a value in a nested object in db by keys sequence."""
for index in range(1,len(keys)):
subitem = self.get_by_path(db, keys[:index])
if not isinstance(subitem, dict):
self.get_by_path(db, keys[:index][:-1])[keys[:index][-1]] = {}
self.get_by_path(db, keys[:-1])[keys[-1]] = value
return db
def get_by_path(self, db, keys):
try: return reduce(operator.getitem, keys, db)
except Exception as e:
return None

How do I use dictionary key,value pair to set class instance attributes "pythonic"ly?

I have created some Python classes to use as multivariate data structures, which are then used for various tasks. In some instances, I like to populate the classes with various value sets. The default parameter filename "ho2.defaults" would look something like this:
name = 'ho2'
mass_option = 'h1o16'
permutation = 'odd'
parity = 'odd'
j_total = 10
lr = 40
br = 60
jmax = 60
mass_lr = 14578.471659
mass_br = 1781.041591
length_lr = ( 1.0, 11.0, 2.65 )
length_br = ( 0.0, 11.0, 2.46 )
use_spline = True
energy_units = 'au'
pes_zpe = -7.407998138300982E-2
pes_cutoff = 0.293994
Currently, I create a dictionary from reading the desired key,value pairs from file, and now I'd like a "pythonic" way of making those dictionary keys be class instance variable names, i.e.
# Instantiate Molecule Class
molecule = Molecule()
# Create Dictionary of default values
default_dict = read_dict_from_file(filename)
# Set populate class instance variables with dictionary values
for key,value in default_dict:
molecule.key = value
So the Class's instance variable "molecule.name" could be set with the dictionary key,value pair. I could do this by hand, but I'ms sure there is a better way to loop through it. In actuality, the dictionary could be large, and I'd rather allow the user to choose which values they want to populate, so the dictionary could change. What am I missing here?

You would use setattr: setattr(molecule, key, value)

The simple way is:
vars(molecule).update(default_dict)
This will clobber any pre-existing attributes though. For a more delicate approach try:
for name, value in default_dict.items():
if not hasattr(molecule, name):
setattr(molecule, name value)

I'd invert the logic so that the object dynamically answers questions:
class Settings(object):
ATTRS = {'foo', 'bar'}
def __init__(self, defaults):
self.__dict__['data'] = defaults.copy()
def __getattr__(self, key):
if key not in self.ATTRS or key not in self.data:
raise AttributeError("'{}' object has no attribute '{}'".format(
self.__class__.__name__, key))
return self.data[key]
def __setattr__(self, key, value):
self.data[key] = value
s = Settings({'a': 'b', 'foo': 'foo!', 'spam': 'eggs'})
print s.foo
try:
print s.spam
except AttributeError:
pass
else:
raise AssertionError("That should have failed because 'spam' isn't in Settings.ATTRS")
try:
print s.bar
except AttributeError:
pass
else:
raise AssertionError("That should have failed because 'bar' wasn't passed in")
class Molecule(settings):
ATTRS = {'name', 'mass_option', ...}
molecule = Molecule(default_dict)

Language: Python(vers3.3); If statements inside while and if statements

My program is unable to go into the last of my if/else statement.
My code:
def main_prog():
while True:
data_dict = {'123': 'viksun', '111': ['tord'], '333': ['test']} # Data storage.
print (data_dict) # Track-print
prompt = input('Enter command >>> ').split(" ")
if prompt[0] == 'lookup':
B_value = name_lookup_return_boolean(data_dict, prompt[1])
if B_value == True:
print (prompt[1],"'s number is:", name_lookup(data_dict, prompt[1]))
else:
print("The name:" ,prompt[1], "do not exist, please try a different name")
Data struction of the dictionary: data_dict
data_dict = {'123': ['name1'], '111': ['tord'], '333': ['test']} # Data storage.
Function descriptions:
- name_lookup_returns_boolean(data_dict, name) returns a boolean value: True if the name is located in the dictionary and False if the name does not exist.
- name_lookup returns the key that corresponds to the specific name.
The functions have been tested separately from the whole program, so they are working. If you need them, I can also post them.
I can't get the program to run the last else statement in the program. This is going to run if the name does not exist in the dictionary: data_dict.
Here is the implementation of the two functions:
def name_lookup(data_dict, name):
for key, value in data_dict.items():
if name in value:
return key
def name_lookup_return_boolean(data_dict, name):
for value in data_dict.items():
if name in value:
return True
else:
return False
This is the variation I have tried to use_but with no sucess:
def name_lookup_version_02(data_dict, name):
for value in data_dict.values():
if name in value:
return True
else:
return False

In this line:
if B_value == True:
I'm guessing that B_value holds a true value, but not True. Try this instead:
if B_value:

The problem is in your name_lookup_return_boolean function. You are returning True for both conditions. Also you should be enumerating both key and value otherwise value will be assigned a tuple.
It should look like this:
def name_lookup_return_boolean(data_dict, name):
for key,value in data_dict.items():
if name in value:
return True
UPDATE: After testing, I realised previous answer was wrong - was only matching the first value

Python StructuredProperty to dictionary

My models all have a method which converts the model to a dictionary:
def to_dict(model):
output = {}
SIMPLE_TYPES = (int, long, float, bool, dict, basestring, list)
for key, prop in model._properties.iteritems():
value = getattr(model, key)
if value is None:
continue
if isinstance(value, SIMPLE_TYPES):
output[key] = value
elif isinstance(value, datetime.date):
dateString = value.strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
output[key] = dateString
elif isinstance(value, ndb.Model):
output[key] = to_dict(value)
else:
raise ValueError('cannot encode ' + repr(prop))
return output
Now, one of my models, X, has a LocalStructuredProperty:
metaData = ndb.LocalStructuredProperty(MetaData, repeated=True)
So, repeated=True means this will be a list of MetaData objects. MetaData is another model, and it also has the same to_dict method.
However, when I call json.dumps(xInstance.to_dict()), I get an exception:
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: MetaData(count=0, date=datetime.datetime(2012, 9, 19, 2, 46, 56, 660000), unique_id=u'8E2C3B07A06547C78AB00DD73B574B8C') is not JSON serializable
How can I handle this?

If you want to handle this in to_dict() and before the level of serializing to JSON, you'll just need a few more cases in your to_dict(). Firstly, you said the to_dict definition above is a method. I would have it delegate to a function or staticmethod so you have something you can call on ints and such without checking the type first. The code will just come out better that way.
def coerce(value):
SIMPLE_TYPES = (int, long, float, bool, basestring)
if value is None or isinstance(value, SIMPLE_TYPES):
return value
elif isinstance(value, datetime.date):
return value.strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
elif hasattr(value, 'to_dict'): # hooray for duck typing!
return value.to_dict()
elif isinstance(value, dict):
return dict((coerce(k), coerce(v)) for (k, v) in value.items())
elif hasattr(value, '__iter__'): # iterable, not string
return map(coerce, value)
else:
raise ValueError('cannot encode %r' % value)
Then just plug that into your to_dict method itself:
def to_dict(model):
output = {}
for key, prop in model._properties.iteritems():
value = coerce(getattr(model, key))
if value is not None:
output[key] = value
return output

All you need to do to serialize is to implement a function
def default_encode(obj):
return obj.to_dict()
and then encode your JSON with
json.dumps(X.to_dict(), default=default_encode)

I figured out how to solve the issue: in the X class, add this to the to_dict() method:
...
if value is None:
continue
if key == 'metaData':
array = list()
for data in value:
array.append(data.to_dict())
output[key] = array
elif isinstance(value, SIMPLE_TYPES):
output[key] = value
...
Though I'm not really sure how to automate this case where it's not based off key, but rather whenever it encounters a list of custom objects, it first converts each object in the list to_dict() first.

How to recursively remove certain keys from a multi-dimensional(depth not known) python dictionary?

I'm using kendoUI Grid in one of my projects. I retrieved a piece of data using their api and found that it added some "unwanted" data to my json/dictionary. After passing the json back to my Pyramid backend, I need to remove these keys. The problem is, the dictionary can be of whatever depth and I don't know the depth in advance.
Example:
product = {
id: "PR_12"
name: "Blue shirt",
description: "Flowery shirt for boys above 2 years old",
_event: {<some unwanted data here>},
length: <some unwanted data>,
items: [{_event: {<some rubbish data>}, length: <more rubbish>, price: 23.30, quantity: 34, color: "Red", size: "Large"}, {_event: {<some more rubbish data>}, length: <even more rubbish>, price: 34.50, quantity: 20, color: "Blue", size: "Large"} ....]
}
I want to remove two keys in particular: "_event" & "length". I tried writing a recursive function to remove the data but I can't seem to get it right. Can someone please help?
Here's what I have:
def remove_specific_key(the_dict, rubbish):
for key in the_dict:
if key == rubbish:
the_dict.pop(key)
else:
# check for rubbish in sub dict
if isinstance(the_dict[key], dict):
remove_specific_key(the_dict[key], rubbish)
# check for existence of rubbish in lists
elif isinstance(the_dict[key], list):
for item in the_dict[key]:
if item == rubbish:
the_dict[key].remove(item)
return the_dict

If you allow remove_specific_key (renamed remove_keys) to accept any object as its first argument, then you can simplify the code:
def remove_keys(obj, rubbish):
if isinstance(obj, dict):
obj = {
key: remove_keys(value, rubbish)
for key, value in obj.iteritems()
if key not in rubbish}
elif isinstance(obj, list):
obj = [remove_keys(item, rubbish)
for item in obj
if item not in rubbish]
return obj
Since you wish to remove more than one key, you might as well let rubbish be a set instead of one particular key.
With the above code, you'd remove '_event' and 'length' keys with
product = remove_keys(product, set(['_event', 'length']))
Edit: remove_key uses dict comprehension, introduced in Python2.7. For older version of Python, the equivalent would be
obj = dict((key, remove_keys(value, rubbish))
for key, value in obj.iteritems()
if key not in rubbish)

Modifying a dict as you iterate it bad, an unnecessary, since you know exactly what key you are looking for. Also, your list of dicts aren't being handled right:
def remove_specific_key(the_dict, rubbish):
if rubbish in the_dict:
del the_dict[rubbish]
for key, value in the_dict.items():
# check for rubbish in sub dict
if isinstance(value, dict):
remove_specific_key(value, rubbish)
# check for existence of rubbish in lists
elif isinstance(value, list):
for item in value:
if isinstance(item, dict):
remove_specific_key(item, rubbish)

dict or list can not be delete while iteratering, so replace the iterator with a test function.
def remove_specific_key(the_dict, rubbish):
if the_dict.has_key(rubbish):
the_dict.pop(rubbish)
else:
for key in the_dict:
if isinstance(the_dict[key], dict):
remove_specific_key(the_dict[key], rubbish)
elif isinstance(the_dict[key], list):
if the_dict[key].count(rubbish):
the_dict[key].remove(rubbish)
return the_dict
d = {"a": {"aa": "foobar"}}
remove_specific_key(d, "aa")
print d
d = {"a": ["aa", "foobar"]}
remove_specific_key(d, "aa")
print d