Suppose we have a list of dictionaries listD where each dictionary is quite deeply nested with more dictionaries. e.g.suppose that the first element of listD is:
listD[0] = {"bar1":{"bar2":{"bar3":1234}}}
Now I want to check if listD[i]["bar1"]["bar2"]["bar3"] == 1234 for all i. For the first element where i = 0, this is easy as we can just use the expression:
listD[0]["bar1"]["bar2"]["bar3"] == 1234
But I cannot simply write a loop like:
for dictelem in listD:
if dictelem["bar1"]["bar2"]["bar3"] == 1234:
print "equals 1234"
This is because some of the dictionary elements of listD might be of the form
listD[i] = {"bar1":{"bar2":"abcd"}} or
listD[i] = {"bar1":{"bar2":None}}
and if I try to access "bar3" when it doesn't exists, an error will be raised.
Right now I am manually specifying in the code to check for the existence of the bar1, bar2 and bar3 keys and whether that are in fact dictionaries or not. But this is really verbose and I'm quite sure there's a simpler way to do it but I can't figure out how.
def dictcheck(d, p, v):
if len(p):
if isinstance(d,dict) and p[0] in d:
return dictcheck(d[p[0]], p[1:], v)
else:
return d == v
You pass one dict d, one path of keys p, and the final value to check v. It will recursively go in the dicts and finally check if the last value is equal to v.
>>> dictcheck({"bar1":{"bar2":{"bar3":1234}}}, ('bar1','bar2','bar3'), 1234)
True
>>> dictcheck({"bar1":1234}, ('bar1','bar2','bar3'), 1234)
False
So, to answer your question (I want to check if listD[i]["bar1"]["bar2"]["bar3"] == 1234 for all i):
all(dictcheck(x, ('bar1','bar2','bar3'), 1234) for x in listD)
just use a try/except block this way:
for dictelem in listD:
try:
if dictelem["bar1"]["bar2"]["bar3"] == 1234:
print "equals 1234"
except TypeError:
pass
When dealing with nested dictionaries, I think of them as a tree where the keys make up the path to the value. With that in mind, I created a non-recursive function, dict_path with takes in a nested dictionary, the key path, and a value in case not found:
def dict_path(dic, path, value_if_not_found=None):
path = path.split('.')
try:
for key in path:
dic = dic[key]
return dic
except (KeyError, TypeError):
return value_if_not_found
listD = [
{"bar1": {"bar2": 'abcd'}},
{"bar1": {"bar2": None}},
{"bar1": {"bar2": {"bar3": 1234}}},
]
for dic in listD:
value = dict_path(dic, 'bar1.bar2.bar3')
if value == 1234:
print 'Equals 1234:', dic
The function keeps traversing the nested dictionary until one of these three conditions occur:
It found the value in question. In this case, return that value
Along the way, the object is a dictionary, but does not have the requested key. In this case, a KeyError is raised
The object is not a dictionary, the TypeError is raised
For case 2 and 3, we simply return value_if_not_found
you can try this
for item in listD:
if item.get("bar1",{}).get("bar2",{}).get("bar3","") == 1234:
print "yeah, gotcha"
Related
I'd like to try and find if a value is in a list of dicts, which can be done easily with:
if any(x['aKey'] == 'aValue' for x in my_list_of_dicts):
But this is only a boolean response, I'd like to not only check if the value is there, but to also access it afterwards, so something like:
for i, dictionary in enumerate(my_list_of_dicts):
if dictionary['aKey'] == 'aValue':
# Then do some stuff to that dictionary here
# my_list_of_dicts[i]['aNewKey'] = 'aNewValue'
Is there a better/more pythonic way of writing this out?
With next function, if expected to find only one target dict:
my_list_of_dicts = [{'aKey': 1}, {'aKey': 'aValue'}]
target_dict = next((d for d in my_list_of_dicts if d['aKey'] == 'aValue'), None)
if target_dict: target_dict['aKey'] = 'new_value'
print(my_list_of_dicts)
The output (input list with updated dictionary):
[{'aKey': 1}, {'aKey': 'new_value'}]
You can use a list comprehension. This will return a list of dictionaries that match your condition.
[x for x in my_list_of_dicts if x['aKey']=='aValue' ]
I am giving the user the ability to check a specific key in a multi-level dictionary. My idea is that they will pass the path to the key like this:
root.subelement1.subelement2.key
This can be of arbitrary length and depth.
Once I have the string (above) from the user, I'll split it and get a list of each individual component:
elements = ['root', 'subelement1', 'subelement2', 'key']
All of this I can do. The next part is where I am stuck. How can I query the dictionary key, specified by the above when it's arbitrary length?
My initial thought was to do something like my_dict[elements[0]][elements[1]]...but that doesn't scale or work when my user doesn't pass exactly the length I expect.
How can I get the data at an arbitrary key depth, in this case?
A couple examples:
User passes country.US.NewYork => I query `my_dict['country']['US']['NewYork']
User passes department.accounting => I query my_dict['department']['accounting']
User passes id => I query my_dict['id']
User passes district.District15.HenryBristow.principal => I query my_dict['district']['District15']['HenryBristow']['principal']
you could do that using reduce which will query the keys in the nested dictionaries:
q = "district.District15.HenryBristow.principal"
my_dict = {"district" : {"District15" : {"HenryBristow" : {"principal" : 12}}}}
from functools import reduce # python 3 only
print(reduce(lambda x,y : x[y],q.split("."),my_dict))
result:
12
If you want to avoid to catch KeyError in case the data doesn't exist with this path, you could use get with a default value as empty dictionary:
reduce(lambda x,y : x.get(y,{}),q.split("."),my_dict)
Trying to get an unknown value returns an empty dictionary. The only drawback is that you don't know from where exactly the path got lost, so maybe leaving the KeyError be raised wouldn't be so bad:
try:
v = reduce(lambda x,y : x[y],q.split("."),my_dict)
except KeyError as e:
print("Missing key: {} in path {}".format(e,q))
v = None
Use recursion. Ex:
root = {
'subelement1': {
'subelement2': {
'key': 'value'
}
}
}
elements = ['subelement1', 'subelement2', 'key']
def getElem(d, keys):
if keys == []:
return None
else:
key = keys[0]
remainingKeys = keys[1:]
if remainingKeys == []:
return d[key]
else:
if type(d[key]) == dict:
return getElem(d[key], remainingKeys)
else:
return None
print(getElem(root, elements))
from a python 2.x perspective, you can do this with reduce.
query_list = keys.split(":")
print reduce(lambda x,y: x[y], [my_dict] + query_list)
But in general, you'll want to do this with a recursive or iterative function if you want to do error handling beyond throwing a KeyError.
You can transverse the dictionary using a for loop:
s = 'root.subelement1.subelement2.key'
d1 = {'root':{'subelement1':{'subelement2':{'key':15, 'key1':18}}}}
new_d = d1
for key in s.split('.'):
new_d = new_d[key]
print(new_d)
Output:
15
u can do this like below
my_dict = someDict
tmpDict = dict(someDict) # a coppy of dict
input = "x.u.z"
array = input.split(".")
for key in array:
tmpDict = tmpDict[key]
print(tmpDict)
but your question is very challenging:
u say if user send country.us then go to my-dict.country.us
but what happen if one of this path in my_dict be a list code will results error
u can handle this by check type
if isinstance(tmpDict , dict ):
tmpDict = tmpDict[key]
else:
# u should say what u want else (a Recursive method u will need)
edit
if user address maybe wrong you should check my_dict have this field or not sample code is below but will be many if i don't like that!
if key not in tmpDict:
print("Bad Path")
return
I have a python dictionary containing n key-value pairs, out of which n-1 values are identical and 1 is not. I need to find the key of the distinct element.
For example: consider a python list [{a:1},{b:1},{c:2},{d:1}]. I need the to get 'c' as the output.
I can use a for loop to compare consecutive elements and then use two more for loops to compare those elements with the other elements. But is there a more efficient way to go about it or perhaps a built-in function which I am unaware of?
If you have a dictionary you can quickly check and find the first value which is different from the next two values cycling around the keys of your dictionary.
Here's an example:
def find_different(d):
k = d.keys()
for i in xrange(0, len(k)):
if d[k[i]] != d[k[(i+1)%len(k)]] and d[k[i]] != d[k[(i+2)%len(k)]]:
return k[i]
>>> mydict = {'a':1, 'b':1, 'c':2, 'd':1}
>>> find_different(mydict)
'c'
Otherwise, if what you have is a list of single-key dictionaries, then you can do it quite nicely mapping your list with a function which "extracts" the values from your elements, then check each one using the same logic.
Here's another working example:
def find_different(l):
mask = map(lambda x: x[x.keys()[0]], l)
for i in xrange(0, len(l)):
if mask[i] != mask[(i+1)%len(l)] and mask[i] != mask[(i+2)%len(l)]:
return l[i].keys()[0]
>>> mylist = [{'a':1},{'b':1},{'c':2},{'d':1}]
>>> find_different(mylist)
'c'
NOTE: these solutions do not work in Python 3 as the map function doesn't return a list and neither does the .keys() method of dictionaries.
Assuming that your "list of pairs" (actually list of dictionaries, sigh) cannot be changed:
from collections import defaultdict
def get_pair(d):
return (d.keys()[0], d.values()[0])
def extract_unique(l):
d = defaultdict(list)
for key, value in map(get_pair, l):
d[value].append(key)
return filter(lambda (v,l): len(l) == 1, d.items())[0][1]
If you already have your dictionary, then you make a list of all of the keys: key_list = yourDic.keys(). Using that list, you can then loop through your dictionary. This is easier if you know one of the values, but below I assume that you do not.
yourDic = {'a':1, 'b':4, 'c':1, 'd':1, }
key_list = yourDic.keys()
previous_value = yourDic[key_list[0]] # Making it so loop gets past first test
count = 0
for key in key_list:
test_value = yourDic[key]
if (test_value != previous_value) and count == 1: # Checks first key
print key_list[count - 1]
break
elif (test_value != previous_value):
print key
break
else:
previous_value = test_value
count += 1
So, once you find the value that is different, it will print the key. If you want it to print the value, too, you just need a print test_value statement
Basically, if I'm trying to access a dict value which I expect to be an iterable is there an easy one-liner to account for that value not being present aside from using some like DefaultDict. There's this
for el in (myDict.get('myIterable') or []):
pass
Doesn't feel particularly pythonic though...
for item in a_dict.get("some_key",[]):
#do whatever
if the item is guaranteed to be a list if present ... if it might be things other than a list you will need a different solution
You can make a subclass of dict that provides a default value with the __missing__(self, key) method:
class EmptyIterableDict(dict):
def __missing__(self, key):
return []
Example usage:
test = EmptyIterableDict()
test['a'] = [3,2,1]
test['b'] = [2,1]
test['c'] = [1]
for v in test['a']:
print v
3
2
1
for v in test['d']:
print v
If you already have a vanilla dict that you want to iterate like that over, you can make a temporary copy:
original = {'a': [1], 'b': [2,3]}
temp = EmptyIterableDict(original)
for v in temp['d']:
print v
An explicit, multi-line approach to this would be:
if 'my_iterable' in my_dict:
for item in my_dict['my_iterable']:
print(item)
which could also be written as a one-line comprehension:
[print(item) for item in my_dict['my_iterable'] if 'my_iterable' in my_dict]
This isn't a one-liner but it accounts for both possible failures.
try:
for item in dictionary[key]:
print(item)
except KeyError:
pass # Key wasn't present in the dictionary.
except TypeError:
pass # Key was present but corresponding item not iterable.
Many SO posts show you how to efficiently check the existence of a key in a dictionary, e.g., Check if a given key already exists in a dictionary
How do I do this for a multi level key? For example, if d["a"]["b"] is a dict, how can I check if d["a"]["b"]["c"]["d"] exists without doing something horrendous like this:
if "a" in d and isInstance(d["a"], dict) and "b" in d["a"] and isInstance(d["a"]["b"], dict) and ...
Is there some syntax like
if "a"/"b"/"c"/"d" in d
What I am actually using this for: we have jsons, parsed into dicts using simplejson, that I need to extract values from. Some of these values are nested three and four levels deep; but sometimes the value doesn't exist at all. So I wanted something like:
val = None if not d["a"]["b"]["c"]["d"] else d["a"]["b"]["c"]["d"] #here d["a"]["b"] may not even exist
EDIT: prefer not to crash if some subkey exists but is not a dictionary, e.g, d["a"]["b"] = 5.
Sadly, there isn't any builtin syntax or a common library to query dictionaries like that.
However, I believe the simplest(and I think it's efficient enough) thing you can do is:
d.get("a", {}).get("b", {}).get("c")
Edit: It's not very common, but there is: https://github.com/akesterson/dpath-python
Edit 2: Examples:
>>> d = {"a": {"b": {}}}
>>> d.get("a", {}).get("b", {}).get("c")
>>> d = {"a": {}}
>>> d.get("a", {}).get("b", {}).get("c")
>>> d = {"a": {"b": {"c": 4}}}
>>> d.get("a", {}).get("b", {}).get("c")
4
This isn't probably a good idea and I wouldn't recommend using this in prod. However, if you're just doing it for learning purposes then the below might work for you.
def rget(dct, keys, default=None):
"""
>>> rget({'a': 1}, ['a'])
1
>>> rget({'a': {'b': 2}}, ['a', 'b'])
2
"""
key = keys.pop(0)
try:
elem = dct[key]
except KeyError:
return default
except TypeError:
# you gotta handle non dict types here
# beware of sequences when your keys are integers
if not keys:
return elem
return rget(elem, keys, default)
UPDATE: I ended up writing my own open-source, pippable library that allows one to do this: https://pypi.python.org/pypi/dictsearch
A non-recursive version, quite similar to #Meitham's solution, which does not mutate the looked-for key. Returns True/False if the exact structure is present in the source dictionary.
def subkey_in_dict(dct, subkey):
""" Returns True if the given subkey is present within the structure of the source dictionary, False otherwise.
The format of the subkey is parent_key:sub_key1:sub_sub_key2 (etc.) - description of the dict structure, where the
character ":" is the delemiter.
:param dct: the dictionary to be searched in.
:param subkey: the target keys structure, which should be present.
:returns Boolean: is the keys structure present in dct.
:raises AttributeError: if subkey is not a string.
"""
keys = subkey.split(':')
work_dict = dct
while keys:
target = keys.pop(0)
if isinstance(work_dict, dict):
if target in work_dict:
if not keys: # this is the last element in the input, and it is in the dict
return True
else: # not the last element of subkey, change the temp var
work_dict = work_dict[target]
else:
return False
else:
return False
The structure that is checked is in the form parent_key:sub_key1:sub_sub_key2, where the : char is the delimiter. Obviously - it will match case-sensitively, and will stop (return False) if there's a list within the dictionary.
Sample usage:
dct = {'a': {'b': {'c': {'d': 123}}}}
print(subkey_in_dict(dct, 'a:b:c:d')) # prints True
print(subkey_in_dict(dct, 'a:b:c:d:e')) # False
print(subkey_in_dict(dct, 'a:b:d')) # False
print(subkey_in_dict(dct, 'a:b:c')) # True
This is what I usually use
def key_in_dict(_dict: dict, key_lookup: str, separator='.'):
"""
Searches for a nested key in a dictionary and returns its value, or None if nothing was found.
key_lookup must be a string where each key is deparated by a given "separator" character, which by default is a dot
"""
keys = key_lookup.split(separator)
subdict = _dict
for k in keys:
subdict = subdict[k] if k in subdict else None
if subdict is None: break
return subdict
Returns the key if exists, or None it it doesn't
key_in_dict({'test': {'test': 'found'}}, 'test.test') // 'found'
key_in_dict({'test': {'test': 'found'}}, 'test.not_a_key') // None