I am giving the user the ability to check a specific key in a multi-level dictionary. My idea is that they will pass the path to the key like this:
root.subelement1.subelement2.key
This can be of arbitrary length and depth.
Once I have the string (above) from the user, I'll split it and get a list of each individual component:
elements = ['root', 'subelement1', 'subelement2', 'key']
All of this I can do. The next part is where I am stuck. How can I query the dictionary key, specified by the above when it's arbitrary length?
My initial thought was to do something like my_dict[elements[0]][elements[1]]...but that doesn't scale or work when my user doesn't pass exactly the length I expect.
How can I get the data at an arbitrary key depth, in this case?
A couple examples:
User passes country.US.NewYork => I query `my_dict['country']['US']['NewYork']
User passes department.accounting => I query my_dict['department']['accounting']
User passes id => I query my_dict['id']
User passes district.District15.HenryBristow.principal => I query my_dict['district']['District15']['HenryBristow']['principal']
you could do that using reduce which will query the keys in the nested dictionaries:
q = "district.District15.HenryBristow.principal"
my_dict = {"district" : {"District15" : {"HenryBristow" : {"principal" : 12}}}}
from functools import reduce # python 3 only
print(reduce(lambda x,y : x[y],q.split("."),my_dict))
result:
12
If you want to avoid to catch KeyError in case the data doesn't exist with this path, you could use get with a default value as empty dictionary:
reduce(lambda x,y : x.get(y,{}),q.split("."),my_dict)
Trying to get an unknown value returns an empty dictionary. The only drawback is that you don't know from where exactly the path got lost, so maybe leaving the KeyError be raised wouldn't be so bad:
try:
v = reduce(lambda x,y : x[y],q.split("."),my_dict)
except KeyError as e:
print("Missing key: {} in path {}".format(e,q))
v = None
Use recursion. Ex:
root = {
'subelement1': {
'subelement2': {
'key': 'value'
}
}
}
elements = ['subelement1', 'subelement2', 'key']
def getElem(d, keys):
if keys == []:
return None
else:
key = keys[0]
remainingKeys = keys[1:]
if remainingKeys == []:
return d[key]
else:
if type(d[key]) == dict:
return getElem(d[key], remainingKeys)
else:
return None
print(getElem(root, elements))
from a python 2.x perspective, you can do this with reduce.
query_list = keys.split(":")
print reduce(lambda x,y: x[y], [my_dict] + query_list)
But in general, you'll want to do this with a recursive or iterative function if you want to do error handling beyond throwing a KeyError.
You can transverse the dictionary using a for loop:
s = 'root.subelement1.subelement2.key'
d1 = {'root':{'subelement1':{'subelement2':{'key':15, 'key1':18}}}}
new_d = d1
for key in s.split('.'):
new_d = new_d[key]
print(new_d)
Output:
15
u can do this like below
my_dict = someDict
tmpDict = dict(someDict) # a coppy of dict
input = "x.u.z"
array = input.split(".")
for key in array:
tmpDict = tmpDict[key]
print(tmpDict)
but your question is very challenging:
u say if user send country.us then go to my-dict.country.us
but what happen if one of this path in my_dict be a list code will results error
u can handle this by check type
if isinstance(tmpDict , dict ):
tmpDict = tmpDict[key]
else:
# u should say what u want else (a Recursive method u will need)
edit
if user address maybe wrong you should check my_dict have this field or not sample code is below but will be many if i don't like that!
if key not in tmpDict:
print("Bad Path")
return
Related
items={
"fruits":
{
"summerFruits":
{
"Mangoes":5,
"melon":2
}
}
}
i converted it to attribute
itemConverted = AttrDict(items)
now i know i can access this by
itemConverted.fruits.summerFruits.Mangoes
but the problem is , i am taking inputs from console as a string so
it will be like
wanted="fruits.summerFruits.Mangoes"
i am trying to get it by
itemConverted.wanted
but it is not working , any suggestions
Get the dictionary keys from the string and then use the dictionary items to recover the value.
items={"fruits":{"summerFruits": {"Mangoes":5, "melon":2}}}
def get_val_from_str(string, dct):
keys = string.split('.')
v = dct
for key in keys:
v = v[key]
return v
console_input = "fruits.summerFruits.Mangoes"
get_val_from_str(console_input, items)
#5
I could not think of a simple solution with the dictionnary converted through AttrDict. However, here is a basic workaround that works with your input :
items = {"fruits":{"summerFruits":{"Mangoes":5,"melon":2}}}
wanted = "fruits.summerFruits.Mangoes"
#Converts the input to a list of strings (previously separated by dots)
words_list = []
element = ''
for char in wanted:
if char == '.': #chosen separator for input
words_list.append(element)
element = ''
else:
element = element + char
words_list.append(element)
#From the dict, get the value at the position indicated by the list of string keys
temp_dict = items
for key in words_list:
temp_dict = temp_dict.get(key)
value = temp_dict
Hoping this would work for your application, even though the syntax for getting the output is slightly different.
Whilst trying to run the following code:
temp3 = {
('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD','EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP'):'EURIBOR_EUR_1Y'
}
print (temp3.get('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD'))
Output:
None
Expected:
EURIBOR_EUR_1Y
You're expecting the get function to unpack the key values and search inside the tuples; it doesn't work that way.
The correct way is to use the key that you used to create the dict.
Example:
temp3 = {
('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD',
'EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP'):'EURIBOR_EUR_1Y'
}
print(temp3.get(('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD',
'EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP')))
Which will output the key; to get all values that have the key that matches you can use the following:
def key_search(needle, haystack):
matches = []
for key, value in haystack.items():
if type(key) in [list, tuple, dict] and needle in key:
matches.append(value)
elif needle == key:
matches.append(value)
return matches
data = {
(1, 2, 3): 'heyyy',
(2, 1): 'heyyy there'
}
print(key_search(1, data))
Output
['heyyy there', 'heyyy']
Where needle is the key you're looking for, and the haystack is your data.
Actually the correct key is ('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD','EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP')
So if you'll try:
print(temp3.get(('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD','EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP')))
You'll get:
EURIBOR_EUR_1Y
You are not using the complete key for the dictionary.
Try with complete key
(temp3.get('EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD','EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP'))
'EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-GBP'
This is what you are looking
print(temp3.get([i for i in temp3 if 'EUR-EURIBOR-1Y-SWAPTION-PHYSICAL-ANNUAL-BOND-QUANTO-USD' in i][0]))
Output:
EURIBOR_EUR_1Y
I am looking to learn how to pass certain keys/values in a dictionary to another function within a for loop. The "certain" keys all share the same initial string and are incremented by a trailing integer like this:
data = {}
data["HMD1"] = [a,b,c]
data["HMD2"] = [d,f,g] #and so on...
There are other keys with dissimilar names witin the same dictionary. Now within a for loop I would like to pass the values for each key that starts with "HMD" to another function. Here is a minimal working example of a failed attempt:
data = {}
data["HMD1"] = [0,2,3]
data["HMD2"] = [5,6,4]
data["not"] = 1237659398
data["HMD3"] = [1,1,1]
def dummyfun(vargin):
print(vargin)
return vargin
for f in range(1,2,1):
out = dummyfun(data[eval(''.join(("HMD",str(f))))])
This was a poor guess, of course it returns an error because eval() tries to evaluate "HMD1" which is not a variable but a key in data. Does anyone know how to do this properly?
You don't need eval at all for this. You only need to build the string with .format for example
for f in range(1,4): #range don't include the end point
out = dummyfun(data["HMD{}".format(f)])
with this you get the desire result. But that will fail if the key is not in the dict, you can check it first, catch the exception or provide a default value in case the desire key is not there
#check first
for f in range(1,4):
key = "HMD{}".format(f)
if key in data:
out = dummyfun(data[key])
#catch the exception
for f in range(1,4):
try:
out = dummyfun(data["HMD{}".format(f)])
except KeyError:
print("key",f,"is not in the data")
#provide a default value
for f in range(1,4):
out = dummyfun(data.get("HMD{}".format(f),None))
Just iterate through the dictionary using a for loop and use an if statement to check for validity of the keys:
for key in yourDict: #a for loop for dict iterates through its keys
if 'HMD' in key: #or you can replace with any other conditional
#DO WHAT YOU WANT TO DO HERE
And here's a quick working example:
>>> data = {'HMD1': [1,2,3], 'HMD23':'heyo mayo', 'HMNOT2':'if this prints, I did something wrong'}
>>> for key in data:
... if 'HMD' in key:
... print data[key]
...
[1, 2, 3]
heyo mayo
With further understand of what you want, you can also look at this backwards and create key strings and print the values that those key's point to:
#let's say you want to print HMD1, HMD2, HMD4, but not anything else
keylist = [#list of keys that you want]
for key in keylist:
if key in data:
print data[key]
and, again, a working example.
>>> data = {'HMD1': [1,2,3], 'HMD3':'heyo mayo, this shouldnt print', 'HMD4':123, 'HMD2':['g', 'h', 'i'], 'HMNOT2':'if this prints, I did something wrong'}
>>> keylist = ['HMD1', 'HMD2', 'HMD4']
>>> for key in keylist:
... if key in data:
... print data[key]
...
[1, 2, 3]
['g', 'h', 'i']
123
Suppose we have a list of dictionaries listD where each dictionary is quite deeply nested with more dictionaries. e.g.suppose that the first element of listD is:
listD[0] = {"bar1":{"bar2":{"bar3":1234}}}
Now I want to check if listD[i]["bar1"]["bar2"]["bar3"] == 1234 for all i. For the first element where i = 0, this is easy as we can just use the expression:
listD[0]["bar1"]["bar2"]["bar3"] == 1234
But I cannot simply write a loop like:
for dictelem in listD:
if dictelem["bar1"]["bar2"]["bar3"] == 1234:
print "equals 1234"
This is because some of the dictionary elements of listD might be of the form
listD[i] = {"bar1":{"bar2":"abcd"}} or
listD[i] = {"bar1":{"bar2":None}}
and if I try to access "bar3" when it doesn't exists, an error will be raised.
Right now I am manually specifying in the code to check for the existence of the bar1, bar2 and bar3 keys and whether that are in fact dictionaries or not. But this is really verbose and I'm quite sure there's a simpler way to do it but I can't figure out how.
def dictcheck(d, p, v):
if len(p):
if isinstance(d,dict) and p[0] in d:
return dictcheck(d[p[0]], p[1:], v)
else:
return d == v
You pass one dict d, one path of keys p, and the final value to check v. It will recursively go in the dicts and finally check if the last value is equal to v.
>>> dictcheck({"bar1":{"bar2":{"bar3":1234}}}, ('bar1','bar2','bar3'), 1234)
True
>>> dictcheck({"bar1":1234}, ('bar1','bar2','bar3'), 1234)
False
So, to answer your question (I want to check if listD[i]["bar1"]["bar2"]["bar3"] == 1234 for all i):
all(dictcheck(x, ('bar1','bar2','bar3'), 1234) for x in listD)
just use a try/except block this way:
for dictelem in listD:
try:
if dictelem["bar1"]["bar2"]["bar3"] == 1234:
print "equals 1234"
except TypeError:
pass
When dealing with nested dictionaries, I think of them as a tree where the keys make up the path to the value. With that in mind, I created a non-recursive function, dict_path with takes in a nested dictionary, the key path, and a value in case not found:
def dict_path(dic, path, value_if_not_found=None):
path = path.split('.')
try:
for key in path:
dic = dic[key]
return dic
except (KeyError, TypeError):
return value_if_not_found
listD = [
{"bar1": {"bar2": 'abcd'}},
{"bar1": {"bar2": None}},
{"bar1": {"bar2": {"bar3": 1234}}},
]
for dic in listD:
value = dict_path(dic, 'bar1.bar2.bar3')
if value == 1234:
print 'Equals 1234:', dic
The function keeps traversing the nested dictionary until one of these three conditions occur:
It found the value in question. In this case, return that value
Along the way, the object is a dictionary, but does not have the requested key. In this case, a KeyError is raised
The object is not a dictionary, the TypeError is raised
For case 2 and 3, we simply return value_if_not_found
you can try this
for item in listD:
if item.get("bar1",{}).get("bar2",{}).get("bar3","") == 1234:
print "yeah, gotcha"
Many SO posts show you how to efficiently check the existence of a key in a dictionary, e.g., Check if a given key already exists in a dictionary
How do I do this for a multi level key? For example, if d["a"]["b"] is a dict, how can I check if d["a"]["b"]["c"]["d"] exists without doing something horrendous like this:
if "a" in d and isInstance(d["a"], dict) and "b" in d["a"] and isInstance(d["a"]["b"], dict) and ...
Is there some syntax like
if "a"/"b"/"c"/"d" in d
What I am actually using this for: we have jsons, parsed into dicts using simplejson, that I need to extract values from. Some of these values are nested three and four levels deep; but sometimes the value doesn't exist at all. So I wanted something like:
val = None if not d["a"]["b"]["c"]["d"] else d["a"]["b"]["c"]["d"] #here d["a"]["b"] may not even exist
EDIT: prefer not to crash if some subkey exists but is not a dictionary, e.g, d["a"]["b"] = 5.
Sadly, there isn't any builtin syntax or a common library to query dictionaries like that.
However, I believe the simplest(and I think it's efficient enough) thing you can do is:
d.get("a", {}).get("b", {}).get("c")
Edit: It's not very common, but there is: https://github.com/akesterson/dpath-python
Edit 2: Examples:
>>> d = {"a": {"b": {}}}
>>> d.get("a", {}).get("b", {}).get("c")
>>> d = {"a": {}}
>>> d.get("a", {}).get("b", {}).get("c")
>>> d = {"a": {"b": {"c": 4}}}
>>> d.get("a", {}).get("b", {}).get("c")
4
This isn't probably a good idea and I wouldn't recommend using this in prod. However, if you're just doing it for learning purposes then the below might work for you.
def rget(dct, keys, default=None):
"""
>>> rget({'a': 1}, ['a'])
1
>>> rget({'a': {'b': 2}}, ['a', 'b'])
2
"""
key = keys.pop(0)
try:
elem = dct[key]
except KeyError:
return default
except TypeError:
# you gotta handle non dict types here
# beware of sequences when your keys are integers
if not keys:
return elem
return rget(elem, keys, default)
UPDATE: I ended up writing my own open-source, pippable library that allows one to do this: https://pypi.python.org/pypi/dictsearch
A non-recursive version, quite similar to #Meitham's solution, which does not mutate the looked-for key. Returns True/False if the exact structure is present in the source dictionary.
def subkey_in_dict(dct, subkey):
""" Returns True if the given subkey is present within the structure of the source dictionary, False otherwise.
The format of the subkey is parent_key:sub_key1:sub_sub_key2 (etc.) - description of the dict structure, where the
character ":" is the delemiter.
:param dct: the dictionary to be searched in.
:param subkey: the target keys structure, which should be present.
:returns Boolean: is the keys structure present in dct.
:raises AttributeError: if subkey is not a string.
"""
keys = subkey.split(':')
work_dict = dct
while keys:
target = keys.pop(0)
if isinstance(work_dict, dict):
if target in work_dict:
if not keys: # this is the last element in the input, and it is in the dict
return True
else: # not the last element of subkey, change the temp var
work_dict = work_dict[target]
else:
return False
else:
return False
The structure that is checked is in the form parent_key:sub_key1:sub_sub_key2, where the : char is the delimiter. Obviously - it will match case-sensitively, and will stop (return False) if there's a list within the dictionary.
Sample usage:
dct = {'a': {'b': {'c': {'d': 123}}}}
print(subkey_in_dict(dct, 'a:b:c:d')) # prints True
print(subkey_in_dict(dct, 'a:b:c:d:e')) # False
print(subkey_in_dict(dct, 'a:b:d')) # False
print(subkey_in_dict(dct, 'a:b:c')) # True
This is what I usually use
def key_in_dict(_dict: dict, key_lookup: str, separator='.'):
"""
Searches for a nested key in a dictionary and returns its value, or None if nothing was found.
key_lookup must be a string where each key is deparated by a given "separator" character, which by default is a dot
"""
keys = key_lookup.split(separator)
subdict = _dict
for k in keys:
subdict = subdict[k] if k in subdict else None
if subdict is None: break
return subdict
Returns the key if exists, or None it it doesn't
key_in_dict({'test': {'test': 'found'}}, 'test.test') // 'found'
key_in_dict({'test': {'test': 'found'}}, 'test.not_a_key') // None