I have a list of strings, from which I have to construct a dict. So, for example, I have:
foo.bar:10
foo.hello.world:30
xyz.abc:40
pqr:100
This is represented as a dict:
{
"foo": {
"bar": 10,
"hello": {
"world": 30
}
},
"xyz": {
"abc": 40
},
"pqr": 100
}
This question is based on the same premise, but the answers discuss hardcoded depths such as:
mydict = ...
mydict['foo']['bar'] = 30
Since the dot seperated strings on the left may be of any depth, I can't figure out a way to build the dict. How should I parse the dot separated string and build the dict?
Building upon the solution in the links, you could
iterate over each line
for each line, extract a list of keys, and its value
recurse into a dictionary with each key using setdefault
assign the value at the bottom
lines = \
'''
foo.bar:10
foo.hello.world:30
xyz.abc:40
pqr:100
'''.splitlines()
d = {}
for l in lines:
k, v = l.split(':')
*f, l = k.split('.')
t = d
for k in f:
t = t.setdefault(k, {})
t[l] = int(v) # don't perform a conversion if your values aren't numeric
print(d)
{
"pqr": 100,
"foo": {
"bar": 10,
"hello": {
"world": 30
}
},
"xyz": {
"abc": 40
}
}
Recursive setdefault traversal learned from here.
Breaking down each step -
Split on :, extract the key-list string and the value
k, v = l.split(':')
Split the key-string on . to get a list of keys. I take the opportunity to partition the keys as well, so I have a separate reference to the last key that will be the key to v.
*f, l = k.split('.')
*f is the catch-all assignment, and f is a list of any number of values (possibly 0 values, if there's only one key in the key-string!)
For each key k in the key list f, recurse down into the "tree" using setdefault. This is similar to recursively traversing a linked list.
for k in f:
t = t.setdefault(k, {})
At the end, the last key value pair comes from l and v.
t[l] = v
What's wrong with incrementally building it?
mydict = {}
mydict["foo"] = {}
mydict["foo"]["bar"] = 30
mydict["foo"]["hello"] = {}
mydict["foo"]["hello"]["world"] = 30
mydict["foo"]["xyz"] = {}
mydict["foo"]["xyz"]["abc"] = 40
mydict["foo"]["pqr"] = 100
# ...
pprint.pprint(mydict) # {'foo': {'bar': 30, 'hello': {'world': 30}, 'pqr': 100, 'xyz': {'abc': 40}}}
Including the parsing, you could use something like this:
import pprint
inp = """foo.bar:10
foo.hello.world:30
xyz.abc:40
pqr:100
"""
mydict = {}
for line in inp.splitlines():
s, v = line.split(':')
parts = s.split(".")
d = mydict
for i in parts[:-1]:
if i not in d:
d[i] = {}
d = d[i]
d[parts[-1]] = v
pprint.pprint(mydict) # {'foo': {'bar': '10', 'hello': {'world': 30'}}, 'pqr': '100', 'xyz': {'abc': '40'}}
One key point to consider in your case is that you either want to create a dictionary in a parent's dictionarys value part or an integer
x = """
foo.bar:10
foo.hello.world:30
xyz.abc:40
pqr.a:100
"""
tree = {}
for item in x.split():
level, value = item.split(":")[0], item.split(":")[1]
t = tree
for part in item.split('.'):
keyval = part.split(":")
if len(keyval) > 1:
#integer
t = t.setdefault(keyval[0], keyval[1])
else:
t = t.setdefault(part, {})
import pprint
pprint.pprint(tree)
Result:
{'foo': {'bar': '10', 'hello': {'world': '30'}},
'pqr': {'a': '100'},
'xyz': {'abc': '40'}}
Related
I have this data:
data =
{
"a":{
"00066554466":{
"Id":650,
"Passwd":"e1c2a06545de9164d7e87cd98bed57c5",
"zone":"Europe/Zurich"
},
"8745212300":{
"Id":400,
"Passwd":"ecb95502daace7f46bf12b484d086e5b",
"zone":"Europe/Zurich"
},
"8745212301":{
"Id":401,
"Passwd":"ecb95502daace7f46bf12b484d086e5b",
"zone":"Europe/Zurich"
},
"8745212302":{
"DevId":402,
"Passwd":"ecb95502daace7f46bf12b484d086e5b",
"zone":"Europe/Zurich"
}
}
}
I would like to group keys with same Passwd. So result should be like the following.
{
"e1c2a06545de9164d7e87cd98bed57c5":[
"00066554466"
],
"ecb95502daace7f46bf12b484d086e5b":[
"8745212300",
"8745212301",
"8745212302"
]
}
I tried with itertools.groupby and with for k,v in xxx, but the result is never what I need.
itertools.groupby works well when then data is already sorted with values to be grouped in a successive order, which might not always be the case with your data.
Rather use dict.setdefault and a nested loop:
out = {}
for d1 in data.values():
for k, d2 in d1.items():
out.setdefault(d2['Passwd'], []).append(k)
print(out)
Variant with a defaultdict:
from collections import defaultdict
out = defaultdict(list)
for d1 in data.values():
for k, d2 in d1.items():
out[d2['Passwd']].append(k)
print(dict(out))
Output:
{'e1c2a06545de9164d7e87cd98bed57c5': ['00066554466'],
'ecb95502daace7f46bf12b484d086e5b': ['8745212300', '8745212301', '8745212302']}
One solution, but probably not the pythonic is just to do:
passwd_group = {}
for k, val in data["a"]:
if val["Passwd"] not in passwd_group:
passwd_group[val["Passwd"]] = []
passwd_group.append(k)
This may not be ideal but got it working.
new_dict = {}
for L in data.keys():
x = data[L]
for M in x.keys():
y = x[M]
for N in y.keys():
if N == "Passwd":
new_list = new_dict.get(y[N], [])
new_list.append(M)
new_dict[y[N]] = new_list
print(new_dict)
I'm trying to create a multiple hierarchy of nested dictionary. The hierarchy levels are separated with a dot(.) in variable B however the final key (A) and value (D) are fixed.
Variables
A = "key"
B = "one.two.three.four"
D = "value"
Desired Output
{ one : { two : {three : {four : {key: value}}}}}
Here, the length of hierarchy (variable B) might increase or decrease based on input. I'm unable to create such dynamic code.
My pseudocode Code
A = "key"
B = "one.two.three.four"
D = "value"
inner_dictionary = {}
whole_dictionary = {}
lst = B.split('.')
length = len(lst)
for i in range(length):
new = lst[-1]
tmp = {A:D}
inner_dictionary.update(tmp)
val = { new : inner_dictionary}
whole_dictionary.update(val)
lst.pop()
print(whole_dictionary)
My Output
{'four': {'key': 'value'}, 'three': {'key': 'value'}, 'two': {'key': 'value'}, 'one': {'key': 'value'}}
I need help on this. Thanks in advance!
Use this:
A = "key"
B = "one.two.three.four"
D = "value"
x = {A: D}
for k in B.split('.')[::-1]:
x = {k: x}
print(x)
Output:
{'one': {'two': {'three': {'four': {'key': 'value'}}}}}
Or, in Python 3.8+, using the walrus operator:
A = "key"
B = "one.two.three.four"
D = "value"
x = {A: D}
[(x := {k: x}) for k in B.split('.')[::-1]]
print(x)
Output:
{'one': {'two': {'three': {'four': {'key': 'value'}}}}}
Note: the second solution takes a lot more time (you can run the following code to check that):
from timeit import timeit
print(timeit("""A = "key"
B = "one.two.three.four"
D = "value"
x = {A: D}
for k in B.split('.')[::-1]:
x = {k: x}"""))
print(timeit("""A = "key"
B = "one.two.three.four"
D = "value"
x = {A: D}
[(x := {k: x}) for k in B.split('.')[::-1]]"""))
The first one takes about 0.5s, the second one about 1s.
One approach using a single for-loop:
A = "key"
B = "one.two.three.four"
D = "value"
start = { A: D }
for k in reversed(B.split(".")):
start = { k : start }
print(start)
Output
{'one': {'two': {'three': {'four': {'key': 'value'}}}}}
This type of problem where a list of values aggregates to a single one can be solved using reduce (a la functional programming):
from functools import reduce
A = "key"
B = "one.two.three.four"
D = "value"
res = reduce(lambda x, y: {y: x}, reversed(B.split(".")), {A: D})
I have a dictionary of key-value pairs, where the value is itself a dictionary. I would like to change the names of key values in that nested dictionary based on a predefined conversion.
I am using two lists to match up the values of the nested keys that I am trying to conver (Old_item1 should become New_item1):
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
old_dict = {
'Company1':
{
'Old_item1':
{
'key1': val,
'key2': val
},
'Old_item2':
{
'key1': val,
'key2': val
}
}
}
I tried this:
new_dict = {}
for i in comparison_list:
for j in new_prods:
new_dict['Company1'][j] = test['Company1'][i]
I get a KeyError: KeyError: 'Company1'
The desired output for each item I add to each list is:
new_dict = {
'Company1':
{
'New_item1':
{
'key1': val # old item key, val
'key2': val # old item key, val
}
}
}
You can make a dictionary of mappings from the old to the new items, and then use it to create a new sub-dictionary for each company. This then needs to be wrapped inside an outer loop over companies (although here there is only one).
For example:
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
old_dict = {'Company1':
{'Old_item1':
{'key1': 2,
'key2': 3},
'Old_item2':
{'key1': 4,
'key2': 5}}}
key_mappings = dict(zip(comparison_list, new_prods))
new_dict = {k: {key_mappings[k1]: v1 for k1, v1 in v.items()}
for k, v in old_dict.items()}
print(new_dict)
gives:
{'Company1': {'New_item1': {'key1': 2, 'key2': 3}, 'New_item2': {'key1': 4, 'key2': 5}}}
Here is the mappings dictionary key_mappings which we used:
{'Old_item1': 'New_item1', 'Old_item2': 'New_item2'}
The easiest way to go about this is to re-create your dictionary with a comprehension:
new_dict = {company: update_item(v) for company, v in old_dict.items()}
Then parse out update_item to it's own function. You could do this inline but it makes it difficult to understand.
conversion_lookup = {
'Old_item1': 'NewItem1',
'Old_item2': 'NewItem2',
}
def update_item(item: dict) -> dict:
return { conversion_lookup.get(k, k): v for k, v in item.items() }
new_dict = {company: update_item(v) for company, v in old_dict.items()}
For the conversion here I'm using a dictionary describing the conversion. If you need to construct this automatedly:
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
conversion_lookup = { v: new_prods[idx] for idx, v in enumerate(comparison_list) }
The reason I like a dictionary is that you can use some_dict.get(a_value, a_default_value). Then, if your value isn't in your conversion dictionary you can fall back to the original value. That's what I'm doing here:
conversion_lookup.get(k, k)
The second k is the original item's value, which is a good thing to use if your conversion list doesn't include what you want.
One approach is to do it step by step. First create the dicts with the new names and old keys, then remove the old keys, and finally add in the new names.
# Keep the values from the old keys
values = [old_dict['Company1'][old_name] for old_name in comparison_list]
# Remove old names, and add in the
for new_key, value, old in zip(new_prods, values, comparison_list):
old_dict['Company1'].pop(old)
old_dict['Company1'][new_key] = value
This question already has answers here:
Access nested dictionary items via a list of keys?
(20 answers)
Closed 4 years ago.
Let's say i have a list of keys
key_lst = ["key1", "key2", "key3"]
and i have a value
value = "my_value"
and an example dict my_dict with this structure
{
"key1": {
"key2": {
"key3": "some_value"
}
},
}
How can I dynamically assign the new value in variable value to my_dict["key1"]["key2"]["key3"] by going thru / looping over my key_lst?
I can not just say my_dict["key1"]["key2"]["key3"] = value since the keys and the number of keys is changing. I always get the keys (the path that i have to save the value at) in a list...
The output I am looking for is {'key1': {'key2': {'key3': 'my_value'}}}. The dictionary structure is predefined.
I'm using Python 3.7
Predefined dictionary structure: functools.reduce
You can define a function using functools.reduce to apply getitem repeatedly and then set a supplied value:
from functools import reduce
from operator import getitem
def set_nested_item(dataDict, mapList, val):
"""Set item in nested dictionary"""
reduce(getitem, mapList[:-1], dataDict)[mapList[-1]] = val
return dataDict
key_lst = ["key1", "key2", "key3"]
value = "my_value"
d = {"key1": {"key2": {"key3": "some_value"}}}
d = set_nested_item(d, key_lst, value)
print(d)
# {'key1': {'key2': {'key3': 'my_value'}}}
Note operator.getitem is used to access dict.__getitem__, or its more commonly used syntactic sugar dict[]. In this instance, functools.reduce calls getitem recursively on dataDict, successively using each value in mapList[:-1] as an argument. With [:-1], we intentionally leave out the last value, so we can use __setitem__ via dict[key] = value for the final key.
Arbitrary dictionary nesting: collections.defaultdict
If you wish to add items at arbitrary branches not yet been defined, you can construct a defaultdict. For this, you can first defaultify your regular dictionary input, then use set_nested_item as before:
from collections import defaultdict
def dd_rec():
return defaultdict(dd_rec)
def defaultify(d):
if not isinstance(d, dict):
return d
return defaultdict(dd_rec, {k: defaultify(v) for k, v in d.items()})
dd = defaultify(d)
key_lst = ["key1", "key2", "key5", "key6"]
value = "my_value2"
dd = set_nested_item(dd, key_lst, value)
print(dd)
# defaultdict(<function __main__.<lambda>>,
# {'key1': defaultdict(<function __main__.<lambda>>,
# {'key2': defaultdict(<function __main__.<lambda>>,
# {'key3': 'my_value',
# 'key5': defaultdict(<function __main__.<lambda>>,
# {'key6': 'my_value2'})})})})
You can iteratively build/access levels using setdefault in a loop:
d = {}
d2 = d
for k in key_lst[:-1]:
d2 = d2.setdefault(k, {})
d2[key_lst[-1]] = value
print(d)
# {'key1': {'key2': {'key3': 'my_value'}}}
d is the reference to your dictionary, and d2 is a throw-away reference that accesses inner levels at each iteration.
This is what you want:
def update(d, key_lst , val):
for k in key_lst[:-1]:
if k not in d:
d[k] = {}
d = d[k]
d[key_lst[-1]] = val
d = {}
update(d, list('qwer'), 0)
# d = {'q': {'w': {'e': {'r': 0}}}}
You could use defaultdict too, it's neat in a sense but prints rather ugly...:
from collections import defaultdict
nest = lambda: defaultdict(nest)
d = nest()
def update(d, key_lst , val):
for k in key_lst[:-1]:
d = d[k]
d[key_lst[-1]] = val
update(d, 'qwer', 0)
I guess you can loop through your keys like this :
d = {}
a = d
for i in key_lst:
a[i] = {}
if i == key_lst[-1]:
a[i] = value
else:
a = a[i]
print(d)
# {'key1': {'key2': {'key3': 'my_value'}}}
Edit: I guess I misread the question and answered as if the dictionnary wasn't already existing. jpp answer is pretty neat otherwise I guess!
key_lst = ["key1", "key2", "key3"]
my_dict={
"key1": {
"key2": {
"key3": "some_value"
}
},
}
val=my_dict
#loop gets second to last key in chain(path) and assigns it to val
for x in key_lst[:-1]:
val=val[x]
#now we can update value of last key, cause dictionary key is passed by reference
val[key_lst[-1]]="new value"
print (my_dict)
#{'key1': {'key2': {'key3': 'new value'}}}
I'm looking for an 'efficient' way to iterate through a dictionary, and replace any key or value that starts with the term 'var'.
For example, if I have this:
data = {
"user_id": "{{var_user_id}}",
"name": "bob",
"{{var_key_name}}": 4
}
and I have this dict of variable values:
variables = {
"user_id": 10,
"key_name": "orders_count"
}
Then I'd like my final data dict to look like this:
data = {
"user_id": 10,
"name": "bob",
"orders_count": 4
}
Since you're treating it like a text template language (and if you are, then why not make it string.format(**variable) compatible syntax?) use text replacement:
import ast
import re
text = re.sub('{{var_(.*?)}}', lambda m: variables[m.groups()[0]], str(data))
data2 = ast.literal_eval(text)
print(data2)
In straight-forward way:
result = {}
for k,v in data.items():
if '{{var_' in k: # if `{{var..}}` placeholder is in key
result[variables[k[6:-2]]] = v
elif '{{var_' in v: # if `{{var..}}` placeholder is in value
result[k] = variables[v[6:-2]]
else:
result[k] = v
print(result)
The output:
{'user_id': 10, 'orders_count': 4, 'name': 'bob'}
This is a pretty manual algorithm, but here goes:
for key, value in data.iteritems():
if "var" in str(key):
#iterate through "variables" to find the match
elif "var" in str(value):
#iterate through "variables" to find the match
#populate "data" with the key value pair
This will work, but it's kind of messy if you can't guarantee uniqueness, especially in the case where a key needs to be replaced.