I have a dictionary with some values that are type list, i need to convert each list in another dictionary and insert this new dictionary at the place of the list.
Basically, I have this dictionary
Dic = {
'name': 'P1',
'srcintf': 'IntA',
'dstintf': 'IntB',
'srcaddr': 'IP1',
'dstaddr': ['IP2', 'IP3', 'IP4'],
'service': ['P_9100', 'SNMP'],
'schedule' : 'always',
}
I need to reemplace the values that are lists
Expected output:
Dic = {
'name': 'P1',
'srcintf': 'IntA',
'dstintf': 'IntB',
'srcaddr': 'IP1',
'dstaddr': [
{'name': 'IP2'},
{'name': 'IP3'},
{'name': 'IP4'}
],
'service': [
{'name': 'P_9100'},
{'name': 'SNMP'}
],
'schedule' : 'always',
}
So far I have come up with this code:
for k,v in Dic.items():
if not isinstance(v, list):
NewDic = [k,v]
print(NewDic)
else:
values = v
keys = ["name"]*len(values)
for item in range(len(values)):
key = keys[item]
value = values[item]
SmallDic = {key : value}
liste.append(SmallDic)
NewDic = [k,liste]
which print this
['name', 'P1']
['srcintf', 'IntA']
['dstintf', 'IntB']
['srcaddr', 'IP1']
['schedule', 'always']
['schedule', 'always']
I think is a problem with the loop for, but so far I haven't been able to figure it out.
You need to re-create the dictionary. With some modifications to your existing code so that it generates a new dictionary & fixing the else clause:
NewDic = {}
for k, v in Dic.items():
if not isinstance(v, list):
NewDic[k] = v
else:
NewDic[k] = [
{"name": e} for e in v # loop through the list values & generate a dict for each
]
print(NewDic)
Result:
{'name': 'P1', 'srcintf': 'IntA', 'dstintf': 'IntB', 'srcaddr': 'IP1', 'dstaddr': [{'name': 'IP2'}, {'name': 'IP3'}, {'name': 'IP4'}], 'service': [{'name': 'P_9100'}, {'name': 'SNMP'}], 'schedule': 'always'}
Related
I have a dictionary with missing values (the key is there, but the associated value is empty). For example I want the dictionary below:
dct = {'ID': '', 'gender': 'male', 'age': '20', 'weight': '', 'height': '5.7'}
to be changed to this form:
dct = {'ID': {'link': '','value': ''}, 'gender': 'male', 'age': '20', 'weight': {'link': '','value': ''}, 'height': '5.7'}
I want the ID and Weight key should be replaced with nested dictionary if its empty.
How can I write that in the most time-efficient way?
I have tried solutions from below links but didnt work,
def update(orignal, addition):
for k, v in addition.items():
if k not in orignal:
orignal[k] = v
else:
if isinstance(v, dict):
update(orignal[k], v)
elif isinstance(v, list):
for i in range(len(v)):
update(orignal[k][i], v[i])
else:
if not orignal[k]:
orignal[k] = v
Error: TypeError: 'str' object does not support item assignment
Fill missing keys by comparing example json in python
Adding missing keys in dictionary in Python
It seems similar with this issue https://stackoverflow.com/a/3233356/6396981
import collections.abc
def update(d, u):
for k, v in u.items():
if isinstance(v, collections.abc.Mapping):
d[k] = update(d.get(k, {}) or {}, v)
else:
d[k] = v
return d
For example in your case:
>>> dict1 = {'ID':'', 'gender':'male', 'age':'20', 'weight':'', 'height':'5.7'}
>>> dict2 = {'ID': {'link':'','value':''}, 'weight': {'link':'','value':''}}
>>>
>>> update(dict1, dict2)
{'ID': {'link': '', 'value': ''}, 'gender': 'male', 'age': '20', 'weight': {'link': '', 'value': ''}, 'height': '5.7'}
>>>
You can iterate through the list and see if the value is an empty string('') if it is, replace it with the default value. Here's a small snippet which does it -
dct = {'ID':'', 'gender':'male', 'age':'20', 'weight':'', 'height':'5.7'}
def update(d, default):
for k, v in d.items():
if v == '':
d[k] = default.copy()
update(dct, {'link':'','value':''})
print(dct)
Output :
{'ID': {'link': '', 'value': ''}, 'gender': 'male', 'age': '20', 'weight': {'link': '', 'value': ''}, 'height': '5.7'}
Note that the dict is passed by reference to the function, so any updates made there will be reflected in the original dictionary as well as seen in the above example.
If your dict is nested and you want the replacement to be done for nested items as well then you can use this function -
def nested_update(d, default):
for k, v in d.items():
if v == '':
d[k] = default.copy()
if isinstance(v, list):
for item in v:
nested_update(item, default)
if isinstance(v, dict):
nested_update(v, default)
here's a small example with list of dictionaries and nested dictionary -
dct = {'ID':'', 'gender':'male', 'age':'20', 'weight':'', 'height':'5.7', "list_data":[{'empty': ''}, {'non-empty': 'value'}], "nested_dict": {"key1": "val1", "missing_nested": ""}}
nested_update(dct, {'key1': 'val1-added', 'key2': 'val2-added'})
print(dct)
Output :
{'ID': {'key1': 'val1-added', 'key2': 'val2-added'}, 'gender': 'male', 'age': '20', 'weight': {'key1': 'val1-added', 'key2': 'val2-added'}, 'height': '5.7', 'list_data': [{'empty': {'key1': 'val1-added', 'key2': 'val2-added'}}, {'non-empty': 'value'}], 'nested_dict': {'key1': 'val1', 'missing_nested': {'key1': 'val1-added', 'key2': 'val2-added'}}}
For "this default dictionary to only specified keys like ID and Weight and not for other keys", you can update the condition of when we replace the value -
def nested_update(d, default):
for k, v in d.items():
if k in ('ID', 'weight') and v == '':
d[k] = default.copy()
if isinstance(v, list):
for item in v:
nested_update(item, default)
if isinstance(v, dict):
nested_update(v, default)
I have a list of dictionaries:
mydict = [
{'name': 'test1', 'value': '1_1'},
{'name': 'test2', 'value': '2_1'},
{'name': 'test1', 'value': '1_2'},
{'name': 'test1', 'value': '1_3'},
{'name': 'test3', 'value': '3_1'},
{'name': 'test4', 'value': '4_1'},
{'name': 'test4', 'value': '4_2'},
]
I would like to use it to create a dictionary where the values are lists or single values depending of number of their occurrences in the list above.
Expected output:
outputdict = {
'test1': ['1_1', '1_2', '1_3'],
'test2': '2_1',
'test3': '3_1',
'test4': ['4_1', '4_2'],
}
I tried to do it the way below but it always returns a list, even when there is just one value element.
outputdict = {}
outputdict.setdefault(mydict.get('name'), []).append(mydict.get('value'))
The current output is:
outputdict = {
'test1': ['1_1', '1_2', '1_3'],
'test2': ['2_1'],
'test3': ['3_1'],
'test4': ['4_1', '4_2'],
}
Do what you have already done, and then convert single-element lists afterwards:
outputdict = {
name: (value if len(value) > 1 else value[0])
for name, value in outputdict.items()
}
You can use a couple of the built-in functions mainly itertools.groupby:
from itertools import groupby
from operator import itemgetter
mydict = [
{'name': 'test1', 'value': '1_1'},
{'name': 'test2', 'value': '2_1'},
{'name': 'test1', 'value': '1_2'},
{'name': 'test1', 'value': '1_3'},
{'name': 'test3', 'value': '3_1'},
{'name': 'test4', 'value': '4_1'},
{'name': 'test4', 'value': '4_2'},
]
def keyFunc(x):
return x['name']
outputdict = {}
# groupby groups all the items that matches the returned value from keyFunc
# in our case it will use the names
for name, groups in groupby(mydict, keyFunc):
# groups will contains an iterator of all the items that have the matched name
values = list(map(itemgetter('value'), groups))
if len(values) == 1:
outputdict[name] = values[0]
else:
outputdict[name] = values
print(outputdict)
I have a rather deep dict that I need to simplify. And I've encountered some problems by doing that.
Here is a small sample of the dictionary that needs to be simplified:
data_dict = {
"DATA": {
"Page1": [{
"Section": [{
"Name": [{
"text": "John"
}],
"ID_Number": [{
"text": "123456"
}]
}]
}],
"Page2": [{
"Section": [{
"Name": [{
"text": "Rob"
}],
"ID_Number": [{
"text": "654321"
}]
}]
}]
}
}
What I've done already:
my_dict = {}
for value in data_dict.values():
for key, val in value.items():
if "Tab" in key:
my_dict[key] = val
if type(val) == list:
for i in val:
for key1, val1 in i.items():
my_dict[key] = val1
result_dict = {}
page_list = []
for keys, values in my_dict.items():
for val in values:
if type(val) != str:
for key1, val1 in val.items():
for x in val1:
result_dict[key1] = x.get('text')
page_list.append(result_dict)
my_dict[keys] = page_list
print("my_dict = ", my_dict)
Current result:
my_dict = {'Page1': [{'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '65432
1'}, {'Name': 'Rob', 'ID_Number': '654321'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '
654321'}, {'Name': 'Rob', 'ID_Number': '654321'}, {'Name': 'Rob', 'ID_Number': '654321'}]}
The problem is that result_dict is being appended to page_list more than once which is unnecessary. Also, my approach is very messy. Is there a cleaner way to get the same result?
Desired result:
my_dict = {"Page1": [{"Name": "John", "ID_Number": "123456"}], "Page2": [{"Name": "Rob", "ID_Number": "654321"}]}
Solution 1 (less loops, but added if statements):
If you want to avoid too many nested for loops. I would take advantage of knowing before-hand the duplicate keys and use that information to easily get to the inner keys or values.
Reference to dict for solution 1 & 2:
data_dict = {"DATA": {"Page1": [{"Section": [{"Name": [{"text": "John"}],"ID_Number": [{"text": "123456"}]}]}],"Page2": [{"Section": [{"Name": [{"text": "Rob"}],"ID_Number": [{"text": "654321"}]}]}]}}
Code:
# Depth #1
old_dict = data_dict["DATA"]
new_dict = {}
for d1_key in old_dict:
d2 = old_dict[d1_key][0]["Section"][0]
for d2_key in d2:
if d2_key == "Name":
new_dict[d1_key] = [{d2_key: d2[d2_key][0]["text"]}]
if d2_key == "ID_Number":
merge = new_dict[d1_key][0]
# Merge above if statement (dict merging)
new_dict[d1_key] = [{**merge, **{d2_key:d2[d2_key][0]["text"]}}]
print(new_dict)
Output:
{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}
Solution 2: (more for loops, more readible)
(Recommended)
Here is a second solution that gives the same desired output that does not take advantage of information about the keys or values, but only looks at the structure of the data. I prefer this one as it is easy to read, modify or extend!
Code:
# Depth #1
old_dict = data_dict["DATA"]
new_dict = {}
unlist = 0
k3_temp = None # instead of merge
v4_temp = None
for k1, v1 in old_dict.items():
for v2 in v1[unlist].values(): # using values because we don't use the Section key
for k3, v3 in v2[unlist].items():
for k4, v4 in v3[unlist].items():
new_dict[k1] = [{k3_temp:v4_temp, k3:v4}]
k3_temp = k3
v4_temp = v4
print(new_dict)
Output:
{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}
Just to see another solution with a ridiculous amount of for loops:
new_dic = {}
inner_list = []
for i in data_dict:
for j in data_dict[i]:
for k in data_dict[i][j]:
for m in k:
for n in k[m]:
for x in n:
for y in n[x]:
for keys, values in y.items():
inner_list.append(values)
new_dic[j] = [{'Name': inner_list[0], 'ID_Number': inner_list[1]}]
inner_list = []
print(new_dic)
output
{'Page1': [{'Name': 'John', 'ID_Number': '123456'}], 'Page2': [{'Name': 'Rob', 'ID_Number': '654321'}]}
I have a list of dictionaries, themselves with nested lists of dictionaries. All of the nest levels have a similar structure, thankfully. I desire to sort these nested lists of dictionaries. I grasp the technique to sort a list of dictionaries by value. I'm struggling with the recursion that will sort the inner lists.
def reorder(l, sort_by):
# I have been trying to add a recursion here
# so that the function calls itself for each
# nested group of "children". So far, fail
return sorted(l, key=lambda k: k[sort_by])
l = [
{ 'name': 'steve',
'children': [
{ 'name': 'sam',
'children': [
{'name': 'sally'},
{'name': 'sabrina'}
]
},
{'name': 'sydney'},
{'name': 'sal'}
]
},
{ 'name': 'fred',
'children': [
{'name': 'fritz'},
{'name': 'frank'}
]
}
]
print(reorder(l, 'name'))
def reorder(l, sort_by):
l = sorted(l, key=lambda x: x[sort_by])
for item in l:
if "children" in item:
item["children"] = reorder(item["children"], sort_by)
return l
Since you state "I grasp the technique to sort a list of dictionaries by value" I will post some code for recursively gathering data from another SO post I made, and leave it to you to implement your sorting technique. The code:
myjson = {
'transportation': 'car',
'address': {
'driveway': 'yes',
'home_address': {
'state': 'TX',
'city': 'Houston'}
},
'work_address': {
'state': 'TX',
'city': 'Sugarland',
'location': 'office-tower',
'salary': 30000}
}
def get_keys(some_dictionary, parent=None):
for key, value in some_dictionary.items():
if '{}.{}'.format(parent, key) not in my_list:
my_list.append('{}.{}'.format(parent, key))
if isinstance(value, dict):
get_keys(value, parent='{}.{}'.format(parent, key))
else:
pass
my_list = []
get_keys(myjson, parent='myjson')
print(my_list)
Is intended to retrieve all keys recursively from the json file. It outputs:
['myjson.address',
'myjson.address.home_address',
'myjson.address.home_address.state',
'myjson.address.home_address.city',
'myjson.address.driveway',
'myjson.transportation',
'myjson.work_address',
'myjson.work_address.state',
'myjson.work_address.salary',
'myjson.work_address.location',
'myjson.work_address.city']
The main thing to note is that if isinstance(value, dict): results in get_keys() being called again, hence the recursive capabilities of it (but only for nested dictionaries in this case).
I have a list of dictionaries in python and I would like to override old value with duplicate value. Please let me know how can I do.
{'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
Output should be:
{'message': [ {'name': 'raghav', 'id': 11}]}
I don't know what you mean by "override old value with duplicate value". If you mean just picking the second dict from the list, you could:
print({k: [v[1]] for (k, v) in data.items()})
If the idea is to update the "name" with a newer value of "id" as you move along the list, then maybe:
def merge_records(data):
records = data['message']
users = {}
for record in records:
name = record['name']
id_ = record['id']
users[name] = id_
new_records = []
for name, id_ in users.items():
new_records.append({'name': name, 'id': id_})
return {'message': new_records}
But, if you have any control over how the data is represented, you might reconsider. You probably want a different data structure.
Here you go:
d = {'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
#loop over outer dictionary
for key, value in d.items():
d[key] = [dict([t for k in value for t in k.items()])]
print(d)
Edit:
As per your requirement:
d = {'message': [ {'name': 'raghav', 'id': 11}, {'name': 'krish', 'id': 20}, {'name': 'anu', 'id': 30}]}
for key, value in d.items():
print [dict((k1,v1)) for k1,v1 in dict([tuple(i.items()) for i in value for val in i.items()]).items()]