Suppose that I have a dict named data like below:
{
001: {
'data': {
'fruit': 'apple',
'vegetable': 'spinach'
},
'text': 'lorem ipsum',
'status': 10
},
002: {
.
.
.
}
}
I want to flatten(?) the data key and convert it to this:
{
001: {
'fruit': 'apple',
'vegetable': 'spinach',
'text': 'lorem ipsum',
'status': 10
},
002: {
.
.
.
}
}
I am trying to achieve this using dict comprehensions. Below implementation is with for loops:
mydict = {}
for id, values in data.items():
mydict[id] = {}
for label, value in values.items():
if label == 'data':
for x, y in value.items():
mydict[id][x] = y
else:
mydict[id][label] = value
I tried below comprehension but it gives syntax error:
mydict = {
id: {x: y} for x, y in value.items() if label == 'data' else {label: value}
for id, values in data.items() for label, value in values.items()}
Is there a way to achieve this using comprehensions only?
With dict expansions:
mydict = {i:{**v['data'], **{k:u for k, u in v.items() if k != "data"}} for i, v in data.items()}
The if clause in a comprehension (dict, list, set, generator) applies to the iteration itself, it can not be used for the production. For that you need conditionals in the production.
Generally speaking, comprehensions are really a reorganisation of a specific kind of (possibly nested) iterations:
a bunch of iterations and conditions, possibly nested
a single append/set
So
for a in b:
if c:
for d in e:
for f in g:
if h:
thing.append(i)
can be comprehension-ified, just move the production (i) to the head and put the other bits in a flat sequence:
thing = [
i
for a in b
if c
for d in e
for f in g
if h
]
Now your comprehension makes no sense, because it starts with iterating value, and there's no else in comprehension filter, and even if we add parens {x: y} for x, y in value.items() is not a value. Comprehensions also do not "merge" items, so with:
mydict = {
id: {label: value}
for id, values in data.items() for label, value in values.items()
}
Well you'll get only the last {label: value} for each id, because that's how dicts work.
Here if you consider the production loop, it's this:
for id, values in data.items():
mydict[id] = {}
This means that is your dict comprehension:
mydict = {
id: {}
for id, values in data.items()
}
the rest of the iteration is filling the value, so it needs to be a separate iteration inside the production:
mydict = {
id: {
label: value ???
for label, value in values.items()
}
for id, values in data.items()
}
In which case you hit the issue that this doesn't quite work, because you can't "conditionally iterate" in comprehensions, it's all or nothing.
Except you can: the right side of in is a normal expression, so you can do whatever you want with it, meaning you can unfold-or-refold:
mydict = {
id: {
x: y
for label, value in values.items()
for x, y in (value.items() if label == 'data' else [(label, value)])
}
for id, values in data.items()
}
This is a touch more expensive in the non-data case as you need to re-wrap the key and value in a tuple and list, but that's unlikely to be a huge deal.
An other alternative, instead of using a conditional comprehension, is to use splatting to merge the two dicts (once of which you create via a comp):
mydict = {
id: {
**values['data'],
**{label: value for label, value in values.items() if label != 'data'}
}
for id, values in data.items()
}
This can also be applied to the original to simplify it:
mydict = {}
for id, values in data.items():
mydict[id] = {}
for label, value in values.items():
if label == 'data':
mydict[id].update(value)
else:
mydict[id][label] = value
let me simplify;
sample_data = {
"001": {
"data": {
"fruit": 'apple',
"vegetable": 'spinach'
},
"text": 'lorem ipsum',
"status": 10
},
"002": {
"data": {
"fruit": 'apple',
"vegetable": 'spinach'
},
"text": 'lorem ipsum',
"status": 10
}
}
for key, row in sample_data.items():
if 'data' in row.keys():
info = sample_data[key].pop('data')
sample_data[key] = {**row, **info}
print(sample_data)
Related
I'm kinda new to python but I have a dict containing sort of key pairs:
{
"ata_smart_attributes_table_0_name":"Raw_Read_Error_Rate",
"ata_smart_attributes_table_0_raw_value":0,
"ata_smart_attributes_table_7_name":"Power_On_Hours",
"ata_smart_attributes_table_7_raw_value":1046,
}
I want to rename the '..0_name' key to the value of that key.
And at the same time the '..0_raw_value' value, has to become the value of the '..0_name' key like so:
{
"Raw_Read_Error_Rate":0,
"Power_On_Hours":1046,
}
I've been breaking my head over this, any suggestions?
Thanks in advance
Try:
dct = {
"ata_smart_attributes_table_0_name": "Raw_Read_Error_Rate",
"ata_smart_attributes_table_0_raw_value": 0,
"ata_smart_attributes_table_7_name": "Power_On_Hours",
"ata_smart_attributes_table_7_raw_value": 1046,
}
out = {
v: dct[k.rsplit("_", maxsplit=1)[0] + "_raw_value"]
for k, v in dct.items()
if k.endswith("_name")
}
print(out)
Prints:
{"Raw_Read_Error_Rate": 0, "Power_On_Hours": 1046}
I am trying to get a nested dict from a list of phrases.
My phrases for example are:
show version
show module
show module 0 det
show running-config
I am expecting a structure like this:
"show":{
"version":None,
"module":{
"0": {
"det"
}
},
"running-config":None
}
What I am trying is: split the phrases, from each array I am converting it to Dict.
for line in commandsOrdered:
value = line[-1]
line.pop(-1)
for key in list(reversed(line[:])):
value = {key: value}
sL.append(value)
And once I have a list of dicts, I am merging the dictionaries.
super_dict = {}
for d in sL:
for k, v in d.items():
super_dict.setdefault(k, []).append(v)
But I am getting this:
{
"show": [
"module",
{
"module": {
"0": "det"
}
},
"running-config",
"version"
],
"0": [
"det"
],
"module": [
{
"0": "det"
}
]
}
The max depth I have is 9 words in a phrase.
Any idea how to solve this?
Thanks
Something like this is fairly straightforward:
commandsOrdered = [
'show version',
'show module',
'show module 0 det',
'show running-config'
]
result = {}
for command in commandsOrdered:
parts = command.split()
d = result
for key in parts[:-1]:
if key not in d or not d[key]:
d[key] = {}
d = d[key]
d[parts[-1]] = None
print(result)
Output:
{'show': {'version': None, 'module': {'0': {'det': None}}, 'running-config': None}}
Not using defaultdict to meet the None requirement. You could easily write this recursively as well though, given the limited depth. That would make for simpler code, but not a faster solution per se.
I have a dictionary with a parent-key and its value is a dict. I want to extract a key,val pair from a list of dict.
given:
{"Premier" : {}}
I want to extract:
all_compseasons = content: [
{
label: "2019/20",
id: 274
},
{
label: "2018/19",
id: 210
}]
So to get:
{"Premier" :
{"2019/20" : 274,
"2018/19" : 210
}
}
I can't seem to find a good way to do it. I've tried below given other examples of the problem, but doesn't work.
compseasons = {}
for comp in all_compseasons:
competition_id = 'Premier'
index = competition_id
compseasons[index]comp['label'] = comp['id']
Your very close. Dictionary keys need to be referenced with surrounding [], so comp['label'] should be [comp['label']]. You can also just use the given dictionary {"Premier" : {}} instead of creating a new one with compseasons = {}, but either will give you the same result.
Working solution:
d = {"Premier": {}}
all_compseasons = [{"label": "2019/20", "id": 274}, {"label": "2018/19", "id": 210}]
for comp in all_compseasons:
d["Premier"][comp["label"]] = comp["id"]
print(d)
# {'Premier': {'2019/20': 274, '2018/19': 210}}
You just made a mistake in how you declared compseasons and how you are accessing the value of premier key which is also a dictionary.
Declaring compseasons = {"Premier" : {}} will not give you KeyError when you are trying to access it via compseasons[index] since Premier has already been inserted as a key.
Second, since your value of Premier itself is a dictionary, you should access the inner key enclosed in [] which would translate to compseasons[index][comp['label']] = comp['id'].
all_compseasons = [
{
'label': "2019/20",
'id': 274
},
{
'label': "2018/19",
'id': 210
}]
compseasons = {"Premier" : {}}
for comp in all_compseasons:
competition_id = 'Premier'
index = competition_id
compseasons[index][comp['label']] = comp['id']
I have the following object in python:
{
name: John,
age: {
years:18
},
computer_skills: {
years:4
},
mile_runner: {
years:2
}
}
I have an array with 100 people with the same structure.
What is the best way to go through all 100 people and make it such that there is no more "years"? In other words, each object in the 100 would look something like:
{
name: John,
age:18,
computer_skills:4,
mile_runner:2
}
I know I can do something in pseudocode:
for(item in list):
if('years' in (specific key)):
specifickey = item[(specific key)][(years)]
But is there a smarter/more efficent way?
Your pseudo-code is already pretty good I think:
for person in persons:
for k, v in person.items():
if isinstance(v, dict) and 'years' in v:
person[k] = v['years']
This overwrites every property which is a dictionary that has a years property with that property’s value.
Unlike other solutions (like dict comprehensions), this will modify the object in-place, so no new memory to keep everything is required.
def flatten(d):
ret = {}
for key, value in d.iteritems():
if isinstance(value, dict) and len(value) == 1 and "years" in value:
ret[key] = value["years"]
else:
ret[key] = value
return ret
d = {
"name": "John",
"age": {
"years":18
},
"computer_skills": {
"years":4
},
"mile_runner": {
"years":2
}
}
print flatten(d)
Result:
{'age': 18, 'mile_runner': 2, 'name': 'John', 'computer_skills': 4}
Dictionary comprehension:
import json
with open("input.json") as f:
cont = json.load(f)
print {el:cont[el]["years"] if "years" in cont[el] else cont[el] for el in cont}
prints
{u'age': 18, u'mile_runner': 2, u'name': u'John', u'computer_skills': 4}
where input.json contains
{
"name": "John",
"age": {
"years":18
},
"computer_skills": {
"years":4
},
"mile_runner": {
"years":2
}
}
Linear with regards to number of elements, you can't really hope for any lower.
As people said in the comments, it isn't exactly clear what your "object" is, but assuming that you actually have a list of dicts like this:
list = [{
'name': 'John',
'age': {
'years': 18
},
'computer_skills': {
'years':4
},
'mile_runner': {
'years':2
}
}]
Then you can do something like this:
for item in list:
for key in item:
try:
item[key] = item[key]['years']
except (TypeError, KeyError):
pass
Result:
list = [{'age': 18, 'mile_runner': 2, 'name': 'John', 'computer_skills': 4}]
For example I have two dicts:
schema = {
'type': 'object',
'properties': {
'reseller_name': {
'type': 'string',
},
'timestamp': {
'type': 'integer',
},
},
'required': ['reseller_name', 'timestamp'],
}
and
schema_add = {
'properties': {
'user_login': {
'type': 'string',
},
},
'required': ['user_login'],
}
How I can get next merged with appending result dict:
schema_result = {
'type': 'object',
'properties': {
'reseller_name': {
'type': 'string',
},
'timestamp': {
'type': 'integer',
},
'user_login': {
'type': 'string',
},
},
'required': ['reseller_name', 'timestamp', 'user_login'],
}
Rules:
Same path is properties and required for scheme and scheme_add in example.
If both dict have dicts with same path, they merged with same rules.
If both dict have lists with same path, then add first list with second.
If both dict have simple values (or dict and non dict or list and non list) with same path, then first value overriding with second.
If only one dict have key with some path, than setting this key and value.
Not sure where the problem likes, but the way you're writing it down is almost like a computer program, and the example is like a test case. Why don't you start from this?
def add_dict(d1, d2):
newdict = {}
for (key, value) in d1.iteritems():
if key in d2: ...
#apply rules, add to newdict, use
else:
#simply add
for (key, value) in d2.iteritems():
if not key in d1:
# simply add
return newdict
This can probably be written more tightly, but might be easier like that to edit.
Edit.. after writing the last comment, couldn't help but write a nicer implementation
def merge_values(a,b):
if a==None or b==None:
return a or b
# now handle cases where both have values
if type(a)==dict:
return add_dict(a, b)
if type(a)==list:
...
def add_dict(d1,d2):
return dict(
[
(key,
merge_values(
d1.get(key,None),
d2.get(key,None)))
for key
in set(d1.keys()).union(d2.keys())
])
My own solution with #Nicolas78 help:
def merge(obj_1, obj_2):
if type(obj_1) == dict and type(obj_2) == dict:
result = {}
for key, value in obj_1.iteritems():
if key not in obj_2:
result[key] = value
else:
result[key] = merge(value, obj_2[key])
for key, value in obj_2.iteritems():
if key not in obj_1:
result[key] = value
return result
if type(obj_1) == list and type(obj_2) == list:
return obj_1 + obj_2
return obj_2
I am adding simple solution of this problem. Assuming that sample data will not change.
def merge_nested_dicts(schema,schema_add):
new_schema = schema
for k in schema:
if k in schema_add.keys():
if isinstance(schema_add[k],dict):
new_schema[k].update(schema_add[k])
if isinstance(schema_add[k],list):
new_schema[k] = new_schema[k]+schema_add[k]
return new_schema
Try this if you know the keys exactly.
schema['properties'].update(schema_add['properties'])
schema['result'].append(schema_add['result'])
result is merged in schema.
If you do not know the keys exactly then one loop is required to find inner list and dictionaries.
for value in schema:
if value is dict:
if schema_add.has_key(value) and schema_add[value] is dict:
schema[value].update(schema_add[value])
elif value is list:
if schema_add.has_key(value) and schema_add[value] is list:
schema[value].append(schema_add[value])
result can be merged into different dict as well.