We Keep Coding

Python Dictionary comprehension with condition

Suppose that I have a dict named data like below:
{
001: {
'data': {
'fruit': 'apple',
'vegetable': 'spinach'
},
'text': 'lorem ipsum',
'status': 10
},
002: {
.
.
.
}
}
I want to flatten(?) the data key and convert it to this:
{
001: {
'fruit': 'apple',
'vegetable': 'spinach',
'text': 'lorem ipsum',
'status': 10
},
002: {
.
.
.
}
}
I am trying to achieve this using dict comprehensions. Below implementation is with for loops:
mydict = {}
for id, values in data.items():
mydict[id] = {}
for label, value in values.items():
if label == 'data':
for x, y in value.items():
mydict[id][x] = y
else:
mydict[id][label] = value
I tried below comprehension but it gives syntax error:
mydict = {
id: {x: y} for x, y in value.items() if label == 'data' else {label: value}
for id, values in data.items() for label, value in values.items()}
Is there a way to achieve this using comprehensions only?

With dict expansions:
mydict = {i:{**v['data'], **{k:u for k, u in v.items() if k != "data"}} for i, v in data.items()}

The if clause in a comprehension (dict, list, set, generator) applies to the iteration itself, it can not be used for the production. For that you need conditionals in the production.
Generally speaking, comprehensions are really a reorganisation of a specific kind of (possibly nested) iterations:
a bunch of iterations and conditions, possibly nested
a single append/set
So
for a in b:
if c:
for d in e:
for f in g:
if h:
thing.append(i)
can be comprehension-ified, just move the production (i) to the head and put the other bits in a flat sequence:
thing = [
i
for a in b
if c
for d in e
for f in g
if h
]
Now your comprehension makes no sense, because it starts with iterating value, and there's no else in comprehension filter, and even if we add parens {x: y} for x, y in value.items() is not a value. Comprehensions also do not "merge" items, so with:
mydict = {
id: {label: value}
for id, values in data.items() for label, value in values.items()
}
Well you'll get only the last {label: value} for each id, because that's how dicts work.
Here if you consider the production loop, it's this:
for id, values in data.items():
mydict[id] = {}
This means that is your dict comprehension:
mydict = {
id: {}
for id, values in data.items()
}
the rest of the iteration is filling the value, so it needs to be a separate iteration inside the production:
mydict = {
id: {
label: value ???
for label, value in values.items()
}
for id, values in data.items()
}
In which case you hit the issue that this doesn't quite work, because you can't "conditionally iterate" in comprehensions, it's all or nothing.
Except you can: the right side of in is a normal expression, so you can do whatever you want with it, meaning you can unfold-or-refold:
mydict = {
id: {
x: y
for label, value in values.items()
for x, y in (value.items() if label == 'data' else [(label, value)])
}
for id, values in data.items()
}
This is a touch more expensive in the non-data case as you need to re-wrap the key and value in a tuple and list, but that's unlikely to be a huge deal.
An other alternative, instead of using a conditional comprehension, is to use splatting to merge the two dicts (once of which you create via a comp):
mydict = {
id: {
**values['data'],
**{label: value for label, value in values.items() if label != 'data'}
}
for id, values in data.items()
}
This can also be applied to the original to simplify it:
mydict = {}
for id, values in data.items():
mydict[id] = {}
for label, value in values.items():
if label == 'data':
mydict[id].update(value)
else:
mydict[id][label] = value

let me simplify;
sample_data = {
"001": {
"data": {
"fruit": 'apple',
"vegetable": 'spinach'
},
"text": 'lorem ipsum',
"status": 10
},
"002": {
"data": {
"fruit": 'apple',
"vegetable": 'spinach'
},
"text": 'lorem ipsum',
"status": 10
}
}
for key, row in sample_data.items():
if 'data' in row.keys():
info = sample_data[key].pop('data')
sample_data[key] = {**row, **info}
print(sample_data)

How to convert values of a specific dictionary key to uppercase?

I have this simplified dict:
{
{
"birthPlace" : "london"
},
"hello": "hello",
"birthPlace" : "rome"
}
And I want to make the value of birthPlace uppercase: how? I tried
smallalphabetDict={}
for key, value in myjson.items():
smallalphabetDict[key.upper()] = value
It doesn't work

This changes all the values of a dict to uppercase, if the value is a string:
d = {......}
for k in d:
if type(d[k]) == str: d[k] = d[k].upper()

pythonic way to check if my dict contains prototyped key hierarchy

I have a dict, lets say mydict
I also know about this json, let's say myjson:
{
"actor":{
"name":"",
"type":"",
"mbox":""
},
"result":{
"completion":"",
"score":{ "scaled":"" },
"success":"",
"timestamp":""
},
"verb":{
"display":{
"en-US":""
},
"id":""
},
"context":{
"location":"",
"learner_id": "",
"session_id": ""
},
"object":{
"definition":{
"name":{
"en-US":""
}
},
"id":"",
"activity_type":""
}
}
I want to know if ALL of myjson keys (with the same hierarchy) are in mydict. I don't care if mydict has more data in it (it can have more data). How do I do this in python?

Make a dictionary of myjson
import json
with open('myjson.json') as j:
new_dict = json.loads(j.read())
Then go through each key of that dictionary, and confirm that the value of that key is the same in both dictionaries
def compare_dicts(new_dict, mydict):
for key in new_dict:
if key in mydict and mydict[key] == new_dict[key]:
continue
else:
return False
return True
EDIT:
A little more complex, but something like this should suit you needs
def compare(n, m):
for key in n:
if key in m:
if m[key] == n[key]:
continue
elif isinstance(n[key], dict) and isinstance(m[key],dict):
if compare(n[key], m[key]):
continue
else:
return False
else:
return False
return True

If you just care about the values and not the keys you can do this:
>>> all(v in mydict.items() for v in myjson.items())
True
Will be true if all values if myjson are in mydict, even if they have other keys.
Edit: If you only care about the keys, use this:
>>> all(v in mydict.keys() for v in myjson.keys())
True
This returns true if every key of myjson is in mydict, even if they point to different values.

golang defining dict like python with and appending value to list in dict

I am new to go and trying to implement python like nested structure as below, I am not able to define the empty dict/map in golang that can have list of specific struct/classobj and while iterating through the data I am not able to append items in map/dict ... I will really appreciate any help on this ... Thanks
items = [
("item1", someObj1),
("item2", someObj2),
("item3", someObj3),
("item3", someObj5),
("item1", someObj4),
]
rectors = {}
for item, obj in items:
try:
rectors[item].append(obj)
except KeyError:
rectors[item] = [obj]
print rectors
# OUTPUT: {'item2': [someObj2], 'item3': [someObj3, someObj5], 'item1': [someObj1, someObj4]}

Its not quite as clean .. but this does roughly what you want and should get you started on the right path:
type someObj struct {
}
func main() {
items := map[string][]someObj{
"item1": []someObj{someObj{}},
"item2": []someObj{someObj{}},
"item3": []someObj{someObj{}},
}
items["item1"] = append(items["item1"], someObj{})
rectors := make(map[string][]someObj)
for key, val := range items {
if obj, exists := rectors[key]; exists {
rectors[key] = append(obj, val...)
} else {
rectors[key] = val
}
}
fmt.Printf("%v", rectors)
}
Output:
map[item3:[{}] item1:[{} {}] item2:[{}]]
The main difference being .. you can't initialize the map and alter an item with a key that already exists (as you appear to be doing in your example with item1 being appended during initialization). So that becomes an extra step after the initialization of the map. You could always just do:
"item1": []someObj{someObj{}, someObj{}},
.. but that didn't seem to be the same as what you were doing.
See it on the Go playground

How merge with appending two nested dictionaries in python?

For example I have two dicts:
schema = {
'type': 'object',
'properties': {
'reseller_name': {
'type': 'string',
},
'timestamp': {
'type': 'integer',
},
},
'required': ['reseller_name', 'timestamp'],
}
and
schema_add = {
'properties': {
'user_login': {
'type': 'string',
},
},
'required': ['user_login'],
}
How I can get next merged with appending result dict:
schema_result = {
'type': 'object',
'properties': {
'reseller_name': {
'type': 'string',
},
'timestamp': {
'type': 'integer',
},
'user_login': {
'type': 'string',
},
},
'required': ['reseller_name', 'timestamp', 'user_login'],
}
Rules:
Same path is properties and required for scheme and scheme_add in example.
If both dict have dicts with same path, they merged with same rules.
If both dict have lists with same path, then add first list with second.
If both dict have simple values (or dict and non dict or list and non list) with same path, then first value overriding with second.
If only one dict have key with some path, than setting this key and value.

Not sure where the problem likes, but the way you're writing it down is almost like a computer program, and the example is like a test case. Why don't you start from this?
def add_dict(d1, d2):
newdict = {}
for (key, value) in d1.iteritems():
if key in d2: ...
#apply rules, add to newdict, use
else:
#simply add
for (key, value) in d2.iteritems():
if not key in d1:
# simply add
return newdict
This can probably be written more tightly, but might be easier like that to edit.
Edit.. after writing the last comment, couldn't help but write a nicer implementation
def merge_values(a,b):
if a==None or b==None:
return a or b
# now handle cases where both have values
if type(a)==dict:
return add_dict(a, b)
if type(a)==list:
...
def add_dict(d1,d2):
return dict(
[
(key,
merge_values(
d1.get(key,None),
d2.get(key,None)))
for key
in set(d1.keys()).union(d2.keys())
])

My own solution with #Nicolas78 help:
def merge(obj_1, obj_2):
if type(obj_1) == dict and type(obj_2) == dict:
result = {}
for key, value in obj_1.iteritems():
if key not in obj_2:
result[key] = value
else:
result[key] = merge(value, obj_2[key])
for key, value in obj_2.iteritems():
if key not in obj_1:
result[key] = value
return result
if type(obj_1) == list and type(obj_2) == list:
return obj_1 + obj_2
return obj_2

I am adding simple solution of this problem. Assuming that sample data will not change.
def merge_nested_dicts(schema,schema_add):
new_schema = schema
for k in schema:
if k in schema_add.keys():
if isinstance(schema_add[k],dict):
new_schema[k].update(schema_add[k])
if isinstance(schema_add[k],list):
new_schema[k] = new_schema[k]+schema_add[k]
return new_schema

Try this if you know the keys exactly.
schema['properties'].update(schema_add['properties'])
schema['result'].append(schema_add['result'])
result is merged in schema.
If you do not know the keys exactly then one loop is required to find inner list and dictionaries.
for value in schema:
if value is dict:
if schema_add.has_key(value) and schema_add[value] is dict:
schema[value].update(schema_add[value])
elif value is list:
if schema_add.has_key(value) and schema_add[value] is list:
schema[value].append(schema_add[value])
result can be merged into different dict as well.