I am a beginner in python, I have faced some problem. I have an object list like this :
[
{
'balance':-32399.0,
'code':u'1011',
'name':u'Stock Valuation Account'
},
{
'balance':-143503.34,
'code':u'1011',
'name':u'Stock Interim Account (Received)'
},
{
'balance':117924.2499995,
'code':u'1011',
'name':u'Stock Interim Account (Delivered)'
},
{
'balance':-3500000.0,
'code':u'1101',
'name':u'Cash'
},
{
'balance':-50000.0,
'code':u'1101',
'name':u'Other Cash'
},
]
I need to sum it based on the code, so the result will be.
[
{
'balance':6819,91,
'code':u'1011',
},
{
'balance':-3550000.0,
'code':u'1101',
},
]
have search over StackOverflow, but still not got what I need.
any help?...
Exactly, using groupby and sum within some comprehensions:
As said in the comments, for using groupby the list need to be presorted.
In addition you can use operator.attrgetter instead of lambdas in the key parameters of sorted and groupby.
l = [
{
'balance':-32399.0,
'code':u'1011',
'name':u'Stock Valuation Account'
},
...
]
from itertools import groupby
import operator
selector_func = operator.attrgetter("code")
l = sorted(l, key=selector_func)
result = [{"code" : code, "balance" : sum(x["balance"] for x in values)} for code, values in groupby(l, selector_func)]
print(result)
Result:
[{'code': '1011', 'balance': -57978.0900005}, {'code': '1101', 'balance': -3550000.0}]
Here you have the live example
Here is an oneliner without any import :
a = ...
result = [{'balance' : sum([i['balance'] for i in a if i['code']==j]), 'code' : j} for j in set([k['code'] for k in a])]
OUTPUT :
[{'balance': -3550000.0, 'code': '1101'}, {'balance': -57978.0900005, 'code': '1011'}]
data = [
{
'balance':-32399.0,
'code':u'1011',
'name':u'Stock Valuation Account'
},
...
]
d = {}
for entry in data:
d[entry['code']] = d.get(entry['code'],0) + entry['balance']
print([{'balance':b,'code':c} for c,b in d.items()])
Would print:
[{'balance': -57978.0900005, 'code': '1011'}, {'balance': -3550000.0, 'code': '1101'}]
Related
categories = [
{
"CategoryUId": "f34cc7a8-ac38-4f1f-a637-08bd034d74f3",
"SubCategory": [
{
"SubCategoryUId": "9b37dbf3-4b4d-4bbb-8bc4-2ce036b69042"
},
{
"SubCategoryUId": "d4131c98-9823-4354-b587-c736cd77df4d"
}
]
},
{
"CategoryUId": "460366f6-c8ef-4e4e-80a7-4ace9c59122c",
"SubCategory": [
{
"SubCategoryUId": "ed6dbfb9-bc1a-4161-b040-f9aba55c995a"
},
{
"SubCategoryUId": "06246a88-fe8a-42fa-aba6-3393af463397"
},
{
"SubCategoryUId": "2f37fd26-fae5-4dc4-9f10-4e87a2f5ae68"
}
]
}
]
A basic example where is categories is a list of dictionaries.
for item in categories:
categoryUId = item['CategoryUId']
for value in item['SubCategory']:
subcategory = value['SubCategoryUId']
subcategoryList.append(subcategory)
dict = {categoryUId : subcategoryList}
I am going through some random python exercises, is it possible to use lambda function or the List comprehension for the above mentioned code snippet.
Please assist me with an approach.
It seems this is the outcome you expected:
out = {d['CategoryUId']: [v['SubCategoryUId'] for v in d['SubCategory']] for d in categories}
The above code with a lambda in map (extremely ugly though; do not recommend):
out = dict(map(lambda d: (d['CategoryUId'], [*map(lambda v:v['SubCategoryUId'], d['SubCategory'])]), categories))
Output:
{'f34cc7a8-ac38-4f1f-a637-08bd034d74f3': ['9b37dbf3-4b4d-4bbb-8bc4-2ce036b69042',
'd4131c98-9823-4354-b587-c736cd77df4d'],
'460366f6-c8ef-4e4e-80a7-4ace9c59122c': ['ed6dbfb9-bc1a-4161-b040-f9aba55c995a',
'06246a88-fe8a-42fa-aba6-3393af463397',
'2f37fd26-fae5-4dc4-9f10-4e87a2f5ae68']}
I have two dictionaries, as below. Both dictionaries have a list of dictionaries as the value associated with their properties key; each dictionary within these lists has an id key. I wish to merge my two dictionaries into one such that the properties list in the resulting dictionary only has one dictionary for each id.
{
"name":"harry",
"properties":[
{
"id":"N3",
"status":"OPEN",
"type":"energetic"
},
{
"id":"N5",
"status":"OPEN",
"type":"hot"
}
]
}
and the other list:
{
"name":"harry",
"properties":[
{
"id":"N3",
"type":"energetic",
"language": "english"
},
{
"id":"N6",
"status":"OPEN",
"type":"cool"
}
]
}
The output I am trying to achieve is:
"name":"harry",
"properties":[
{
"id":"N3",
"status":"OPEN",
"type":"energetic",
"language": "english"
},
{
"id":"N5",
"status":"OPEN",
"type":"hot"
},
{
"id":"N6",
"status":"OPEN",
"type":"cool"
}
]
}
As id: N3 is common in both the lists, those 2 dicts should be merged with all the fields. So far I have tried using itertools and
ds = [d1, d2]
d = {}
for k in d1.keys():
d[k] = tuple(d[k] for d in ds)
Could someone please help in figuring this out?
Here is one of the approach:
a = {
"name":"harry",
"properties":[
{
"id":"N3",
"status":"OPEN",
"type":"energetic"
},
{
"id":"N5",
"status":"OPEN",
"type":"hot"
}
]
}
b = {
"name":"harry",
"properties":[
{
"id":"N3",
"type":"energetic",
"language": "english"
},
{
"id":"N6",
"status":"OPEN",
"type":"cool"
}
]
}
# Create dic maintaining the index of each id in resp dict
a_ids = {item['id']: index for index,item in enumerate(a['properties'])} #{'N3': 0, 'N5': 1}
b_ids = {item['id']: index for index,item in enumerate(b['properties'])} #{'N3': 0, 'N6': 1}
# Loop through one of the dict created
for id in a_ids.keys():
# If same ID exists in another dict, update it with the key value
if id in b_ids:
b['properties'][b_ids[id]].update(a['properties'][a_ids[id]])
# If it does not exist, then just append the new dict
else:
b['properties'].append(a['properties'][a_ids[id]])
print (b)
Output:
{'name': 'harry', 'properties': [{'id': 'N3', 'type': 'energetic', 'language': 'english', 'status': 'OPEN'}, {'id': 'N6', 'status': 'OPEN', 'type': 'cool'}, {'id': 'N5', 'status': 'OPEN', 'type': 'hot'}]}
It might help to treat the two objects as elements each in their own lists. Maybe you have other objects with different name values, such as might come out of a JSON-formatted REST request.
Then you could do a left outer join on both name and id keys:
#!/usr/bin/env python
a = [
{
"name": "harry",
"properties": [
{
"id":"N3",
"status":"OPEN",
"type":"energetic"
},
{
"id":"N5",
"status":"OPEN",
"type":"hot"
}
]
}
]
b = [
{
"name": "harry",
"properties": [
{
"id":"N3",
"type":"energetic",
"language": "english"
},
{
"id":"N6",
"status":"OPEN",
"type":"cool"
}
]
}
]
a_names = set()
a_prop_ids_by_name = {}
a_by_name = {}
for ao in a:
an = ao['name']
a_names.add(an)
if an not in a_prop_ids_by_name:
a_prop_ids_by_name[an] = set()
for ap in ao['properties']:
api = ap['id']
a_prop_ids_by_name[an].add(api)
a_by_name[an] = ao
res = []
for bo in b:
bn = bo['name']
if bn not in a_names:
res.append(bo)
else:
ao = a_by_name[bn]
bp = bo['properties']
for bpo in bp:
if bpo['id'] not in a_prop_ids_by_name[bn]:
ao['properties'].append(bpo)
res.append(ao)
print(res)
The idea above is to process list a for names and ids. The names and ids-by-name are instances of a Python set. So members are always unique.
Once you have these sets, you can do the left outer join on the contents of list b.
Either there's an object in b that doesn't exist in a (i.e. shares a common name), in which case you add that object to the result as-is. But if there is an object in b that does exist in a (which shares a common name), then you iterate over that object's id values and look for ids not already in the a ids-by-name set. You add missing properties to a, and then add that processed object to the result.
Output:
[{'name': 'harry', 'properties': [{'id': 'N3', 'status': 'OPEN', 'type': 'energetic'}, {'id': 'N5', 'status': 'OPEN', 'type': 'hot'}, {'id': 'N6', 'status': 'OPEN', 'type': 'cool'}]}]
This doesn't do any error checking on input. This relies on name values being unique per object. So if you have duplicate keys in objects in both lists, you may get garbage (incorrect or unexpected output).
My json object:
"students": [
{
"name" : "ben",
"hometown" : "unknown"
},
{
"name" : "sam",
"hometown" : "unknown"
}
]
}
with this list
"hometowns":{California,Colorado}
change to this:
"students": [
{
"name" : "ben",
"hometown" : "California"
},
{
"name" : "sam",
"hometown" : "Colorado"
}
]
}
I need to loop and check if the key = "hometown" and change its value like
students[1].hometown == hometowns[1].
First, note that your syntax is a bit off (and your towns are actually states). After correcting the syntax, we can use the zip() function to iterate over both lists together:
hometowns = ["California", "Colorado"]
students = [{"name": "ben", "hometown": "unknown"},
{"name": "sam", "hometown": "unknown"}]
for student, hometown in zip(students, hometowns):
student['hometown'] = hometown
students
[{'name': 'ben', 'hometown': 'California'},
{'name': 'sam', 'hometown': 'Colorado'}]
You can do something like this
hometowns=["California","Colorado"]
students=[{"name" : "ben",},{"name" : "sam"}]
for student,town in zip(students,hometown):
student["hometown"]=town
I assumed you were trying to specify hometowns and students as variables rather than elements of a larger dictionary, which would change the syntax somewhat.
here is an alternative simple solution if you have not learnt about zip() in python:
hometowns = ["California","Colorado"]
a = {"students": [
{
"name" : "ben",
"hometown" : "unknown"
},
{
"name" : "sam",
"hometown" : "unknown"
}
] }
num = 0
for j in a["students"]:
j["hometown"] = hometowns[num]
num += 1
print(a)
I want to convert my dictionary to this format. I have tried using groupby but not able to achieve the expected format.
input = [
{'algorithms': 'BLOWFISH', 'dcount': 5.8984375},
{'algorithms': 'AES-256', 'dcount': 5.609375},
{'algorithms': 'AES-256', 'dcount': 9.309375},
{'algorithms': 'RSA', 'dcount': 8.309375},
{'algorithms': 'BLOWFISH','dcount': 6.309375}
]
Expected output:
output = [
{
name: "BLOWFISH",
data: [5.8984375,6.309375]
},
{
name: "AES-256",
data: [5.609375,9.309375]
},
{
name: 'RSA',
data: [8.309375]
}
]
You need to sort input before itertools.groupby will work:
The operation of groupby() is similar to the uniq filter in Unix. It
generates a break or new group every time the value of the key
function changes (which is why it is usually necessary to have sorted
the data using the same key function). That behavior differs from
SQL’s GROUP BY which aggregates common elements regardless of their
input order.
from itertools import groupby
import json
input = [
{
"algorithms": "BLOWFISH",
"dcount": 5.8984375
},
{
"algorithms": "AES-256",
"dcount": 5.609375
},
{
"algorithms": "AES-256",
"dcount": 9.309375
},
{
"algorithms": "RSA",
"dcount": 8.309375
},
{
"algorithms": "BLOWFISH",
"dcount": 6.309375
}
]
output = [
{
"name": k,
"data": [d["dcount"] for d in g]
}
for k, g in groupby(sorted(input, key=lambda d: d["algorithms"]),
key=lambda d: d["algorithms"])
]
print(json.dumps(output, indent=4))
Output:
[
{
"name": "AES-256",
"data": [
5.609375,
9.309375
]
},
{
"name": "BLOWFISH",
"data": [
5.8984375,
6.309375
]
},
{
"name": "RSA",
"data": [
8.309375
]
}
]
Had a go at this for you:
data = [
{'algorithms': 'BLOWFISH', 'dcount': 5.8984375},
{'algorithms': 'AES-256', 'dcount': 5.609375},
{'algorithms': 'AES-256', 'dcount': 9.309375},
{'algorithms': 'RSA', 'dcount': 8.309375},
{'algorithms': 'BLOWFISH', 'dcount': 6.309375}
]
series = []
firstRun = True
for i in data:
found = False
if firstRun:
series.append(
{
'name': i['algorithms'],
'data': [i['dcount']]
}
)
firstRun = False
else:
for j in series:
if i['algorithms'] == j['name']:
j['data'].append(i['dcount'])
found = True
else:
continue
if not found:
series.append(
{
'name': i['algorithms'],
'data': [i['dcount']]
}
)
This should give you your desired output:
>>> print(series)
[{'name': 'BLOWFISH', 'data': [5.8984375, 6.309375]}, {'name': 'AES-256', 'data': [5.609375, 9.309375]}, {'name': 'RSA', 'data': [8.309375]}]
You can do that with no additional module:
def group_algorithms(input_list):
out = []
names = []
for algorithms in input_list:
if algorithms['algorithms'] in names:
out[names.index(algorithms['algorithms'])]["data"].append(algorithms['dcount'])
else:
out.append({"name": algorithms['algorithms'],
"data": [algorithms['dcount']]})
names.append(algorithms['algorithms'])
return out
I'm trying to deepmerge lists to get a specific json.
what I want to achieve is this format (the order of the elements is irelevant):
{
"report": {
"context": [{
"report_id": [
"Report ID 30"
],
"status": [
"Status 7"
],
"fallzahl": [
"Fallzahl 52"
],
"izahl": [
"IZahl 20"
]
}
],
"körpergewicht": [{
"any_event_en": [{
"gewicht": [{
"|magnitude": 185.44,
"|unit": "kg"
}
],
"kommentar": [
"Kommentar 94"
],
"bekleidung": [{
"|code": "at0011"
}
]
}
]
}
]
}
}
I try to deepmerge dicts and lists to achieve this specific format. My baseline are some dicts:
{'körpergewicht': [{'any_event_en': [{'gewicht': [{'|magnitude': '100', '|unit': 'kg'}]}]}]}
{'körpergewicht': [{'any_event_en': [{'bekleidung': [{'|code': 'at0013'}]}]}]}
{'körpergewicht': [{'any_event_en': [{'kommentar': ['none']}]}]}
{'context': [{'status': ['fatty']}]}
{'context': [{'fallzahl': ['123']}]}
{'context': [{'report_id': ['123']}]}
{'context': [{'izahl': ['123']}]}
what I tried to do is following I have a dict called tmp_dict in that I hold a baseline dict as I loop through. The so called collect_dict is the dict in that I try to merge my baseline dicts. element holds my current baseline dict.
if (index == (len(element)-1)): #when the baseline dict is traversed completely
if tmp_dict:
first_key_of_tmp_dict=list(tmp_dict.keys())[0]
if not (first_key_of_tmp_dict in collect_dict):
collect_dict.update(tmp_dict)
else:
merge(tmp_dict,collect_dict)
else:
collect_dict.update(tmp_dict)
and I also wrote a merge method:
def merge(tmp_dict,collect_dict):
first_common_key_of_dicts=list(tmp_dict.keys())[0]
second_depth_key_of_tmp_dict=list(tmp_dict[first_common_key_of_dicts][0].keys())[0]
second_depth_tmp_dict=tmp_dict[first_common_key_of_dicts][0]
second_depth_key_of_coll_dict=collect_dict[first_common_key_of_dicts][0]
if not second_depth_key_of_tmp_dict in second_depth_key_of_coll_dict:
collect_dict.update(second_depth_tmp_dict)
else:
merge(second_depth_tmp_dict,second_depth_key_of_coll_dict)
what I'm coming up with goes in the right direction but is far from beeing my desired output:
{"report": {
"k\u00f6rpergewicht": [{
"any_event_en": [{
"kommentar": ["none"]
}
],
"bekleidung": [{
"|code": "at0013"
}
],
"gewicht": [{
"|magnitude": "100",
"|unit": "kg"
}
]
}
],
"context": [{
"fallzahl": ["234"]
}
],
"report_id": ["234"],
"status": ["s"],
"izahl": ["234"]
}
}
With another set of inputs:
{'atemfrequenz': {'context': [{'status': [{'|code': 'at0012'}]}]}},
{'atemfrequenz': {'context': [{'kategorie': ['Kategorie']}]}},
{'atemfrequenz': {'atemfrequenz': [{'messwert': [{'|magnitude': '123', '|unit': 'min'}]}]}}
I would like to achieve the following output:
"atemfrequenz": {
"context": [
{
"status": [
{
"|code": "at0012"
}
],
"kategorie": [
"Kategorie"
]
}
],
"atemfrequenz": [
{
"messwert": [
{
"|magnitude": 123,
"|unit": "/min"
}
]
}
]
}
This code should get the correct answer. I removed the special character (ö) to prevent errors.
dd = [
{'korpergewicht': [{'any_event_en': [{'gewicht': [{'|magnitude': '100', '|unit': 'kg'}]}]}] },
{'korpergewicht': [{'any_event_en': [{'bekleidung': [{'|code': 'at0013'}]}]}]},
{'korpergewicht': [{'any_event_en': [{'kommentar': ['none']}]}]},
{'context': [{'status': ['fatty']}]},
{'context': [{'fallzahl': ['123']}]},
{'context': [{'report_id': ['123']}]},
{'context': [{'izahl': ['123']}]}
]
def merge(d):
if (type(d) != type([])): return d
if (type(list(d[0].values())[0])) == type(""): return d
keys = list(set(list(k.keys())[0] for k in d))
lst = [{k:[]} for k in keys]
for e in lst:
for k in d:
if (list(e.keys())[0] == list(k.keys())[0]):
e[list(e.keys())[0]] += k[list(k.keys())[0]]
for e in lst:
if (type(e[list(e.keys())[0]][0]) == type({})):
e[list(e.keys())[0]] = merge(e[list(e.keys())[0]])
for i in lst[1:]: lst[0].update(i)
lst2 = [] # return list of single dictionary
lst2.append(lst[0])
return lst2
dx = merge(dd)
dx = {'report': dx[0]} # no list at lowest level
print(dx)
Output (formatted)
{'report': {
'korpergewicht': [{
'any_event_en': [{
'kommentar': ['none'],
'bekleidung': [{'|code': 'at0013'}],
'gewicht': [{'|magnitude': '100', '|unit': 'kg'}]}]}],
'context': [{
'report_id': ['123'],
'izahl': ['123'],
'fallzahl': ['123'],
'status': ['fatty']}]}}
Concerning the second data set provided, the data needs to structured to match the previous data set.
This data set works correctly:
dd = [
{'atemfrequenz': [{'context': [{'status': [{'|code': 'at0012'}]}]}]},
{'atemfrequenz': [{'context': [{'kategorie': ['Kategorie']}]}]},
{'atemfrequenz': [{'atemfrequenz': [{'messwert': [{'|magnitude': '123', '|unit': 'min'}]}]}]}
]
Output (formatted)
{'report': {
'atemfrequenz': [{
'atemfrequenz': [{
'messwert': [{'|magnitude': '123', '|unit': 'min'}]}],
'context': [{
'kategorie': ['Kategorie'],
'status': [{'|code': 'at0012'}]}]}]}}