My data is below
food ID
name
ingredients
ingredient ID
amount
unit
1
rice
red
R1
10
g
1
soup
blue
B1
20
g
1
soup
yellow
Y1
30
g
and I want to convert it like this
{
'data': [
{
'name': 'rice',
'ingredients': [
{
'name': 'red',
'ingredient_id':'R1',
'amount': 10,
'unit': 'g',
}
]
},
{
'name': 'soup',
'ingredients': [
{
'name': 'blue',
'ingredient_id':'B1',
'amount': 20,
'unit': 'g',
},
{
'name': 'yellow',
'ingredient_id':'Y1',
'amount': 30,
'unit': 'g',
}
]
}
]
}
How can I do it? Do I need to use the same library as pandas?
Yes you can modify your data by using custom code function inside python.
For your required format you need to use this code for format your data into json.
import pandas as pd
data = [[1, 'rice', 'red', 'R1', 10, 'g'],
[1, 'soup', 'blue', 'B1', 20, 'g'],
[1, 'soup', 'yellow', 'Y1', 30, 'g'],
[1, 'apple', 'yellow', 'Y1', 30, 'g']]
df = pd.DataFrame(data, columns=['food ID', 'name', 'ingredients', 'ingredient ID', 'amount', 'unit'])
def convert_data_group(group):
ingredients = [{'name': row['ingredients'], 'ingredient_id': row['ingredient ID'], 'amount': row['amount'], 'unit': row['unit']} for _, row in group.iterrows()]
return {'name': group.iloc[0]['name'], 'ingredients': ingredients}
unique_names = df['name'].unique().tolist()
result = []
for name in unique_names:
group = df[df['name'] == name]
result.append(convert_data_group(group))
final_result = {'datas': result}
print(final_result)
Your final result will be:
{'datas': [{'name': 'rice', 'ingredients': [{'name': 'red', 'ingredient_id': 'R1', 'amount': 10, 'unit': 'g'}]}, {'name': 'soup', 'ingredients': [{'name': 'blue', 'ingredient_id': 'B1', 'amount': 20, 'unit': 'g'}, {'name': 'yellow', 'ingredient_id': 'Y1', 'amount': 30, 'unit': 'g'}]}, {'name': 'apple', 'ingredients': [{'name': 'yellow', 'ingredient_id': 'Y1', 'amount': 30, 'unit': 'g'}]}]}
Related
Using Python3, I'm trying to move on key value pair in a dictionary list up on level.
I have a variable called product that has the following:
[
{'color': 'red',
'shape': 'round',
'extra': {'price': 'large',
'onsale': 'yes',
'instock: 'yes'}
},
{'color': 'blue',
'shape': 'square',
'extra': {'price': 'small',
'onsale': 'no',
'instock: 'yes'}
}
]
I'd like to move the key value pair of "instock" within extra up one level, to be on par with color, shape, extra - so this:
[
{'color': 'red',
'shape': 'round',
'extra': {'price': 'large',
'instock: 'yes'},
'onsale': 'yes'
},
{'color': 'blue',
'shape': 'square',
'extra': {'price': 'small',
'onsale': 'no'},
'instock: 'yes'
}
]
I tried playing with the following code that I found here:
result = {}
for i in products:
if i["href"] not in result:
result[i["selection_id"]] = {'selection_id': i["selection_id"], 'other_data': i["other_data"], 'value_dict': []}
result[i["selection_id"]]["value_dict"].append({'value': i["value"], "value_name": i["value_name"]})
It didn't work for me.
Any help or additional literature that I can find online would be greatly appreciated!
Very simple: iterate through the list. For each dict, copy "extra"."instock" up one level and delete the original:
for outer_dict in product:
outer_dict["instock"] = outer_dict["extra"]["instock"]
del outer_dict["extra"]["instock"]
for outer_dict in product:
print(outer_dict)
Output:
{'color': 'red', 'shape': 'round', 'extra': {'price': 'large', 'onsale': 'yes'}, 'instock': 'yes'}
{'color': 'blue', 'shape': 'square', 'extra': {'price': 'small', 'onsale': 'no'}, 'instock': 'yes'}
lst = [
{'color': 'red',
'shape': 'round',
'extra': {'price': 'large',
'onsale': 'yes',
'instock': 'yes'}
},
{'color': 'blue',
'shape': 'square',
'extra': {'price': 'small',
'onsale': 'no',
'instock': 'yes'}
}
]
for d in lst:
d['instock'] = d['extra'].pop('instock')
# pretty print on screen:
from pprint import pprint
pprint(lst)
Prints:
[{'color': 'red',
'extra': {'onsale': 'yes', 'price': 'large'},
'instock': 'yes',
'shape': 'round'},
{'color': 'blue',
'extra': {'onsale': 'no', 'price': 'small'},
'instock': 'yes',
'shape': 'square'}]
Or you could use:
d['extra'].pop('instock', 'no')
in case there's no instock key (the default value is no in this case)
products = [
{'color': 'red',
'shape': 'round',
'extra': {'price': 'large',
'onsale': 'yes',
'instock': 'yes'}
},
{'color': 'blue',
'shape': 'square',
'extra': {'price': 'small',
'onsale': 'no',
'instock': 'yes'}
}
]
result_list = []
result = {}
for item in products:
for key,values in item.items():
if isinstance(values,dict):
for inner_key, inner_value in values.items():
#remove me if you want all of the inner items to level-up
if inner_key == "instock":
result[inner_key] = inner_value
else:
result[key] = values
result_list.append(result)
print (result_list)
output:
[{'color': 'blue', 'shape': 'square', 'instock': 'yes'}, {'color': 'blue', 'shape': 'square', 'instock': 'yes'}]
added comment to clarify where to modify in case you want other key to be level-up as well
I have two lists whose elements are dictionaries.
list1 = [
{'id': 1, 'color': 'purple', 'size': 10},
{'id': 2, 'color': 'red', 'size': 25},
{'id': 3, 'color': 'orange', 'size': 1},
{'id': 4, 'color': 'black', 'size': 100},
{'id': 5, 'color': 'green', 'size': 33}
]
list2 = [
{'id': 2, 'width': 22, 'age': 22.3},
{'id': 5, 'width': 9, 'age': 1.7}
]
I want a third list that is the same length as the larger list, and where there is a dictionary element in the smaller list that has an id that matches a dictionary element in the larger list, merge the two dictionaries, so that the final output would look like:
list3 = [
{'id': 1, 'color': 'purple', 'size': 10},
{'id': 2, 'color': 'red', 'size': 25, 'width': 22, 'age': 22.3},
{'id': 3, 'color': 'orange', 'size': 1},
{'id': 4, 'color': 'black', 'size': 100},
{'id': 5, 'color': 'green', 'size': 33, 'width': 9, 'age': 1.7}
]
Ideally if this could be done without looping over both lists, that would be ideal.
Try this nested list comprehension with a dictionary with unpacking, and a next, as well as another list comprehension:
list3 = [{**i, **next(iter([x for x in list2 if x['id'] == i['id']]), {})} for i in list1]
And now:
print(list3)
Is:
[{'id': 1, 'color': 'purple', 'size': 10}, {'id': 2, 'color': 'red', 'size': 25, 'width': 22, 'age': 22.3}, {'id': 3, 'color': 'orange', 'size': 1}, {'id': 4, 'color': 'black', 'size': 100}, {'id': 5, 'color': 'green', 'size': 33, 'width': 9, 'age': 1.7}]
Use defaultdict
from collections import defaultdict
list1 = [
{'id': 1, 'color': 'purple', 'size': 10},
{'id': 2, 'color': 'red', 'size': 25},
{'id': 3, 'color': 'orange', 'size': 1},
{'id': 4, 'color': 'black', 'size': 100},
{'id': 5, 'color': 'green', 'size': 33}
]
list2 = [
{'id': 2, 'width': 22, 'age': 22.3},
{'id': 5, 'width': 9, 'age': 1.7}
]
dict1 = defaultdict(dict)
for l in (list1, list2):
for elem in l:
dict1[elem['id']].update(elem)
list3 = dict1.values()
print(list(list3))
O/P:
[
{
'id': 1,'color': 'purple','size': 10
},
{
'id': 2,'color': 'red','size': 25,'width': 22,'age': 22.3
},
{
'id': 3,'color': 'orange','size': 1
},
{
'id': 4,'color': 'black','size': 100
},
{
'id': 5, 'color': 'green','size': 33, 'width': 9,'age': 1.7
}
]
list3 is not guaranteed to be sorted (.values() returns items in no
specific order, you can try this to sort.
from operator import itemgetter
...
new_list = sorted(dict1.values(), key=itemgetter("id"))
I have dicts like this :
list_dicts = [
{
'code': 'A1', 'name': 'White',
},
{
'code': 'A2', 'name': 'Black',
},
{
'code': 'A1', 'name': 'White',
},
{
'code': 'A3', 'name': 'Red',
}
]
And how to get dicts look like this from above dicts ?
list_dicts = [
{
'code': 'A1', 'name': 'White', 'qty': 2
},
{
'code': 'A2', 'name': 'Black', 'qty': 1
},
{
'code': 'A3', 'name': 'Red', 'qty': 1
}
]
..........................
Assuming that you actually have a list of dicts:
list_dict = [
{'code': 'A1', 'name': 'White'},
{'code': 'A2', 'name': 'Black'},
{'code': 'A1', 'name': 'White'},
{'code': 'A3', 'name': 'Red'}
]
Then you can use collections.Counter to count the values:
In []:
from collections import Counter
Counter((x['code'], x['name']) for x in list_dict)
Out[]:
Counter({('A1', 'White'): 2, ('A2', 'Black'): 1, ('A3', 'Red'): 1})
It's a fairly easy exercise to turn this back into the list of dictionaries you are looking for.
Just use a defaultdict to store the counts of each (code, name):
from collections import defaultdict
list_dict = [{'code': 'A1', 'name': 'White'},
{'code': 'A2', 'name': 'Black'},
{'code': 'A1', 'name': 'White'},
{'code': 'A3', 'name': 'Red'},
]
d = defaultdict(int)
for dic in list_dict:
key = dic['code'], dic['name']
d[key] += 1
>>> print(dict(d))
{('A1', 'White'): 2, ('A2', 'Black'): 1, ('A3', 'Red'): 1}
Then just create a new list, and insert the new data from the dictionary above:
new_dict_list = [{'code':code, 'name':name, 'qty':qty} for (name, code), qty in d.items()]
>>> print(new_dict_list)
[{'code': 'A1', 'name': 'White', 'qty': 2}, {'code': 'A2', 'name': 'Black', 'qty': 1}, {'code': 'A3', 'name': 'Red', 'qty': 1}]
I'd like a function that can group a list of dictionaries into sublists of dictionaries depending on an arbitrary set of keys that all dictionaries have in common.
For example, I'd like the following list to be grouped into sublists of dictionaries depending on a certain set of keys
l = [{'name':'b','type':'new','color':'blue','amount':100},{'name':'c','type':'new','color':'red','amount':100},{'name':'d','type':'old','color':'gold','amount':100},{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100},{'name':'g','type':'normal','color':'red','amount':100}]
If I wanted to group by type, the following list would result, which has a sublists where each sublist has the same type:
[[{'name':'b','type':'new','color':'blue','amount':100},{'name':'c','type':'new','color':'red','amount':100}],[{'name':'d','type':'old','color':'gold','amount':100},{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100}],[{'name':'g','type':'normal','color':'red','amount':100}]]
If I wanted to group by type and color, the following would result where the list contains sublists that have the same type and color:
[[{'name':'b','type':'new','color':'blue','amount':100}],[{'name':'c','type':'new','color':'red','amount':100}],[{'name':'d','type':'old','color':'gold','amount':100}],[{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100}],[{'name':'g','type':'normal','color':'red','amount':100}]]
I understand the following function can group by one key, but I'd like to group by multiple keys:
def group_by_key(l,i):
l = [list(grp) for key, grp in itertools.groupby(sorted(l, key=operator.itemgetter(i)), key=operator.itemgetter(i))]
This is my attempt using the group_by_function above
def group_by_multiple_keys(l,*keys):
for key in keys:
l = group_by_key(l,key)
l = [item for sublist in l for item in sublist]
return l
The issue there is that it ungroups it right after it grouped it by a key. Instead, I'd like to re-group it by another key and still have one list of sublists.
itertools.groupby() + operator.itemgetter() will do what you want. groupby() takes an iterable and a key function, and groups the items in the iterable by the value returned by passing each item to the key function. itemgetter() is a factory that returns a function, which gets the specified items from any item passed to it.
from __future__ import print_function
import pprint
from itertools import groupby
from operator import itemgetter
def group_by_keys(iterable, keys):
key_func = itemgetter(*keys)
# For groupby() to do what we want, the iterable needs to be sorted
# by the same key function that we're grouping by.
sorted_iterable = sorted(iterable, key=key_func)
return [list(group) for key, group in groupby(sorted_iterable, key_func)]
dicts = [
{'name': 'b', 'type': 'new', 'color': 'blue', 'amount': 100},
{'name': 'c', 'type': 'new', 'color': 'red', 'amount': 100},
{'name': 'd', 'type': 'old', 'color': 'gold', 'amount': 100},
{'name': 'e', 'type': 'old', 'color': 'red', 'amount': 100},
{'name': 'f', 'type': 'old', 'color': 'red', 'amount': 100},
{'name': 'g', 'type': 'normal', 'color': 'red', 'amount': 100}
]
Examples:
>>> pprint.pprint(group_by_keys(dicts, ('type',)))
[[{'amount': 100, 'color': 'blue', 'name': 'b', 'type': 'new'},
{'amount': 100, 'color': 'red', 'name': 'c', 'type': 'new'}],
[{'amount': 100, 'color': 'gold', 'name': 'd', 'type': 'old'},
{'amount': 100, 'color': 'red', 'name': 'e', 'type': 'old'},
{'amount': 100, 'color': 'red', 'name': 'f', 'type': 'old'}],
[{'amount': 100, 'color': 'red', 'name': 'g', 'type': 'normal'}]]
>>>
>>> pprint.pprint(group_by_keys(dicts, ('type', 'color')))
[[{'amount': 100, 'color': 'blue', 'name': 'b', 'type': 'new'}],
[{'amount': 100, 'color': 'red', 'name': 'c', 'type': 'new'}],
[{'amount': 100, 'color': 'gold', 'name': 'd', 'type': 'old'}],
[{'amount': 100, 'color': 'red', 'name': 'e', 'type': 'old'},
{'amount': 100, 'color': 'red', 'name': 'f', 'type': 'old'}],
[{'amount': 100, 'color': 'red', 'name': 'g', 'type': 'normal'}]]
In a list containing dictionaries, how do I split it based on unique values of dictionaries? So for instance, this:
t = [
{'name': 'xyz', 'value': ['K','L', 'M', 'N']},
{'name': 'abc', 'value': ['O', 'P', 'K']}
]
becomes this:
t = [
{'name': 'xyz', 'value': 'K'},
{'name': 'xyz', 'value': 'L'},
{'name': 'xyz', 'value': 'M'},
{'name': 'xyz', 'value': 'N'},
{'name': 'abc', 'value': 'O'},
{'name': 'xyz', 'value': 'P'},
{'name': 'xyz', 'value': 'K'}
]
You can do this with a list comprehension. Iterate through each dictionary d, and create a new dictionary for each value in d['values']:
>>> t = [ dict(name=d['name'], value=v) for d in t for v in d['value'] ]
>>> t
[{'name': 'xyz', 'value': 'K'},
{'name': 'xyz', 'value': 'L'},
{'name': 'xyz', 'value': 'M'},
{'name': 'xyz', 'value': 'N'},
{'name': 'abc', 'value': 'O'},
{'name': 'abc', 'value': 'P'},
{'name': 'abc', 'value': 'K'}]