I want to generate combinations of an object of lists in Python and was looking into itertools.product or similar functions to calculate combinations. Since itertools.product generates combinations from multiple arrays
I have an object that looks like this:
{
"Cities": [
{
"Id": 1,
"Value": "New York"
},
{
"Id": 2,
"Value": "Boston"
}
],
"People": [
{
"Id": 1,
"Value": "Michael"
},
{
"Id": 2,
"Value": "Ryan"
},
{
"Id": 3,
"Value": "Jim"
},
{
"Id": 4,
"Value": "Phyllis"
}
]
}
And I want to generate a list that shows all combinations of people living in each city. So in the case above a list of 8 values.
My code below looks like:
import json
import itertools
def main():
combinations = []
with open('people.json') as f:
data = json.load(f)
combinations = list(itertools.product(*data))
print(combinations)
if __name__ == "__main__":
main()
When running I get a completely different result:
How can I modify my code to get the result I want?
Note: It doesn't have to use itertools, I just thought itertools is used for calculations like these
To perform the job you have described, using the data you have shown, this script will do it:
import json
import itertools
def main():
combinations = []
with open('people.json') as f:
data = json.load(f)
combinations = list(itertools.product(data['Cities'], data['People']))
print(combinations)
if __name__ == "__main__":
main()
The only difference is that I'm specifying which data to use from the data structure.
The output (I have formatted it for readability):
[
({"Id": 1, "Value": "New York"},
{"Id": 1, "Value": "Michael"}),
({"Id": 1, "Value": "New York"},
{"Id": 2, "Value": "Ryan"}),
({"Id": 1, "Value": "New York"},
{"Id": 3, "Value": "Jim"}),
({"Id": 1, "Value": "New York"},
{"Id": 4, "Value": "Phyllis"}),
({"Id": 2, "Value": "Boston"},
{"Id": 1, "Value": "Michael"}),
({"Id": 2, "Value": "Boston"},
{"Id": 2, "Value": "Ryan"}),
({"Id": 2, "Value": "Boston"},
{"Id": 3, "Value": "Jim"}),
({"Id": 2, "Value": "Boston"},
{"Id": 4, "Value": "Phyllis"})
]
If you instead wanted to perform a product between whatever keys are in the data set, you would want to do itertools.product(data.values()), but the code I have shown is clearer.
Why you get the output that you do:
When you do list(itertools.product(*data)), the same thing is passed to product that you see when you do:
for x in data:
print(x)
ie., you did
itertools.product(['Cities', 'People'])
which is why you got the product of the characters in those two strings (yay duck typing!)
[
('C', 'P'),
('C', 'e'),
('C', 'o'),
('C', 'p'),
('C', 'l'),
('C', 'e'),
('i', 'P'),
('i', 'e'),
('i', 'o'),
('i', 'p'),
('i', 'l'),
('i', 'e'),
...
]
How to get the output you want:
You're using product() right, but giving it the wrong data.
cities = [c['Value'] for c in data['Cities']] # Extract all cities Value from list-of-dicts
people = [c['Value'] for c in data['People']] # Extract all people Value from list-of-dicts
print(list(itertools.product(cities, people))) # Product
This gives the output:
[
('New York', 'Michael'),
('New York', 'Ryan'),
('New York', 'Jim'),
('New York', 'Phyllis'),
('Boston', 'Michael'),
('Boston', 'Ryan'),
('Boston', 'Jim'),
('Boston', 'Phyllis')
]
If you want the dict objects instead of the Value keys, you can simply pass those objects to product():
print(list(itertools.product(data['Cities'], data['People']))) # Product
which gives
[
({'Id': 1, 'Value': 'New York'}, {'Id': 1, 'Value': 'Michael'}),
({'Id': 1, 'Value': 'New York'}, {'Id': 2, 'Value': 'Ryan'}),
({'Id': 1, 'Value': 'New York'}, {'Id': 3, 'Value': 'Jim'}),
({'Id': 1, 'Value': 'New York'}, {'Id': 4, 'Value': 'Phyllis'}),
({'Id': 2, 'Value': 'Boston'}, {'Id': 1, 'Value': 'Michael'}),
({'Id': 2, 'Value': 'Boston'}, {'Id': 2, 'Value': 'Ryan'}),
({'Id': 2, 'Value': 'Boston'}, {'Id': 3, 'Value': 'Jim'}),
({'Id': 2, 'Value': 'Boston'}, {'Id': 4, 'Value': 'Phyllis'})
]
as expected.
Related
I have a list of dictionary. I want to convert this list into dictionary using parent and child relation. I have try many time. But its difficult for me.
Thanks in advance for solving the problem.
Input =
data = [
{
"_id": 1,
"label": "Property",
"index": 1
},
{
"_id": 2,
"label": "Find Property",
"index": 1,
"parent_id": 1
},
{
"_id": 3,
"label": "Add Property",
"index": 2,
"parent_id": 1
},
{
"_id": 4,
"label": "Offer",
"index": 2
},
{
"_id": 5,
"label": "My Offer",
"index": 1,
"parent_id": 4
},
{
"_id": 6,
"label": "Accept",
"index": 1,
"parent_id": 5
}
]
I have a list of dictionary. I want to convert this list into dictionary using parent and child relation. I have try many time. But its difficult for me.
Thanks in advance for solving the problem.
Expected Output:
[
{
"_id": 1,
"label": "Property",
"index": 1,
"children" : [
{
"_id": 2,
"label": "Find Property",
"index": 1
},
{
"_id": 3,
"label": "Add Property",
"index": 2
}
]
},
{
"_id": 4,
"label": "Offer",
"index": 2,
"children" : [
{
"_id": 5,
"label": "My Offer",
"index": 1,
"children" : [
{
"_id": 6,
"label": "Accept",
"index": 1
}
]
}
]
},
]
I would do it like this. Keep in mind that this solution also affects the original data list.
parents = list()
# First, create a new dict where the key is property id and the value
# is the property itself.
indexed = {d["_id"]:d for d in data}
for id_, item in indexed.items():
# If a property doesn't have "parent_id" key it means that
# this is the root property, appending it to the result list.
if "parent_id" not in item:
parents.append(item)
continue
# Saving parent id for convenience.
p_id = item["parent_id"]
# Adding a children list if a parent doesn't have it yet.
if "children" not in indexed[p_id]:
indexed[p_id]["children"] = list()
indexed[p_id]["children"].append(item)
And the result is:
import pprint
pprint.pprint(parents)
[{'_id': 1,
'children': [{'_id': 2, 'index': 1, 'label': 'Find Property', 'parent_id': 1},
{'_id': 3, 'index': 2, 'label': 'Add Property', 'parent_id': 1}],
'index': 1,
'label': 'Property'},
{'_id': 4,
'children': [{'_id': 5,
'children': [{'_id': 6,
'index': 1,
'label': 'Accept',
'parent_id': 5}],
'index': 1,
'label': 'My Offer',
'parent_id': 4}],
'index': 2,
'label': 'Offer'}]
I have a CSV file that will be imported and converted into a dictionary.
with open(r"DictionaryQuestion.csv", encoding='utf-8-sig') as csvfile:
csvReader = csv.DictReader(csvfile)
for row in map(dict, csvReader):
print(row)
Example Input
I want to be able to transpose the data so that the Discount & NonDiscount rows will be added as columns with their associated amount as well as getting rid of duplicates. Essentially, I want a new dictionary so that I can zip through it.
This is the desired output.
Desired Output as Dictionary
You can use itertools.groupby() to group records by productId and then update your data.
Below I've converted a list which has records same as yours and created new list with data as expected.
data = [
{
"ProductId": "1", "Brand": "Brand1", "rateamount": 1, "rate_type": "Discount"
},
{
"ProductId": "1", "Brand": "Brand1", "rateamount": 2, "rate_type": "NonDiscount"
},
{
"ProductId": "2", "Brand": "Brand2", "rateamount": 3, "rate_type": "Discount"
},
{
"ProductId": "2", "Brand": "Brand2", "rateamount": 4, "rate_type": "NonDiscount"
},
{
"ProductId": "3", "Brand": "Brand3", "rateamount": 5, "rate_type": "Discount"
},
{
"ProductId": "3", "Brand": "Brand3", "rateamount": 6, "rate_type": "NonDiscount"
},
{
"ProductId": "4", "Brand": "Brand4", "rateamount": 7, "rate_type": "Discount"
},
{
"ProductId": "4", "Brand": "Brand4", "rateamount": 2, "rate_type": "NonDiscount"
},
]
Solution
Assuming you data is ordered by productId, otherwise you'll need to order it before grouping.
import itertools
groups = itertools.groupby(data, lambda e: {"ProductId": e["ProductId"], "Brand": e["Brand"]})
output = []
for group, items in groups:
el = dict(group)
for item in items:
if item["rate_type"] == "Discount":
el["Discount"] = item["rateamount"]
else:
el["NonDiscount"] = item["rateamount"]
output.append(el)
print(output)
Above for loop can be converted to a map
import itertools
groups = itertools.groupby(data, lambda e: {"ProductId": e["ProductId"], "Brand": e["Brand"]})
output = map(
lambda group: dict(
**group[0],
**{
item["rate_type"]: item["rateamount"] for item in group[1]
}),
groups
)
print(list(output))
Both prints
[
{'ProductId': '1', 'Brand': 'Brand1', 'Discount': 1, 'NonDiscount': 2},
{'ProductId': '2', 'Brand': 'Brand2', 'Discount': 3, 'NonDiscount': 4},
{'ProductId': '3', 'Brand': 'Brand3', 'Discount': 5, 'NonDiscount': 6},
{'ProductId': '4', 'Brand': 'Brand4', 'Discount': 7, 'NonDiscount': 2}
]
data = {
"persons": {"1": {"name": "siddu"}, "2": {"name": "manju"}},
"cars": {
"model1": {
"make": 1990,
"company_details": {
"name": "Ford Corporation",
"country": "US",
"some_list": [1, 2, 1],
},
},
"model2": {
"make": 1990,
"company_details": {
"name": "Ford Corporation",
"country": "US",
"some_list": [1, 2, 1, 1, 1],
},
},
},
}
This is my python object, How can I identify the Key's-Value is a list. example here, after traversing through 'print(data["cars"]["model1"]["company_details"]["some_list"])'I get the list, since it is small dictionary it was easy, but how can I identify the same if I encounter list as a value for some other key in future.
Example:
data = {
"persons": {"1": {"name": "siddu"}, "2": {"name": "manju"}},
"cars": {
"model1": {
"make": 1990,
"company_details": {
"name": "Ford Corporation",
"country": "US",
"some_list": [1, 2, 1],
},
},
"model2": {
"make": 1990,
"company_details": {
"name": "Ford Corporation",
"country": ["US", "UK", "IND"],
"some_list": [1, 2, 1, 1, 1],
},
},
},
}
Can anyone please suggest/guide me to understand how to identify the key's value is a list.
The final goal is to remove the duplicates in the list if any exists?
Thank you very much:)
You can have a recursive function that goes to any depth and make the items of the list unique like below:
In [8]: def removeDuplicatesFromList(di):
...: for key, val in di.items():
...: if isinstance(val, dict):
...: removeDuplicatesFromList(val)
...: elif isinstance(val, list):
...: di[key] =list(set(val))
...: else:
...: continue
...:
...:
In [9]: removeDuplicatesFromList(data)
In [10]: data
Out[10]:
{'persons': {'1': {'name': 'siddu'}, '2': {'name': 'manju'}},
'cars': {'model1': {'make': 1990,
'company_details': {'name': 'Ford Corporation',
'country': 'US',
'some_list': [1, 2]}},
'model2': {'make': 1990,
'company_details': {'name': 'Ford Corporation',
'country': 'US',
'some_list': [1, 2]}}}}
I have data in python that looks like this:
[['a', 'b', 'c', 50],
['a', 'b', 'd', 100],
['a', 'b', 'e', 67],
['a', 'g', 'c', 12],
['q', 'k', 'c', 11],
['q', 'b', 'p', 11]]
where each element of the list is a complete hierarchical path, and the last element is the size of the path. To do a visualization in D3, I need the data to be in the flare data format - seen here:
https://github.com/d3/d3-hierarchy/blob/master/test/data/flare.json
So a short piece would look like this
{
"name": "root",
"children": [
{
"name": "a",
"children": [
{
"name": "b",
"children": [
{"name": "c", "value": 50},
{"name": "d", "value": 100},
{"name": "e", "value": 67},
]
},
{
"name": "g",
"children": [
{"name": "c", "value": 12},
]
},
and so forth...
From what I've been looking up, I think the solution is recursive, and would use the json library on a Python dictionary, but I can't seem to get it to work. Any help is greatly appreciated.
Here's a solution using recursion:
def add_to_flare(n, flare):
children = flare["children"]
if len(n) == 2:
children.append({"name": n[0], "value": n[1]})
else:
for c in children:
if c["name"] == n[0]:
add_to_flare(n[1:], c)
return
children.append({"name": n[0], "children": []})
add_to_flare(n[1:], children[-1])
flare = {"name": "root", "children": []}
for i in data:
add_to_flare(i, flare)
To display it nicely, we can use the json library:
import json
print(json.dumps(flare, indent=1))
{
"name": "root",
"children": [
{
"name": "a",
"children": [
{
"name": "b",
"children": [
{
"name": "c",
"value": 50
},
{
"name": "d",
"value": 100
},
{
"name": "e",
"value": 67
}
]
},
{
"name": "g",
"children": [
{
"name": "c",
"value": 12
}
]
}
]
},
{
"name": "q",
"children": [
{
"name": "k",
"children": [
{
"name": "c",
"value": 11
}
]
},
{
"name": "b",
"children": [
{
"name": "p",
"value": 11
}
]
}
]
}
]
}
Try this:
master = []
for each in your_list:
head = master
for i in range(len(each)):
names = [e['name'] for e in head]
if i == len(each) - 2:
head.append({'name': each[i], 'value': each[i+1]})
break
if each[i] in names:
head = head[names.index(each[i])]['children']
else:
head.append({'name': each[i], 'children': []})
head = head[-1]['children']
results:
[{'children': [{'children': [{'name': 'c', 'value': 50},
{'name': 'd', 'value': 100},
{'name': 'e', 'value': 67}],
'name': 'b'},
{'children': [{'name': 'c', 'value': 12}], 'name': 'g'}],
'name': 'a'},
{'children': [{'children': [{'name': 'c', 'value': 11}], 'name': 'k'},
{'children': [{'name': 'p', 'value': 11}], 'name': 'b'}],
'name': 'q'}]
Please note that name and children are flipped in this dictionary since it's unordered. But the resulting structure is the same.
put it in root to get your target:
my_dict = {'name':'root', 'children': master}
Assuming your list of lists is stored in variable l, you can do:
o = []
for s in l:
c = o
for i, n in enumerate(['root'] + s[:-1]):
for d in c:
if n == d['name']:
break
else:
c.append({'name': n})
d = c[-1]
if i < len(s) - 1:
if 'children' not in d:
d['children'] = []
c = d['children']
else:
d['value'] = s[-1]
so that o[0] becomes:
{'children': [{'children': [{'children': [{'name': 'c', 'value': 50},
{'name': 'd', 'value': 100},
{'name': 'e', 'value': 67}],
'name': 'b'},
{'children': [{'name': 'c', 'value': 12}],
'name': 'g'}],
'name': 'a'},
{'children': [{'children': [{'name': 'c', 'value': 11}],
'name': 'k'},
{'children': [{'name': 'p', 'value': 11}],
'name': 'b'}],
'name': 'q'}],
'name': 'root'}
I've list of ordered dict that includes some duplicate Ids in data.. something like this
[OrderedDict([('caseId', 20), ('userId', 1), ('emailStatus', 21)]),
OrderedDict([('caseId', 20), ('userId', 1), ('emailStatus', 20)]),
OrderedDict([('caseId', 18), ('userId', 4), ('emailStatus', 21)]),
OrderedDict([('caseId', 19), ('userId', 3), ('emailStatus', 21)]),
OrderedDict([('caseId', 18), ('userId', 1), ('emailStatus', 20)]),
OrderedDict([('caseId', 20), ('userId', 3), ('emailStatus', 21)]),
OrderedDict([('caseId', 18), ('userId', 4), ('emailStatus', 20)]),
OrderedDict([('caseId', 19), ('userId', 1), ('emailStatus', 20)])]
I want to get a list of nested lists, something like this;
[{
"caseId": "20",
"users": [
{
"userId": "1",
"emailStatus": [
{
"emailStatus" : "20"
},
{
"emailStatus" : "21"
}
]
},
{
"userId": "3",
"emailStatus": [
{
"emailStatus" : "21"
}
]
}
]
},
{
"caseId": "19",
"users": [
{
"userId": "1",
"emailStatus": [
{
"emailStatus" : "20"
}
]
},
{
"userId": "3",
"emailStatus": [
{
"emailStatus" : "21"
}
]
}
]
},
{
"caseId": "18",
"users": [
{
"userId": "1",
"emailStatus": [
{
"emailStatus" : "20"
}
]
},
{
"userId": "4",
"emailStatus": [
{
"emailStatus" : "20"
},
{
"emailStatus" : "21"
}
]
}
]
}
]
presenting a nested list like this;
I tried to achieve this by iterating both lists but couldn't get any idea how to keep record of previous and next records and same data.. that's so confusing.. if anyone can give me a start that how I can iterate my list, it would be very kind of you.
Many regards..
Updated Question
More detailed question here
First, you can use a loop and dict.setdefault to group the data in a nested dict:
temp = {}
for d in lst:
temp.setdefault(d["caseId"], {}).setdefault(d["userId"], []).append(d["emailStatus"])
print(temp)
# {18: {1: [20], 4: [21, 20]}, 19: {1: [20], 3: [21]}, 20: {1: [21, 20], 3: [21]}}
Or using a collections.defaultdict:
temp = defaultdict(lambda: defaultdict(list))
for d in lst:
temp[d["caseId"]][d["userId"]].append(d["emailStatus"])
Then, use a nested mixed dict and list comprehension to aggregate your final result:
res = [{"caseId": case, "users": [{"userId": user, "emailStatus": [{"emailStatus": s} for s in status]}
for user, status in users.items()]}
for case, users in temp.items()]
print(res)
# [{'caseId': 18, 'users': [{'userId': 1, 'emailStatus': [{'emailStatus': 20}]}, {'userId': 4, 'emailStatus': [{'emailStatus': 21}, {'emailStatus': 20}]}]},
# {'caseId': 19, 'users': [{'userId': 1, 'emailStatus': [{'emailStatus': 20}]}, {'userId': 3, 'emailStatus': [{'emailStatus': 21}]}]},
# {'caseId': 20, 'users': [{'userId': 1, 'emailStatus': [{'emailStatus': 21}, {'emailStatus': 20}]}, {'userId': 3, 'emailStatus': [{'emailStatus': 21}]}]}]