I'm trying to create a dictionary with some specific values but it gets the same values multiple times:
readers = Readers.objects.all()
count = 0
readersResult = {}
test = {
"avatar": "",
"firstName": "",
"percent": "",
"lastName": ""
}
for reader in readers:
test["percent"] = "value from another place"
test["firstName"] = reader.firstName
test["lastName"] = reader.lastName
test["avatar"] = reader.avatar
print("TEST: ", test)
readersResult[count] = test
count = count + 1
print("RESULT":, readersResult)
My output is:
web_1 | TEST: {'avatar': '/images/avatars/71.png', 'firstName': 'abc', 'percent': '37.08999158957107', 'lastName': 'def'}
web_1 | TEST: {'avatar': '/images/avatars/61.png', 'firstName': 'abc', 'percent': '4.037005887300253', 'lastName': 'def'}
web_1 | RESULT: {0: {'avatar': '/images/avatars/61.png', 'firstName': 'abc', 'percent': '4.037005887300253', 'lastName': 'def'}, 1: {'avatar': '/images/avatars/61.png', 'firstName': 'abc', 'percent': '4.037005887300253', 'lastName': 'def'}}
What am I doing wrong?
Here, in your code, you make a dict named test, and then in each loop iteration you add it again to the readersResult.
test = {
"avatar": "",
"firstName": "",
"percent": "",
"lastName": ""
}
for reader in readers:
test["percent"] = "value from another place"
test["firstName"] = reader.firstName
test["lastName"] = reader.lastName
test["avatar"] = reader.avatar
print("TEST: ", test)
readersResult[count] = test
count = count + 1
You need to create a new dict with each loop iteration. So exchange this part of your code to this:
for reader in readers:
test = {}
test["percent"] = "value from another place"
test["firstName"] = reader.firstName
test["lastName"] = reader.lastName
test["avatar"] = reader.avatar
print("TEST: ", test)
readersResult[count] = test
count = count + 1
PS: You don't need to use the below code to define the keys in the dictionary and then set its values somewhere else. So you can change:
teste = {
"avatar": [],
"firstName": [],
"percent": [],
"lastName": []
}
with:
test = {}
You are assigning the same dictionary (test) to multiple entries in readersResult, as opposed to making a new one for each reader.
You can use a list as the dictionary value:
readers = Readers.objects.all()
count = 0
readersResult = {}
test = {
"avatar": [],
"firstName": [],
"percent": [],
"lastName": []
}
for reader in readers:
test["percent"].append("value from another place")
test["firstName"].append(reader.firstName)
test["lastName"].append(reader.lastName)
test["avatar"].append(reader.avatar)
print("TEST: ", test)
readersResult[count] = test
count = count + 1
print("RESULT":, readersResult)
Examples can be found here.
Related
Given a Pandas table of thousands of rows, where the left most spaces of a row determine if it's a sub structure of the above row or not.
Parameter | Value
'country' 'Germany'
' city' 'Berlin'
' area' 'A1'
' city' 'Munchen'
' comment' 'a comment'
'country' 'France'
' city' 'Paris'
' comment' 'a comment'
'state' 'California'
' comment' '123'
Where I have information about if a parameter is a list or not.
{
'country': list,
'city': list
'state': list
}
I would want to create the following nested structure
{
"country": [
{
"Germany": {
"city": [
{
"Berlin": {
"area": "A1"
}
},
{
"Munchen": {
"comment": "a comment"
}
}
]
}
},
{
"France": {
"city": [
{
"Paris": {
"comment": "a comment"
}
}
]
}
}
],
"state": [
{
"California": {
"comment": 123
}
}
]
}
Since the knowledge about what level the sub structure depends on only the row before, I thought that a for loop would be good. But I am clearly missing something fundamental about creating nested dictionaries using for loops. It could be a recursive solution as well, but I am unsure if it would be easier here.
This is my current attempt which is obviously a mess.
import pandas as pd
params = ['country',' city',' area',' city',' comment','country',' city',' comment','state',' comment']
vals = ['Germany','Berlin','A1','Munich','acomment','France','Paris','acomment','California','123']
conf = {'country':'list','city':'list'}
df = pd.DataFrame()
df['param'] = params
df['vals']= vals
output_dict = dict()
level_path = dict()
for param,vals in df.values:
d = output_dict
hiearchy_level = sum( 1 for _ in itertools.takewhile(str.isspace,param)) ## Count number of left most spaces
param = param.lstrip()
if hiearchy_level > 0:
base_path = level_path[str(hiearchy_level-1)]
else:
base_path = []
path = base_path + [param]
for p in path:
if p in conf: ## It should be a list
d.setdefault(p,[{}])
d = d[p][-1] ## How to understand if I should push a new list element or enter an existing one?
else:
d.setdefault(p,{})
d = d[p]
d[param] = vals
level_path[str(hiearchy_level)] = path
and the output being
{'country': [{'country': 'France',
'city': [{'city': 'Paris',
'area': {'area': 'A1'},
'comment': {'comment': 'a comment'}}]}],
'state': {'state': 'California', 'comment': {'comment': '123'}}}
I don't understand how I should be able to step in and out of the list elements in the for loop, knowing if I should push a new dictionary or enter an already existing one.
Any input on what I am missing would be appreciated.
I am trying to create one nested Python dictionary called Results.
I am using AWS Rekognition to get an image and output the results.
The results_dict only contains one result after it's complated, and I wish to have all the results in one nested loop
I'm trying to get:
{
"Results": [
{
"Name": "Human",
"Confidence": 98.87621307373047,
},
{
"Name": "Face",
"Confidence": 98.87621307373047,
},
{
"Name": "Person",
"Confidence": 98.87621307373047,
},
]
}
But I'm getting:
{
'Results':
{
'Name': 'Paper',
'Confidence': 57.299766540527344
}
}
The code is replacing the text, and I want to add another set of Name and Confidence.
My code is:
import boto3
import json
BUCKET = "*****"
FOLDER = 'testing/'
JOEY = FOLDER + "Joey_30_Sept.png"
BEYONCE = FOLDER + "beyonce_rekognition_moderation_testing.jpg"
MANBEARD = FOLDER + "man_beard.jpg"
MEN = FOLDER + "men_group.jpg"
client = boto3.client('rekognition')
response = client.detect_labels(Image=
{'S3Object': {
'Bucket': BUCKET,
'Name': JOEY
}},
MaxLabels = 10,
MinConfidence=0)
results_dict = {}
results_dict['Results'] = {}
results_dict['Results']['Name'] = ""
results_dict['Results']['Confidence'] = ""
for label in response['Labels']:
name = label['Name'] #to get the whole bounding box.
confidence = label['Confidence']
name_str = str(name)
conf_str = str(confidence)
results_dict["Results"]["Name"] = label['Name']
results_dict["Results"]["Confidence"] = label['Confidence']
print(results_dict)
You defined results_dict['Results'] as dictionary as dict not list:
...
results_dict = {}
results_dict['Results'] = []
results_dict['Results']['Name'] = ""
results_dict['Results']['Confidence'] = ""
for label in response['Labels']:
name = label['Name'] #to get the whole bounding box.
confidence = label['Confidence']
name_str = str(name)
conf_str = str(confidence)
results_dict['Results'].append({["Name": name_str, "Confidence": conf_str })
print(results_dict)
I have a cte query that returns me results of values that are linked (i.e. child -> parent).
Then in Python I am trying to create a nested dictionary that would represent something like this:
{
"name": "Child_example",
"parents": [
{
"name": "child_parent_1",
"parents": [{"name": "child_parent_1_parent", "parents": [{"name": "end", "parents": []}]}]
},
{
"name": "child_parent_2",
"parents": [{"name": "end", "parents": []}]
},
{
"name": "child_parent_3",
"parents": [{"name": "child_parent_3_parent", "parents": [{"name": "end", "parents": []}]}]
}
]
}
My input data looks something like so (it can have more data):
child_col
parent_col
name
depth
Child_example
child_parent_1_col
child_parent_1
0
Child_example
child_parent_2_col
child_parent_2
0
Child_example
child_parent_3_col
child_parent_3
0
child_parent_1_col
child_parent_1_parent
1_parent
1
child_parent_2_col
end
1_parent
1
child_parent_3_col
child_parent_3_parent
3_parent
1
child_parent_3_parent
end
end_3
2
child_parent_1_parent
end
end_1
2
However with my code so far:
r_dict = defaultdict(list)
depth_zero = [x for x in rows if x.depth == 0]
for row in depth_zero:
r_dict['name'] = row.path_key
r_dict['parents'].append({'name': row.path_parent_key, 'parents': []})
depth_not_zero = [x for x in rows if x.depth != 0]
# Set inner levels
for parent in r_dict['parents']:
name = parent['name']
inner_parent = parent['parents'].copy()
for row in depth_not_zero:
if row.path_key == name:
inner_parent.append({'name': row.path_parent_key, 'parents': []})
name = row.path_parent_key
parent['parents'] = inner_parent
I only manage to achieve to append it to initial "parents", instead of setting the ultimate nested "parents". I know it is to do with this line of code:
inner_parent.append({'name': row.path_parent_key, 'parents': []})
But I cannot work out how to essentially get and set it. Would this be a case for recursion instead of the way I am doing it?
Below is an example of the first nested dictionary output that I am currently creating with my code:
{
"name": "Child_example",
"parents": [
{
"name": "child_parent_1",
"parents": [
{"name": "child_parent_1", "parents": []}, {"name": "end", "parents": []}
]
}
]
}
I'm a bit baffled by the way you are assigning the "name" value: "Child_example" comes from child_col, "child_parent_1" from name, "child_parent_3_parent" from parent_col. So I simplified it a bit: I put in the second column of the child row the same value as in the first column of its parents rows. That said, if you really need to take the names from different columns it's just a matter of adding some ifs.
My proposal is to loop over the rows in reverse order, creating the inner dicts and then moving them into the outer ones:
rows = [["c1","p1c1",0],
["c1","p2c1",0],
["c1","p3c1",0],
["p1c1","p1p1c1",1],
["p2c1","end",1],
["p3c1","p1p3c1",1],
["p1p3c1","end",2],
["p1p1c1","end",2]]
r_dict = {}
for row in reversed(rows):
if row[1] == "end":
r_dict[row[0]] = {"name":row[0], "parents":[]}
else:
if not row[0] in r_dict:
r_dict[row[0]] = {"name":row[0], "parents":[]}
r_dict[row[0]]["parents"].append(r_dict[row[1]])
del r_dict[row[1]]
r_dict
{'c1': {'name': 'c1', 'parents': [{'name': 'p3c1', 'parents': [{'name': 'p1p3c1', 'parents': []}]}, {'name': 'p2c1', 'parents': []}, {'name': 'p1c1', 'parents': [{'name': 'p1p1c1', 'parents': []}]}]}}
I'm trying to extract values from a nested dictionary if a value matches the value in a list.
data = [
{
"id": 12345678,
"list_id": 12345,
"creator_id": 1234567,
"entity_id": 1234567,
"created_at": "2020-01-30T00:43:55.256-08:00",
"entity": {
"id": 123456,
"type": 0,
"first_name": "John",
"last_name": "Doe",
"primary_email": "john#fakemail.com",
"emails": [
"john#fakemail.com"
]
}
},
{
"id": 12345678,
"list_id": 12345,
"creator_id": 1234567,
"entity_id": 1234567,
"created_at": "2020-01-30T00:41:54.375-08:00",
"entity": {
"id": 123456,
"type": 0,
"first_name": "Jane",
"last_name": "Doe",
"primary_email": "jane#fakemail.com",
"emails": [
"jane#fakemail.com"
]
}
}
]
The code is as follows.
match_list = ['jane#fakemail.com',[]]
first_names = []
email = []
for i in match_list:
for record in data:
if 'primary_email' == i:
email.append(record.get('entity',{}).get('primary_email', None))
first_names.append(record.get('entity',{}).get('first_name', None))
print(first_names)
print(email)
Instead of returning the matching values this only returns empty lists. Any help here would be much appreciated.
The expected output is
first_names = ['Jane'] and email = ['jane#fakemail.com']
Store temporary values in variables, to make your code easier to handle:
emails = []
names = []
match_list = ['jane#fakemail.com',[]]
for item in data:
entry = item.get('entity', {})
fName = entry.get('first_name', '')
pMail = entry.get('primary_email', '')
if pMail in match_list:
print (fName)
print (pMail)
emails.append(pMail)
names.append(fName)
Output:
Jane
jane#fakemail.com
In the 6th line of your code
if 'primary_email' == i:
You're comparing elements from match_list (that is 'i') to literally the string called 'primary_email' instead of the actual email. since 'jane#fakemail.com' is not equal to 'primary_email' (literally the string).
Instead use
if record['entity']['primary_email'] == i:
and your code should work as expected.
In your code you would always get an empty list as you are comparing 'primary_email'==i which will always be False.
Change it to record['entity']['primary_email']==i.
And here there is no need to use get.Since if mail doesn't match with any of the primary_email then nothing happens. primary_email will only be added when it meets the condition d['entity']['primary_email']==mail.
Try this I refactored your code little bit.
In [25]: for mail in match_list:
...: for d in data:
...: if d['entity']['primary_email']==mail:
...: first_name.append(d['entity']['first_name'])
...: emails.append(d['entity']['primary_email'])
output
In [26]: emails
Out[26]: ['jane#fakemail.com']
In [27]: first_name
Out[27]: ['Jane']
I have a CSV file named data.csv. I am converting CSV file in to nested JSON and inserting it in to mongodb using python.Below is the code, but here i wanna label FirstName and LastName under the parent called 'Name'. Can somebody help?
import json
import pandas as pd
from pymongo import MongoClient
try:
conn = MongoClient()
print("Connected successfully!!!")
except:
print("Could not connect to MongoDB")
# database
db = conn.database
collection = db.collection3
df = pd.read_csv(r'C:\Users\swetha1\Desktop\data.csv')
def get_nested_rec(key, grp):
rec = {}
rec['PrimaryId'] = key[0]
rec['FirstName'] = key[1]
rec['LastName'] = key[2]
rec['City'] = key[3]
for field in ['CarName','DogName']:
rec[field] = list(grp[field].unique())
return rec
records = []
for key, grp in df.groupby(['PrimaryId','FirstName','LastName','City']):
rec = get_nested_rec(key, grp)
records.append(rec)
records = dict(data = records)
r=json.dumps(records,default=int,indent=4)
my_dict = json.loads(r)
print(my_dict)
collection.insert(my_dict)
print('inserted')
the above code converts CSV to nested JSON
output converting CSV to nested JSON is as follows:
Connected successfully!!!
{
"data": [
{
"PrimaryId": 100,
"FirstName": "John",
"LastName": "Smith",
"City": "NewYork",
"CarName": [
"Toyota",
"BMW"
],
"DogName": [
"Spike",
"Rusty"
]
},
{
"PrimaryId": 101,
"FirstName": "Ben",
"LastName": "Swan",
"City": "Sydney",
"CarName": [
"Volkswagen",
"Ford",
"Audi"
],
"DogName": [
"Buddy",
"Max"
]
},
{
"PrimaryId": 102,
"FirstName": "Julia",
"LastName": "Brown",
"City": "London",
"CarName": [
"Mini"
],
"DogName": [
"Lucy"
]
}
]
}
desired output structure should be something like this:
PS: i have given blank values here just for sample.
"info":[
{ "primaryId":" "
"City":" "
"Name":
{ "FirstName":" "
"LastName" :" "
}
"CarName":
{ "car1": " "
"car2": " "
}
"DogName":
{ "Dog1": " "
"Dog2": " "
}
}]
data.CSV
PrimaryId,FirstName,LastName,City,CarName,DogName
100,John,Smith,NewYork,Toyota,Spike
100,John,Smith,NewYork,BMW,Spike
100,John,Smith,NewYork,Toyota,Rusty
100,John,Smith,NewYork,BMW,Rusty
101,Ben,Swan,Sydney,Volkswagen,Buddy
101,Ben,Swan,Sydney,Ford,Buddy
101,Ben,Swan,Sydney,Audi,Buddy
101,Ben,Swan,Sydney,Volkswagen,Max
101,Ben,Swan,Sydney,Ford,Max
101,Ben,Swan,Sydney,Audi,Max
102,Julia,Brown,London,Mini,Lucy
This is one approach.
Demo:
import pandas as pd
df = pd.read_csv(filename)
def get_nested_rec(key, grp):
rec = {}
rec['PrimaryId'] = key[0]
rec['City'] = key[3]
rec["Name"] = [{'FirstName': key[1], 'LastName': key[2]}] #Update.
for field in ['CarName','DogName']:
rec[field] = list(grp[field].unique())
return rec
records = []
for key, grp in df.groupby(['PrimaryId','FirstName','LastName','City']):
rec = get_nested_rec(key, grp)
records.append(rec)
records = dict(data = records)
print(records)
Output:
{'data': [{'CarName': ['Toyota', 'BMW'],
'City': 'NewYork',
'DogName': ['Spike', 'Rusty'],
'Name': [{'FirstName': 'John', 'LastName': 'Smith'}],
'PrimaryId': 100},
{'CarName': ['Volkswagen', 'Ford', 'Audi'],
'City': 'Sydney',
'DogName': ['Buddy', 'Max'],
'Name': [{'FirstName': 'Ben', 'LastName': 'Swan'}],
'PrimaryId': 101},
{'CarName': ['Mini'],
'City': 'London',
'DogName': ['Lucy'],
'Name': [{'FirstName': 'Julia', 'LastName': 'Brown'}],
'PrimaryId': 102}]}