We Keep Coding

Add an object to every json array in python

I'm trying to figure out how to add the same object to every array.
I'm requesting data from the server for the "first game". When I get it back, it doesn't include any data referencing the first game. So I need to edit it before I send it to my server to save.
I have a json request that looks like this:
{
'dateTime': '2022-07-01T01:00:00.000000',
'httpStatus': 'OK',
'message': 'SUCCESS',
'details': None,
'detailsList': [
{
'date': '2021-07-01T00:00:00',
'tcount': 0,
'first_name': 'Sam',
'last_name': 'Smith'
},
{
'user_reg_date': '2022-06-01T00:00:00',
'tcount': 0,
'first_name': 'Bob',
'last_name': 'Jones'
}]
}
I'm trying to figure out how to add an object to each json array (hope I'm saying that the right way) before I then send it to a mongodb.
In this example:
'game': 'first'
It would then look like this:
{
'dateTime': '2022-07-01T01:00:00.000000',
'httpStatus': 'OK',
'message': 'SUCCESS',
'details': None,
'detailsList': [
{
'date': '2021-07-01T00:00:00',
'tcount': 0,
'first_name': 'Sam',
'last_name': 'Smith',
'game': 'first'
},
{
'user_reg_date': '2022-06-01T00:00:00',
'tcount': 0,
'first_name': 'Bob',
'last_name': 'Jones',
'game': 'first'
}]
}
if there is a better way to do this, that would work as well.

Your asking something similar to this question. But want to loop through a JSON array in Python.
You are not trying to add an 'object' to json 'array'. You have a JSON object, of which there are object members (or more commonly, properties). You have an array as an object member of which you want to modify each object in the array to add a new property.
The code below is the above link with some modification to fit your needs. request_data is the JSON object you give in your example above. You need enumerate to know which index to edit.
for index, item in enumerate(request_data['detailsList']):
# Edit the array entry
item["game"] = 'first'
# Save it
request_data['detailsList'][index] = item

Convert XML to List of Dictionaries in python

I'm very new to python and please treat me as same.
When i tried to convert the XML content into List of Dictionaries I'm getting output but not as expected and tried a lot playing around.
XML Content:
<project>
<panelists>
<panelist panelist_login="pradeep">
<login/>
<firstname/>
<lastname/>
<gender/>
<age>0</age>
</panelist>
<panelist panelist_login="kumar">
<login>kumar</login>
<firstname>kumar</firstname>
<lastname>Pradeep</lastname>
<gender/>
<age>24</age>
</panelist>
</panelists>
</project>
Code i have used:
import xml.etree.ElementTree as ET
tree = ET.parse(xml_file.xml) # import xml from
root = tree.getroot()
Panelist_list = []
for item in root.findall('./panelists/panelist'): # find all projects node
Panelist = {} # dictionary to store content of each projects
panelist_login = {}
panelist_login = item.attrib
Panelist_list.append(panelist_login)
for child in item:
Panelist[child.tag] = child.text
Panelist_list.append(Panelist)
print(Panelist_list)
Output:
[{
'panelist_login': 'pradeep'
}, {
'login': None,
'firstname': None,
'lastname': None,
'gender': None,
'age': '0'
}, {
'panelist_login': 'kumar'
}, {
'login': 'kumar',
'firstname': 'kumar',
'lastname': 'Pradeep',
'gender': None,
'age': '24'
}]
and I'm Expecting for the below Output
[{
'panelist_login': 'pradeep',
'login': None,
'firstname': None,
'lastname': None,
'gender': None,
'age': '0'
}, {
'panelist_login': 'kumar'
'login': 'kumar',
'firstname': 'kumar',
'lastname': 'Pradeep',
'gender': None,
'age': '24'
}]
I have refereed so many stack overflow questions on xml tree but still didn't helped me.
any help/suggestion is appreciated.

Your code is appending the dict panelist_login with the tag attributes to the list, in this line: Panelist_list.append(panelist_login) separately from the Panelist dict. So for every <panelist> tag the code appends 2 dicts: one dict of tag attributes and one dict of subtags. Inside the loop you have 2 append() calls, which means 2 items in the list for each time through the loop.
But you actually want a single dict for each <panelist> tag, and you want the tag attribute to appear inside the Panelist dict as if it were a subtag also.
So have a single dict, and update the Panelist dict with the tag attributes instead of keeping the tag attributes in a separate dict.
for item in root.findall('./panelists/panelist'): # find all projects node
Panelist = {} # dictionary to store content of each projects
panelist_login = item.attrib
Panelist.update(panelist_login) # make panelist_login the first key of the dict
for child in item:
Panelist[child.tag] = child.text
Panelist_list.append(Panelist)
print(Panelist_list)
I get this output, which I think is what you had in mind:
[
{'panelist_login': 'pradeep',
'login': None,
'firstname': None,
'lastname': None,
'gender': None,
'age': '0'},
{'panelist_login': 'kumar',
'login': 'kumar',
'firstname': 'kumar',
'lastname': 'Pradeep',
'gender': None,
'age': '24'}
]

how to use nested dictionary in python?

I am trying to write some code with the Hunter.io API to automate some of my b2b email scraping. It's been a long time since I've written any code and I could use some input. I have a CSV file of Urls, and I want to call a function on each URL that outputs a dictionary like this:
`{'domain': 'fromthebachrow.com', 'webmail': False, 'pattern': '{f}{last}', 'organization': None, 'emails': [{'value': 'fbach#fromthebachrow.com', 'type': 'personal', 'confidence': 91, 'sources': [{'domain': 'fromthebachrow.com', 'uri': 'http://fromthebachrow.com/contact', 'extracted_on': '2017-07-01'}], 'first_name': None, 'last_name': None, 'position': None, 'linkedin': None, 'twitter': None, 'phone_number': None}]}`
for each URL I call my function on. I want my code to return just the email address for each key labeled 'value'.
Value is a key that is contained in a list that itself is an element of the directory my function outputs. I am able to access the output dictionary to grab the list that is keyed to 'emails', but I don't know how to access the dictionary contained in the list. I want my code to return the value in that dictionary that is keyed with 'value', and I want it to do so for all of my urls.
from pyhunyrt import PyHunter
import csv
file=open('urls.csv')
reader=cvs.reader (file)
urls=list(reader)
hunter=PyHunter('API Key')
for item in urls:
output=hunter.domain_search(item)
output['emails'`
which returns a list that looks like this for each item:
[{
'value': 'fbach#fromthebachrow.com',
'type': 'personal',
'confidence': 91,
'sources': [{
'domain': 'fromthebachrow.com',
'uri': 'http://fromthebachrow.com/contact',
'extracted_on': '2017-07-01'
}],
'first_name': None,
'last_name': None,
'position': None,
'linkedin': None,
'twitter': None,
'phone_number': None
}]
How do I grab the first dictionary in that list and then access the email paired with 'value' so that my output is just an email address for each url I input initially?

To grab the first dict (or any item) in a list, use list[0], then to grab a value of a key value use ["value"]. To combine it, you should use list[0]["value"]

Sorting a list of dictionaries by all keys being unique

Pulling my hair out with this one.
I have a list of dictionaries without a unique primary ID key for each unique entry (the dictionary is built on the fly):
dicts = [{'firstname': 'john', 'lastname': 'doe', 'code': 'crumpets'},
{'firstname': 'john', 'lastname': 'roe', 'code': 'roe'},
{'firstname': 'john', 'lastname': 'doe', 'code': 'crumpets'},
{'firstname': 'thom', 'lastname': 'doe', 'code': 'crumpets'},
]
How do I go about filtering out lists of dictionaries like this where any repeating {} within the list are removed? So I need to check if all three of the dictionary keys match up with another in the list...and then discard that from the dict if that check is met.
So, for my example above, the first and third "entries" need to be removed as they are duplicates.

You use create frozensets from the dicts and put those in a set to remove dupes:
dcts = [dict(d) for d in set(frozenset(d.items()) for d in dcts)]
print(dcts)
[{'code': 'roe', 'firstname': 'john', 'lastname': 'roe'},
{'code': 'crumpets', 'firstname': 'thom', 'lastname': 'doe'},
{'code': 'crumpets', 'firstname': 'john', 'lastname': 'doe'}]
If you choose to remove all entries of the duplicates you can use a counter:
from collections import Counter
dcts = [dict(d) for d, cnt in Counter(frozenset(d.items()) for d in dcts).items()
if cnt==1]
print(dcts)
[{'code': 'roe', 'firstname': 'john', 'lastname': 'roe'},
{'code': 'crumpets', 'firstname': 'thom', 'lastname': 'doe'}]

Remove duplicates in a list of non-hashable elements requires you to make them hashable on the fly:
def remove_duplicated_dicts(elements):
seen = set()
result = []
for element in elements:
element_as_tuple = tuple(element.items())
if element_as_tuple not in seen:
seen.add(element_as_tuple)
result.append(element)
return result
d = [{'firstname': 'john', 'lastname': 'doe', 'code': "crumpets"},
{'firstname': 'john', 'lastname': 'roe', 'code': "roe"},
{'firstname': 'john', 'lastname': 'doe', 'code': "crumpets"},
{'firstname': 'thom', 'lastname': 'doe', 'code': "crumpets"},
]
print(remove_duplicated_dicts(d))
PS.
Non-obvious differences with the accepted answer of Moses Koledoye (as of 2017-06-19 at 13:00:00):
preservation of the original list order;
faster conversions: dict -> tuple instead of dict -> frozendict -> dict (take it with a grain of salt: I have made no benchmark).

Given the values of the dictionary are hashable, we can generate our own uniqness filter:
def uniq(iterable, key = lambda x:x):
keys = set()
for item in iterable:
ky = key(item)
if ky not in keys:
yield item
keys.add(ky)
We can then simply use the filter, like:
list(uniq(dicts,key=lambda x:(x['firstname'],x['lastname'],x['code'])))
The filter maintains the original order, and will - for this example - generate:
>>> list(uniq(dicts,key=lambda x:(x['firstname'],x['lastname'],x['code'])))
[{'code': 'crumpets', 'firstname': 'john', 'lastname': 'doe'},
{'code': 'roe', 'firstname': 'john', 'lastname': 'roe'},
{'code': 'crumpets', 'firstname': 'thom', 'lastname': 'doe'}]

Printing dictornary with list order

I have the following code, it is an address book (a list of dictionaries) and a list. The idea is to print the items from the dictionary in the order of the list (since dicts mess up the order of everything). When I run this code, it throws an exception (as shown below). I am not sure what I'm doing wrong as I have tried many different variations and I keep coming back to this wondering why it's not working.
Please help?
addressBook = [
{
'Nickname': 'Jimmy',
'Name': 'James Roberts',
'Address': '2/50 Robe Street',
'Phone': '0273503342'
},
{
'Nickname': 'Bob',
'Name': 'Robert',
'Address': '1 Vivan Street',
'Phone': '067578930'
}
]
addressFields = ['Nickname', 'Name', 'Address', 'Phone']
def listAll(addressBook, addressFields):
for i in addressBook:
for key in addressFields:
print("{0} {1}".format(key, addressBook[i][key]))
print("{0} {1}".format(key, addressBook[i][key]))
TypeError: list indices must be integers, not dict

First, you're missing a ' in the addressBook literal after 'James Roberts. Second, the issue was that you were doing addressBook[i][key] instead of i[key]. i already refers to a dictionary contained in addressBook, so your code was trying to use a list's element as an index to itself.
def listAll(addressBook, addressFields):
for i in addressBook:
for key in addressFields:
print('{} {}'.format(key, i[key]))
Python 3-friendly one-liner:
def listAll(addressBook, addressFields):
print(*('{} {}'.format(j, i[j]) for i in addressBook for j in addressFields), sep='\n')

Alternatively in a single line:
print('\n'.join(element for element in [j+" "+ i[j] for i in addressBook for j in addressFields]
))

#!/usr/bin/python
addressBook = [{'Nickname': 'Jimmy', 'Name': 'James Roberts', 'Address': '2/50 Robe Street', 'Phone': '0273503342'},{'Nickname': 'Bob', 'Name': 'Robert', 'Address': '1 Vivan Street', 'Phone': '067578930'}]
addressFields = ['Nickname', 'Name', 'Address', 'Phone']
def listAll(addressBook, addressFields):
for i in addressBook:
for val in addressFields:
print("{0} {1}".format(val, i[val]))
listAll(addressBook, addressFields)

After posting this question I found the answer.
I converted the addressBook for loop to a range(len(addressBook)) and it worked.
for i in range(len(addressBook)):