We Keep Coding

Writing a GET functionality for json-like data in Python

I am working on a coding challenge for self-development and I came across a question where I am given an input like this:
add {"id":1,"last":"Doe","first":"John","location":{"city":"Oakland","state":"CA","postalCode":"94607"},"active":true}
add {"id":2,"last":"Doe","first":"Jane","location":{"city":"San Francisco","state":"CA","postalCode":"94105"},"active":true}
add {"id":3,"last":"Black","first":"Jim","location":{"city":"Spokane","state":"WA","postalCode":"99207"},"active":true}
add {"id":4,"last":"Frost","first":"Jack","location":{"city":"Seattle","state":"WA","postalCode":"98204"},"active":false}
get {"location":{"state":"WA"},"active":true}
get {"id":1}
get {"active":true}
delete {"active":true}
get {}
And what I am doing is adding the entries that start with add to a list called database = []:
json_input = []
database = []
for line in sys.stdin:
json_input.append(line.split("', "))
for i in range(0, len(json_input)):
if json_input[i][0] == 'add':
database.append(json_input[i][1])
What I want to do is to print out every entry that matches what follows get and delete every entry that matches what follows delete. This is where I am stuck. Currently, this is what json_input() looks like. database is empty:
[
['add {"id":1,"last":"Doe","first":"John","location":{"city":"Oakland","state":"CA","postalCode":"94607"},"active":true}\n'],
['add {"id":2,"last":"Doe","first":"Jane","location":{"city":"San Francisco","state":"CA","postalCode":"94105"},"active":true}\n'],
['add {"id":3,"last":"Black","first":"Jim","location":{"city":"Spokane","state":"WA","postalCode":"99207"},"active":true}\n'],
['add {"id":4,"last":"Frost","first":"Jack","location":{"city":"Seattle","state":"WA","postalCode":"98204"},"active":false}\n'],
['get {"location":{"state":"WA"},"active":true}\n'], ['get {"id":1}\n'],
['get {"active":true}\n'], ['delete {"active":true}\n'],
['get {}']
]

Perhaps an easy-to-read way to handle this would be a simple class that maintains a list of records. You can add methods for the various commands you want to handle. Then it's just a matter of defining the methods and processing the input to pass to the methods. Here's a possible way (without any frills like error checking):
import json
raw_data = '''add {"id":1,"last":"Doe","first":"John","location":{"city":"Oakland","state":"CA","postalCode":"94607"},"active":true}
add {"id":2,"last":"Doe","first":"Jane","location":{"city":"San Francisco","state":"CA","postalCode":"94105"},"active":true}
add {"id":3,"last":"Black","first":"Jim","location":{"city":"Spokane","state":"WA","postalCode":"99207"},"active":true}
add {"id":4,"last":"Frost","first":"Jack","location":{"city":"Seattle","state":"WA","postalCode":"98204"},"active":false}
get {"location":{"state":"WA"},"active":true}
get {"id":1}
get {"active":true}
delete {"active":true}
get {}'''
class Data:
#staticmethod
def matches(obj, query):
if not isinstance(query, dict):
return obj == query
return all(Data.matches(obj.get(key), q) for key, q in query.items())
def __init__(self):
self.data = []
def add(self, record):
self.data.append(record)
def get(self, query):
for item in self.data:
if (Data.matches(item, query)):
print(item)
def delete(self, query):
self.data = [record for record in self.data if not Data.matches(record, query)]
data = Data()
for line in raw_data.split('\n'):
command, line = line.split(None, 1)
command = getattr(data, command)
command(json.loads(line))
This will print the records from WA then the active:True records. Then after deleting the True records it will print everything (the result of the {} query), which is the only one left -- the active:False record:
{'id': 3, 'last': 'Black', 'first': 'Jim', 'location': {'city': 'Spokane', 'state': 'WA', 'postalCode': '99207'}, 'active': True}
{'id': 1, 'last': 'Doe', 'first': 'John', 'location': {'city': 'Oakland', 'state': 'CA', 'postalCode': '94607'}, 'active': True}
{'id': 1, 'last': 'Doe', 'first': 'John', 'location': {'city': 'Oakland', 'state': 'CA', 'postalCode': '94607'}, 'active': True}
{'id': 2, 'last': 'Doe', 'first': 'Jane', 'location': {'city': 'San Francisco', 'state': 'CA', 'postalCode': '94105'}, 'active': True}
{'id': 3, 'last': 'Black', 'first': 'Jim', 'location': {'city': 'Spokane', 'state': 'WA', 'postalCode': '99207'}, 'active': True}
{'id': 4, 'last': 'Frost', 'first': 'Jack', 'location': {'city': 'Seattle', 'state': 'WA', 'postalCode': '98204'}, 'active': False}
If this were a test or a serious coding challenge, you would probably want to look carefully at matches() to make sure it properly handles edge cases (I didn't do that).

Problem when saving json data into python list

I'm trying to get two attributes at the time from my json data and add them as an item on my python list. However, when trying to add those two: ['emailTypeDesc']['createdDate'] it throws an error. Could someone help with this? thanks in advance!
json:
{
'readOnly': False,
'senderDetails': {'firstName': 'John', 'lastName': 'Doe', 'emailAddress': 'johndoe#gmail.com', 'emailAddressId': 123456, 'personalId': 123, 'companyName': 'ACME‘},
'clientDetails': {'firstName': 'Jane', 'lastName': 'Doe', 'emailAddress': 'janedoe#gmail.com', 'emailAddressId': 654321, 'personalId': 456, 'companyName': 'Lorem Ipsum‘}},
'notesSection': {},
'emailList': [{'requestId': 12345667, 'emailId': 9876543211, 'emailType': 3, 'emailTypeDesc': 'Email-In', 'emailTitle': 'SampleTitle 1', 'createdDate': '15-May-2020 11:15:52', 'fromMailList': [{'firstName': 'Jane', 'lastName': 'Doe', 'emailAddress': 'janedoe#gmail.com',}]},
{'requestId': 12345667, 'emailId': 14567775, 'emailType': 3, 'emailTypeDesc': 'Email-Out', 'emailTitle': 'SampleTitle 2', 'createdDate': '16-May-2020 16:15:52', 'fromMailList': [{'firstName': 'Jane', 'lastName': 'Doe', 'emailAddress': 'janedoe#gmail.com',}]},
{'requestId': 12345667, 'emailId': 12345, 'emailType': 3, 'emailTypeDesc': 'Email-In', 'emailTitle': 'SampleTitle 3', 'createdDate': '17-May-2020 20:15:52', 'fromMailList': [{'firstName': 'Jane', 'lastName': 'Doe', 'emailAddress': 'janedoe#gmail.com',}]
}
python:
final_list = []
data = json.loads(r.text)
myId = [(data['emailList'][0]['requestId'])]
for each_req in myId:
final_list.append(each_req)
myEmailList = [mails['emailTypeDesc']['createdDate'] for mails in data['emailList']]
for each_requ in myEmailList:
final_list.append(each_requ)
return final_list
This error comes up when I run the above code:
TypeError: string indices must be integers
Desired output for final_list:
[12345667, 'Email-In', '15-May-2020 11:15:52', 'Email-Out', '16-May-2020 16:15:52', 'Email-In', '17-May-2020 20:15:52']
My problem is definetely in this line:
myEmailList = [mails['emailTypeDesc']['createdDate'] for mails in data['emailList']]
because when I run this without the second attribute ['createdDate'] it would work, but I need both attributes on my final_list:
myEmailList = [mails['emailTypeDesc'] for mails in data['emailList']]

I think you're misunderstanding the syntax. mails['emailTypeDesc']['createdDate'] is looking for the key 'createdDate' inside the object mails['emailTypeDesc'], but in fact they are two items at the same level.
Since mails['emailTypeDesc'] is a string, not a dictionary, you get the error you have quoted. It seems that you want to add the two items mails['emailTypeDesc'] and mails['createdDate'] to your list. I'm not sure if you'd rather join these together into a single string or create a sub-list or something else. Here's a sublist option.
myEmailList = [[mails['emailTypeDesc'], mails['createdDate']] for mails in data['emailList']]

Strings in JSON must be in double quotes, not single.
Edit: As well as names.

Mapping names in Python

I have been given the following list of dictionaries:
names = [
{'first_name': 'Jane', 'last_name': 'Doe'},
{'first_name': 'John', 'last_name': 'Kennedy'},
{'first_name': 'Ada', 'last_name': 'Lovelace'}
]
Part a was to return an array of full names, which I did as follows:
[user['first_name'] +' '+ user['last_name'] for user in names]
It returned the following:
['Jane Doe', 'John Kennedy', 'Ada Lovelace']
Part b is to Now do the same thing above, only return a list of dictionaries, with 'name' being the key. Result should be:
python
[{'name':'Jane Doe'},{'name':'John Kennedy'},{'name': 'Ada Lovelace'}]
I have tried everything I can think of. From trying to change the key, to changing back to a list and then back to a dictionary. I'm very new at Python and would appreciate any help possible.

[{'name': '{first_name} {last_name}'.format(**n)} for n in names]

The following comprehension using join will work:
result = [{'name': ' '.join((d['first_name'], d['last_name']))} for d in names]
# [{'name': 'Jane Doe'}, {'name': 'John Kennedy'}, {'name': 'Ada Lovelace'}]

Adjust your list comprehension to the following:
names = [
{'first_name': 'Jane', 'last_name': 'Doe'},
{'first_name': 'John', 'last_name': 'Kennedy'},
{'first_name': 'Ada', 'last_name': 'Lovelace'}
]
result = [{'name':d['first_name']+' '+ d['last_name']} for d in names]
print(result)
The output:
[{'name': 'Jane Doe'}, {'name': 'John Kennedy'}, {'name': 'Ada Lovelace'}]

list(map(lambda d: {'name': ' '.join((d['first_name'], d['last_name']))},names))

Python: retrieve arbitrary dictionary path and amend data?

Simple Python question, but I'm scratching my head over the answer!
I have an array of strings of arbitrary length called path, like this:
path = ['country', 'city', 'items']
I also have a dictionary, data, and a string, unwanted_property. I know that the dictionary is of arbitrary depth and is dictionaries all the way down, with the exception of the items property, which is always an array.
[CLARIFICATION: The point of this question is that I don't know what the contents of path will be. They could be anything. I also don't know what the dictionary will look like. I need to walk down the dictionary as far as the path indicates, and then delete the unwanted properties from there, without knowing in advance what the path looks like, or how long it will be.]
I want to retrieve the parts of the data object (if any) that matches the path, and then delete the unwanted_property from each.
So in the example above, I would like to retrieve:
data['country']['city']['items']
and then delete unwanted_property from each of the items in the array. I want to amend the original data, not a copy. (CLARIFICATION: By this I mean, I'd like to end up with the original dict, just minus the unwanted properties.)
How can I do this in code?
I've got this far:
path = ['country', 'city', 'items']
data = {
'country': {
'city': {
'items': [
{
'name': '114th Street',
'unwanted_property': 'foo',
},
{
'name': '8th Avenue',
'unwanted_property': 'foo',
},
]
}
}
}
for p in path:
if p == 'items':
data = [i for i in data[p]]
else:
data = data[p]
if isinstance(data, list):
for d in data:
del d['unwanted_property']
else:
del data['unwanted_property']
The problem is that this doesn't amend the original data. It also relies on items always being the last string in the path, which may not always be the case.
CLARIFICATION: I mean that I'd like to end up with:
{
'country': {
'city': {
'items': [
{
'name': '114th Street'
},
{
'name': '8th Avenue'
},
]
}
}
}
Whereas what I have available in data is only [{'name': '114th Street'}, {'name': '8th Avenue'}].
I feel like I need something like XPath for the dictionary.

The problem you are overwriting the original data reference. Change your processing code to
temp = data
for p in path:
temp = temp[p]
if isinstance(temp, list):
for d in temp:
del d['unwanted_property']
else:
del temp['unwanted_property']
In this version, you set temp to point to the same object that data was referring to. temp is not a copy, so any changes you make to it will be visible in the original object. Then you step temp along itself, while data remains a reference to the root dictionary. When you find the path you are looking for, any changes made via temp will be visible in data.
I also removed the line data = [i for i in data[p]]. It creates an unnecessary copy of the list that you never need, since you are not modifying the references stored in the list, just the contents of the references.
The fact that path is not pre-determined (besides the fact that items is going to be a list) means that you may end up getting a KeyError in the first loop if the path does not exist in your dictionary. You can handle that gracefully be doing something more like:
try:
temp = data
for p in path:
temp = temp[p]
except KeyError:
print('Path {} not in data'.format(path))
else:
if isinstance(temp, list):
for d in temp:
del d['unwanted_property']
else:
del temp['unwanted_property']

The problem you are facing is that you are re-assigning the data variable to an undesired value. In the body of your for loop you are setting data to the next level down on the tree, for instance given your example data will have the following values (in order), up to when it leaves the for loop:
data == {'country': {'city': {'items': [{'name': '114th Street', 'unwanted_property': 'foo',}, {'name': '8th Avenue', 'unwanted_property': 'foo',},]}}}
data == {'city': {'items': [{'name': '114th Street', 'unwanted_property': 'foo',}, {'name': '8th Avenue', 'unwanted_property': 'foo',},]}}
data == {'items': [{'name': '114th Street', 'unwanted_property': 'foo',}, {'name': '8th Avenue', 'unwanted_property': 'foo',},]}
data == [{'name': '114th Street', 'unwanted_property': 'foo',}, {'name': '8th Avenue', 'unwanted_property': 'foo',},]
Then when you delete the items from your dictionaries at the end you are left with data being a list of those dictionaries as you have lost the higher parts of the structure. Thus if you make a backup reference for your data you can get the correct output, for example:
path = ['country', 'city', 'items']
data = {
'country': {
'city': {
'items': [
{
'name': '114th Street',
'unwanted_property': 'foo',
},
{
'name': '8th Avenue',
'unwanted_property': 'foo',
},
]
}
}
}
data_ref = data
for p in path:
if p == 'items':
data = [i for i in data[p]]
else:
data = data[p]
if isinstance(data, list):
for d in data:
del d['unwanted_property']
else:
del data['unwanted_property']
data = data_ref

def delKey(your_dict,path):
if len(path) == 1:
for item in your_dict:
del item[path[0]]
return
delKey( your_dict[path[0]],path[1:])
data
{'country': {'city': {'items': [{'name': '114th Street', 'unwanted_property': 'foo'}, {'name': '8th Avenue', 'unwanted_property': 'foo'}]}}}
path
['country', 'city', 'items', 'unwanted_property']
delKey(data,path)
data
{'country': {'city': {'items': [{'name': '114th Street'}, {'name': '8th Avenue'}]}}}

You need to remove the key unwanted_property.
names_list = []
def remove_key_from_items(data):
for d in data:
if d != 'items':
remove_key_from_items(data[d])
else:
for item in data[d]:
unwanted_prop = item.pop('unwanted_property', None)
names_list.append(item)
This will remove the key. The second parameter None is returned if the key unwanted_property does not exist.
EDIT:
You can use pop even without the second parameter. It will raise KeyError if the key does not exist.
EDIT 2: Updated to recursively go into depth of data dict until it finds the items key, where it pops the unwanted_property as desired and append into the names_list list to get the desired output.

Using operator.itemgetter you can compose a function to return the final key's value.
import operator, functools
def compose(*functions):
'''returns a callable composed of the functions
compose(f, g, h, k) -> f(g(h(k())))
'''
def compose2(f, g):
return lambda x: f(g(x))
return functools.reduce(compose2, functions, lambda x: x)
get_items = compose(*[operator.itemgetter(key) for key in path[::-1]])
Then use it like this:
path = ['country', 'city', 'items']
unwanted_property = 'unwanted_property'
for thing in get_items(data):
del thing[unwanted_property]
Of course if the path contains non-existent keys it will throw a KeyError - you probably should account for that:
path = ['country', 'foo', 'items']
get_items = compose(*[operator.itemgetter(key) for key in path[::-1]])
try:
for thing in get_items(data):
del thing[unwanted_property]
except KeyError as e:
print('missing key:', e)

You can try this:
path = ['country', 'city', 'items']
previous_data = data[path[0]]
previous_key = path[0]
for i in path:
previous_data = previous_data[i]
previous_key = i
if isinstance(previous_data, list):
for c, b in enumerate(previous_data):
if "unwanted_property" in b:
del previous_data[c]["unwanted_property"]
current_dict = {}
previous_data_dict = {}
for i, a in enumerate(path):
if i == 0:
current_dict[a] = data[a]
previous_data_dict = data[a]
else:
if a == previous_key:
current_dict[a] = previous_data
else:
current_dict[a] = previous_data_dict[a]
previous_data_dict = previous_data_dict[a]
data = current_dict
print(data)
Output:
{'country': {'city': {'items': [{'name': '114th Street'}, {'name': '8th Avenue'}]}}, 'items': [{'name': '114th Street'}, {'name': '8th Avenue'}], 'city': {'items': [{'name': '114th Street'}, {'name': '8th Avenue'}]}}

How to print the keys and values of a dictionary as table? [duplicate]

This question already has answers here:
List of dicts to/from dict of lists
(14 answers)
Closed 6 years ago.
I have a dictionary that stores a list of items for each key as shown:
name_dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
I want to print the data in the dictionary as table format:
FirstName,MiddleName,LastName # the keys of the dictionary
Elizabeth,H.,Perkins #the values of the keys in one line (the first item in the list)
Scott,T.,Joseph #the values of the keys in new line (the following item in the list)
How to solve this problem?
I have tried doing the suggested solution by Gareth Latty, but that did not work.
with open('C:/Output.csv',"w+") as file:
w = csv.DictWriter(file,name_dict.keys())
w.writeheader()
w.writerow(name_dict)
It outputs the following:
MiddleName,LastName,FirstName
"['H.', 'T.']","['Perkins', 'Joseph']","['Perkins', 'Joseph']"
Any idea how to output the values (the item in the list) of each of the keys in new row?

csv.DictWriter expects a dictionary with the field:single_line pairs for each row which is unfortunately not what you have, you basically need to convert your data structure to be a list of dicts for the single lines:
[{'MiddleName': 'H.', 'FirstName': 'Elizabeth ', 'LastName': 'Perkins'}, {'MiddleName': 'T.', 'FirstName': 'Scott ', 'LastName': 'Joseph'}]
You can convert it with something like this:
import csv
def seperate_entries(dict_of_lists):
iters = [(k,iter(v)) for k,v in dict_of_lists.items()]
try:
while True:
yield {k:next(v) for k,v in iters}
except StopIteration:
return
name_dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
with open('sample.csv',"w+") as file:
w = csv.DictWriter(file,name_dict.keys())
w.writeheader()
w.writerows(seperate_entries(name_dict))

I think you misuse dict. When you have multiple values, you should use a list of dicts and not a dict which the values are lists. Instead of
Dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
You should do:
Dict = [{'FirstName': 'Elizabeth', 'MiddleName': 'H.', 'LastName': 'Perkins'}, {'FirstName': 'Joseph', 'MiddleName': 'T. ', 'LastName': 'Scott'}]
or in a more readable version:
Dict = [
{'FirstName': 'Elizabeth', 'MiddleName': 'H.', 'LastName': 'Perkins'},
{'FirstName': 'Joseph', 'MiddleName': 'T. ', 'LastName': 'Scott' }
]
If you want to print one line (one dictionary of the list), you can do something like that:
def printDict(d):
print d["FirstName"] + "," + d["MiddleName"] + "," + d["LastName"]
And if you want to print each of the elements in the list you have:
def printList(l):
for i in l:
printDict(i)
And just use it like that:
printList(Dict)
With your first (original) Dict, accessing Dict["FirstName"] would return a list, and when printed it would print as:
["Elizabeth", "Joesph"]
But with the second (new way I suggested) Dict, accessing Dict[0]["FirstName"] would return a string, and will print like:
Elizabeth

To access the keys in the Dictionary, you just need to do the following:
middleNames=Dict['Middlename']
firstNames=Dict['FirstName']
lastNames=Dict['LastName']
You now have access to the values stored in the inner list information, this can similarly be accessed by the following:
# Find how many items read (optional)
len(middleNames)
# Then iterate through the items
for mName in middleName:
print mName # this will print each list item e.g.
H.
T.
# for brevity, this is the same as doing this...
middleName[0] etc
Hope this helps.

You need define how many row you have.
Just flat it into rows with all keys in Dict.
import csv
Dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
len_row = 2
with open('Output.csv', "w+") as file:
w = csv.DictWriter(file, Dict.keys())
w.writeheader()
for i in range(len_row):
row = {}
for k in Dict.keys():
for v in Dict.values():
row[k] = Dict[k][i]
w.writerow(row)

I'm sure there are more efficient ways of achieving what you want but this lays down a simple outline of what you want and shows how you can achieve it.
names = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
output = open('output.txt','w')
#NOTE: this will write headers in alphabetical format. You will need to change that so it follows the FirstName, MiddleName, LastName pattern
for key in sorted(names.keys()):
output.write(key + ',')
output.write('\n')
#assuming we always have same amount of middle, first, and last names
for i in range(len(names['FirstName'])):
personFname = ''
personLname = ''
personMname = ''
for key in names.keys():
if key == 'MiddleName':
personMname = names[key][i]
elif key == 'FirstName':
personFname = names[key][i]
elif key == 'LastName':
personLname = names[key][i]
output.write(personFname + ',' + personMname + ',' + personLname)
output.write('\n')
output.close()