Converting a RESTful API response from list to dictionary

Converting a RESTful API response from list to dictionary - python

Currently I have a function (shown below) that makes a GET request from an API that I made myself
def get_vehicles(self):
result = "http://127.0.0.1:8000/vehicles"
response = requests.get(result)
data = response.content
data_dict = json.loads(data)
return data_dict
The data I got is in this format. Which is a list of dictionary
data_dict = [{'colour': 'Black', 'cost': 10, 'latitude': -37.806152, 'longitude': 144.95787, 'rentalStatus': 'True', 'seats': 4, 'user': None, 'vehicleBrand': 'Toyota', 'vehicleID': 1, 'vehicleModel': 'Altis'}, {'colour': 'White', 'cost': 15, 'latitude': -37.803913, 'longitude': 144.964859, 'rentalStatus': 'False', 'seats': 4, 'user': {'firstname': 'Test', 'imageName': None, 'password': 'password', 'surname': 'Ing', 'userID': 15, 'username': 'Testing'}, 'vehicleBrand': 'Honda', 'vehicleID': 3, 'vehicleModel': 'Civic'}]
Is it possible to convert it to just a dictionary? Example:
data_dict = {'colour': 'Black', 'cost': 10, 'latitude': -37.806152, 'longitude': 144.95787, 'rentalStatus': 'True', 'seats': 4, 'user': None, 'vehicleBrand': 'Toyota', 'vehicleID': 1, 'vehicleModel': 'Altis'}, {'colour': 'White', 'cost': 15, 'latitude': -37.803913, 'longitude': 144.964859, 'rentalStatus': 'False', 'seats': 4, 'user': {'firstname': 'Test', 'imageName': None, 'password': 'password', 'surname': 'Ing', 'userID': 15, 'username': 'Testing'}, 'vehicleBrand': 'Honda', 'vehicleID': 3, 'vehicleModel': 'Civic'}

No, the second result is a tuple, not a dict.
data_dict = {'colour': 'Black', 'cost': 10, 'latitude': -37.806152, 'longitude': 144.95787, 'rentalStatus': 'True', 'seats': 4, 'user': None, 'vehicleBrand': 'Toyota', 'vehicleID': 1, 'vehicleModel': 'Altis'}, {'colour': 'White', 'cost': 15, 'latitude': -37.803913, 'longitude': 144.964859, 'rentalStatus': 'False', 'seats': 4, 'user': {'firstname': 'Test', 'imageName': None, 'password': 'password', 'surname': 'Ing', 'userID': 15, 'username': 'Testing'}, 'vehicleBrand': 'Honda', 'vehicleID': 3, 'vehicleModel': 'Civic'}
print(type(data_dict))
# <class 'tuple'>
It is the same as:
data_dict = ({'colour': 'Black', 'cost': 10, 'latitude': -37.806152, 'longitude': 144.95787, 'rentalStatus': 'True', 'seats': 4, 'user': None, 'vehicleBrand': 'Toyota', 'vehicleID': 1, 'vehicleModel': 'Altis'}, {'colour': 'White', 'cost': 15, 'latitude': -37.803913, 'longitude': 144.964859, 'rentalStatus': 'False', 'seats': 4, 'user': {'firstname': 'Test', 'imageName': None, 'password': 'password', 'surname': 'Ing', 'userID': 15, 'username': 'Testing'}, 'vehicleBrand': 'Honda', 'vehicleID': 3, 'vehicleModel': 'Civic'})
That's why it is a tuple.
If you only want to merge them in a dict,it seems to be impossible because dict couldn't have the same keys.But you could merge the value as a list,like:
d = {key: list(value) for key, value in zip(data_dict[0].keys(), zip(data_dict[0].values(), data_dict[1].values()))}
print(d)
Result(Make sure they has the same length):
{
'colour': ['Black', 'White'],
'cost': [10, 15],
'latitude': [-37.806152, -37.803913],
'longitude': [144.95787, 144.964859],
'rentalStatus': ['True', 'False'],
'seats': [4, 4],
'user': [None, {
'firstname': 'Test',
'imageName': None,
'password': 'password',
'surname': 'Ing',
'userID': 15,
'username': 'Testing'
}],
'vehicleBrand': ['Toyota', 'Honda'],
'vehicleID': [1, 3],
'vehicleModel': ['Altis', 'Civic']
}

This a list of dictionaries.
Therefore you can access them using the array syntax: data_dict[0] for the first element for example.

Related

spark.read.json error: (java.lang.ArrayStoreException: java.util.HashMap)

I'm using the code below to pull in multiple json files into 1 using pagination. When I try to create a spark dataframe I get the error '
java.lang.ArrayStoreException: java.util.HashMap' during spark.read.json. Below the code block I've provided output from printing the 'issues' data set (minus proprietary info). I've done a bit of research and can't figure out what I can try to make this work. Any assistance would be greatly appreciated!!
import requests
import json
limit = 2
startat = 0
issues = []
for page_num in range(2):
startat = page_num*50
url = f"https://URL/rest/api/2/search?jql=TEST&startAt={startat}&maxResults={limit}"
req = requests.get(url, headers={'Accept': 'application/json', 'Authorization': 'Basic xxxxxxxxxxxxxxxxxxxxxxxxxx'})
data = req.json()
issues.extend(data['issues'])
jsonDF = spark.read.json(issues)
jsonDF.printSchema()
[{'expand': 'operations,versionedRepresentations,editmeta,changelog,customfield_10010.requestTypePractice,renderedFields', 'id': '11441', 'self': 'https://my.url.net/rest/api/2/issue/11441', 'key': 'TS-1401', 'fields': {'statuscategorychangedate': '2022-11-29T07:05:17.359-0800', 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10004', 'id': '10004', 'description': 'Functionality or a feature expressed as a user goal.', 'iconUrl': 'https://my.url.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium', 'name': 'Story', 'subtask': False, 'avatarId': 10315, 'hierarchyLevel': 0}, 'parent': {'id': '11420', 'key': 'TS-1380', 'self': 'https://my.url.net/rest/api/2/issue/11420', 'fields': {'summary': 'Clone30 - Migration Epics', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10000', 'id': '10000', 'description': 'A big user story that needs to be broken down. Created by Jira Software - do not edit or delete.', 'iconUrl': 'https://my.url.net/images/icons/issuetypes/epic.svg', 'name': 'Epic', 'subtask': False, 'hierarchyLevel': 1}}}, 'timespent': None, 'project': {'self': 'https://my.url.net/rest/api/2/project/10001', 'id': '10001', 'key': 'TS', 'name': 'Project', 'projectTypeKey': 'software', 'simplified': False, 'avatarUrls': {'48x48': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556', '24x24': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=small', '16x16': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=xsmall', '32x32': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=medium'}}, 'customfield_10033': None, 'fixVersions': [], 'aggregatetimespent': None, 'customfield_10034': [], 'customfield_10035': None, 'resolution': None, 'customfield_10036': None, 'customfield_10037': None, 'customfield_10027': None, 'customfield_10028': None, 'customfield_10029': None, 'resolutiondate': None, 'workratio': -1, 'watches': {'self': 'https://my.url.net/rest/api/2/issue/TS-1401/watchers', 'watchCount': 1, 'isWatching': True}, 'lastViewed': '2022-12-08T10:06:57.022-0800', 'created': '2022-11-29T07:05:16.501-0800', 'customfield_10020': None, 'customfield_10021': None, 'customfield_10022': None, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'customfield_10023': None, 'customfield_10024': None, 'customfield_10025': None, 'customfield_10026': None, 'labels': [], 'customfield_10016': None, 'customfield_10017': None, 'customfield_10018': {'hasEpicLinkFieldDependency': False, 'showField': False, 'nonEditableReason': {'reason': 'EPIC_LINK_SHOULD_BE_USED', 'message': 'To set an epic as the parent, use the epic link instead'}}, 'customfield_10019': '0|i008a3:', 'timeestimate': None, 'aggregatetimeoriginalestimate': None, 'versions': [], 'issuelinks': [], 'assignee': None, 'updated': '2022-11-29T07:05:20.759-0800', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'components': [], 'timeoriginalestimate': None, 'description': 'Data owner completes template (understand scope of migration efforts)', 'customfield_10010': None, 'customfield_10014': 'TS-1380', 'customfield_10015': None, 'customfield_10005': None, 'customfield_10006': None, 'customfield_10007': None, 'security': None, 'customfield_10008': None, 'customfield_10009': None, 'aggregatetimeestimate': None, 'summary': 'Template', 'creator': {'self': 'https://my.url.net/rest/api/2/user?accountId=5d669f4bf81f2c0d99ee9e38', 'accountId': '5d669f4bf81f2c0d99ee9e38', 'emailAddress': 'test#aol.com', 'avatarUrls': {'48x48': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '24x24': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '16x16': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '32x32': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png'}, 'displayName': 'Joe Test', 'active': True, 'timeZone': 'America/Los_Angeles', 'accountType': 'atlassian'}, 'subtasks': [{'id': '11442', 'key': 'TS-1402', 'self': 'https://my.url.net/rest/api/2/issue/11442', 'fields': {'summary': 'Complete Template with table/views required (in) and produced (out)', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10006', 'id': '10006', 'description': "A small piece of work that's part of a larger task.", 'iconUrl': 'https://my.url.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10316?size=medium', 'name': 'Sub-task', 'subtask': True, 'avatarId': 10316, 'hierarchyLevel': -1}}}], 'reporter': {'self': 'https://my.url.net/rest/api/2/user?accountId=5d669f4bf81f2c0d99ee9e38', 'accountId': '5d669f4bf81f2c0d99ee9e38', 'emailAddress': 'test#aol.com', 'avatarUrls': {'48x48': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '24x24': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '16x16': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '32x32': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png'}, 'displayName': 'Joe Test', 'active': True, 'timeZone': 'America/Los_Angeles', 'accountType': 'atlassian'}, 'aggregateprogress': {'progress': 0, 'total': 0}, 'customfield_10001': None, 'customfield_10002': None, 'customfield_10003': None, 'customfield_10004': None, 'customfield_10038': None, 'environment': None, 'duedate': None, 'progress': {'progress': 0, 'total': 0}, 'votes': {'self': 'https://my.url.net/rest/api/2/issue/TS-1401/votes', 'votes': 0, 'hasVoted': False}}}, {'expand': 'operations,versionedRepresentations,editmeta,changelog,customfield_10010.requestTypePractice,renderedFields', 'id': '11438', 'self': 'https://my.url.net/rest/api/2/issue/11438', 'key': 'TS-1398', 'fields': {'statuscategorychangedate': '2022-11-29T07:05:09.126-0800', 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10004', 'id': '10004', 'description': 'Functionality or a feature expressed as a user goal.', 'iconUrl': 'https://my.url.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium', 'name': 'Story', 'subtask': False, 'avatarId': 10315, 'hierarchyLevel': 0}, 'parent': {'id': '11420', 'key': 'TS-1380', 'self': 'https://my.url.net/rest/api/2/issue/11420', 'fields': {'summary': 'Clone30 - Migration Epics', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10000', 'id': '10000', 'description': 'A big user story that needs to be broken down. Created by Jira Software - do not edit or delete.', 'iconUrl': 'https://my.url.net/images/icons/issuetypes/epic.svg', 'name': 'Epic', 'subtask': False, 'hierarchyLevel': 1}}}, 'timespent': None, 'project': {'self': 'https://my.url.net/rest/api/2/project/10001', 'id': '10001', 'key': 'TS', 'name': 'Project', 'projectTypeKey': 'software', 'simplified': False, 'avatarUrls': {'48x48': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556', '24x24': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=small', '16x16': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=xsmall', '32x32': 'https://my.url.net/rest/api/2/universal_avatar/view/type/project/avatar/10556?size=medium'}}, 'fixVersions': [], 'customfield_10033': None, 'customfield_10034': [], 'aggregatetimespent': None, 'customfield_10035': None, 'resolution': None, 'customfield_10036': None, 'customfield_10037': None, 'customfield_10027': None, 'customfield_10028': None, 'customfield_10029': None, 'resolutiondate': None, 'workratio': -1, 'lastViewed': None, 'watches': {'self': 'https://my.url.net/rest/api/2/issue/TS-1398/watchers', 'watchCount': 1, 'isWatching': True}, 'created': '2022-11-29T07:05:08.312-0800', 'customfield_10020': None, 'customfield_10021': None, 'customfield_10022': None, 'customfield_10023': None, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'customfield_10024': None, 'customfield_10025': None, 'customfield_10026': None, 'labels': [], 'customfield_10016': None, 'customfield_10017': None, 'customfield_10018': {'hasEpicLinkFieldDependency': False, 'showField': False, 'nonEditableReason': {'reason': 'EPIC_LINK_SHOULD_BE_USED', 'message': 'To set an epic as the parent, use the epic link instead'}}, 'customfield_10019': '0|i008ae:y', 'timeestimate': None, 'aggregatetimeoriginalestimate': None, 'versions': [], 'issuelinks': [], 'assignee': None, 'updated': '2022-11-29T07:05:22.417-0800', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'components': [], 'timeoriginalestimate': None, 'description': 'Creating reports/reporting cubes; need to find out reports used', 'customfield_10010': None, 'customfield_10014': 'TS-1380', 'customfield_10015': None, 'customfield_10005': None, 'customfield_10006': None, 'security': None, 'customfield_10007': None, 'customfield_10008': None, 'customfield_10009': None, 'aggregatetimeestimate': None, 'summary': '\xa0create reports/cubes', 'creator': {'self': 'https://my.url.net/rest/api/2/user?accountId=5d669f4bf81f2c0d99ee9e38', 'accountId': '5d669f4bf81f2c0d99ee9e38', 'emailAddress': 'test#aol.com', 'avatarUrls': {'48x48': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '24x24': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '16x16': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '32x32': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png'}, 'displayName': 'Joe Test', 'active': True, 'timeZone': 'America/Los_Angeles', 'accountType': 'atlassian'}, 'subtasks': [{'id': '11439', 'key': 'TS-1399', 'self': 'https://my.url.net/rest/api/2/issue/11439', 'fields': {'summary': 'Confirm: any reporting cubes required using this data are created and in production?', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10006', 'id': '10006', 'description': "A small piece of work that's part of a larger task.", 'iconUrl': 'https://my.url.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10316?size=medium', 'name': 'Sub-task', 'subtask': True, 'avatarId': 10316, 'hierarchyLevel': -1}}}, {'id': '11440', 'key': 'TS-1400', 'self': 'https://my.url.net/rest/api/2/issue/11440', 'fields': {'summary': 'Confirm: any structured reports using this data are created and in production?', 'status': {'self': 'https://my.url.net/rest/api/2/status/10003', 'description': '', 'iconUrl': 'https://my.url.net/', 'name': 'Backlog', 'id': '10003', 'statusCategory': {'self': 'https://my.url.net/rest/api/2/statuscategory/2', 'id': 2, 'key': 'new', 'colorName': 'blue-gray', 'name': 'To Do'}}, 'priority': {'self': 'https://my.url.net/rest/api/2/priority/3', 'iconUrl': 'https://my.url.net/images/icons/priorities/medium.svg', 'name': 'Medium', 'id': '3'}, 'issuetype': {'self': 'https://my.url.net/rest/api/2/issuetype/10006', 'id': '10006', 'description': "A small piece of work that's part of a larger task.", 'iconUrl': 'https://my.url.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10316?size=medium', 'name': 'Sub-task', 'subtask': True, 'avatarId': 10316, 'hierarchyLevel': -1}}}], 'reporter': {'self': 'https://my.url.net/rest/api/2/user?accountId=5d669f4bf81f2c0d99ee9e38', 'accountId': '5d669f4bf81f2c0d99ee9e38', 'emailAddress': 'test#aol.com', 'avatarUrls': {'48x48': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '24x24': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '16x16': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png', '32x32': 'https://secure.gravatar.com/avatar/69b7db33e65c274c27a07b28b356e329?d=https%3A%2F%2Favatar-management--avatars.us-west-2.test.png'}, 'displayName': 'Joe Test', 'active': True, 'timeZone': 'America/Los_Angeles', 'accountType': 'atlassian'}, 'aggregateprogress': {'progress': 0, 'total': 0}, 'customfield_10001': None, 'customfield_10002': None, 'customfield_10003': None, 'customfield_10004': None, 'customfield_10038': None, 'environment': None, 'duedate': None, 'progress': {'progress': 0, 'total': 0}, 'votes': {'self': 'https://my.url.net/rest/api/2/issue/TS-1398/votes', 'votes': 0, 'hasVoted': False}}}]

How to separate values in a dictionary to be put into CSV? [duplicate]

This question already has answers here:
How can I convert JSON to CSV?
(26 answers)
Closed 3 years ago.
I am trying to write my JSON output to CSV, but I'm not sure how to separate my values into individual columns
This is my current code
with open('dict.csv', 'w') as csv_file:
writer = csv.writer(csv_file)
for key, value in response.json().items():
writer.writerow([value])
print(value)
This is the csv file I am getting:
current csv file
This is the desired csv file/output I want to get:
desired output
This is an example of my JSON Output
[{'id': '123', 'custom_id': '12', 'company': 28, 'company_name': 'Sunshine'}, {'id': '224', 'custom_id': '14', 'company': 38, 'company_name': 'Flowers'},
{'id': '888', 'custom_id': '10', 'company': 99, 'company_name': 'Fields'}]
how about this JSON format? (a more complicated one)
[{'id': '777', 'custom_id': '000112', 'company': 28, 'company_name':
'Weddings Inc', 'delivery_address': '25 olive park terrace, 61234', 'delivery_timeslot': {'lower': '2019-12-06T10:00:00Z', 'upper': '2019-12-06T13:00:00Z', 'bounds': '[)'}, 'sender_name': 'Joline', 'sender_email': '', 'sender_contact': '91234567', 'removed': None, 'recipient_name': 'Joline', 'recipient_contact': '91866655', 'notes': '', 'items': [{'id': 21668, 'name': 'Loose hair flowers', 'quantity': 1, 'metadata': {}, 'removed': None}, {'id': 21667, 'name': "Groom's Boutonniere", 'quantity': 1, 'metadata': {}, 'removed': None}, {'id': 21666, 'name': 'Bridal Bouquet', 'quantity': 1, 'metadata': {}, 'removed': None}], 'latitude': '1.1234550920764211111', 'longitude': '103.864352476201000000', 'created': '2019-08-15T05:40:30.385467Z', 'updated': '2019-08-15T05:41:27.930110Z', 'status': 'pending', 'verbose_status': 'Pending', 'logs': [{'id': 56363, 'order': '50c402', 'order_custom_id': '000112', 'order_delivery_address': '25 olive park terrace, 61234', 'order_delivery_timeslot': {'lower': '2019-12-06T10:00:00Z', 'upper': '2019-12-06T13:00:00Z', 'bounds': '[)'}, 'message': 'Order was created.', 'failure_reason': None, 'success_code': None, 'success_description': None, 'created': '2019-08-15T05:40:30.431790Z', 'removed': None}, {'id': 56364, 'order': '50c402d8-7c76-45b5-b883-e2fb887a507e', 'order_custom_id': 'INV-000112', 'order_delivery_address': '25 olive park terrace, 61234', 'order_delivery_timeslot': {'lower': '2019-12-06T10:00:00Z', 'upper': '2019-12-06T13:00:00Z', 'bounds': '[)'}, 'message': 'Order is pending.', 'failure_reason': None, 'success_code': None, 'success_description': None, 'created': '2019-08-15T05:40:30.433139Z', 'removed': None}], 'reschedule_requests': [], 'signature': None},
{'id': '241', 'custom_id': '000123', 'company': 22, 'company_name': 'Pearl Pte Ltd', 'delivery_address': '90 Merchant Road, Hotel Royal, 223344', 'delivery_timeslot': {'lower': '2019-11-29T10:00:00Z', 'upper': '2019-11-29T13:00:00Z', 'bounds': '[)'}, 'sender_name': 'Vera Smith', 'sender_email': '', 'sender_contact': '81234567', 'removed': None, 'recipient_name': 'Vera Smith', 'recipient_contact': '81234561', 'notes': '', 'items': [{'id': 22975, 'name': 'Custom wrapped bouquet', 'quantity': 2, 'metadata': {}, 'removed': None}, {'id': 22974, 'name': "Parents' boutonniere x 3", 'quantity': 1, 'metadata': {}, 'removed': None}, {'id': 22973, 'name': "Groom's boutonniere", 'quantity': 1, 'metadata': {}, 'removed': None}, {'id': 22972, 'name': 'Loose hair flowers', 'quantity': 1, 'metadata': {}, 'removed': None}, {'id': 22971, 'name': 'Bridal Bouquet', 'quantity': 1, 'metadata': {}, 'removed': None}], 'latitude': '1.28821802835873000000', 'longitude': '103.84569230314800000000', 'created': '2019-08-30T03:20:17.477528Z', 'updated': '2019-08-30T03:29:25.307856Z', 'status': 'pending', 'verbose_status': 'Pending', 'logs': [{'id': 59847, 'order': '24117085-9104-4442-841b-4a734f801d39', 'order_custom_id': 'INV-000123', 'order_delivery_address': '90 Merchant Road, Hotel Royal, 223344', 'order_delivery_timeslot': {'lower': '2019-11-29T10:00:00Z', 'upper': '2019-11-29T13:00:00Z', 'bounds': '[)'}, 'message': 'Order was created.', 'failure_reason': None, 'success_code': None, 'success_description': None, 'created': '2019-08-30T03:20:17.511250Z', 'removed': None}, {'id': 59848, 'order': '24117085-9104-4442-841b-4a734f801d39', 'order_custom_id': 'INV-000123', 'order_delivery_address': '90 Merchant Road, Hotel Royal, 223344', 'order_delivery_timeslot': {'lower': '2019-11-29T10:00:00Z', 'upper': '2019-11-29T13:00:00Z', 'bounds': '[)'}, 'message': 'Order is pending.', 'failure_reason': None, 'success_code': None, 'success_description': None, 'created': '2019-08-30T03:20:17.513132Z', 'removed': None}], 'reschedule_requests': [], 'signature': None}]

Use pandas library:
df.to_csv() - Write object to a comma-separated values (csv) file.
Ex.
import pandas as pd
data = [{'id': '123', 'custom_id': '12', 'company': 28, 'company_name': 'Sunshine'},
{'id': '224', 'custom_id': '14', 'company': 38, 'company_name': 'Flowers'},
{'id': '888', 'custom_id': '10', 'company': 99, 'company_name': 'Fields'}]
df = pd.DataFrame(data)
df.to_csv('sample.csv')

Try:
import csv
csv_file = 'my_file.csv'
csv_columns = ['id', 'custom_id', 'company', 'company_name']
dict_data = [{'id': '123', 'custom_id': '12', 'company': 28, 'company_name': 'Sunshine'}, {'id': '224', 'custom_id': '14', 'company': 38, 'company_name': 'Flowers'}, {'id': '888', 'custom_id': '10', 'company': 99, 'company_name': 'Fields'}]
try:
with open(csv_file, 'w') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
writer.writeheader()
for data in dict_data:
writer.writerow(data)
except IOError:
print("I/O error")

Given your response data in json format
response = [{'id': '123', 'custom_id': '12', 'company': 28, 'company_name': 'Sunshine'},
{'id': '224', 'custom_id': '14', 'company': 38, 'company_name': 'Flowers'},
{'id': '888', 'custom_id': '10', 'company': 99, 'company_name': 'Fields'}]
You can convert it to a list of lists using
header = [response[0].keys()]
data = [row.values() for row in response]
csv_list = header + data
And then save it to csv using
with open('dict.csv', "w") as f:
for row in csv_list:
f.write("%s\n" % ','.join(str(col) for col in row))
This should yield your desired output

Sort a value in python list of dict

I am looking to sort a list of dictionary based on a key which is in the format \d+:\d+
The list of dictionary should have the expected key(age)
lst = [
[{'id': 'SOMEDATA_31_30', 'age': '31:30', 'values':0, 'hasdata': False, 'name': u'SOMEDATA'}]
[{'id': 'SOMEDATA1_32_29', 'age': '32:29', 'values':1, 'hasdata': False, 'name': u'SOMEDATA1'}]
[{'id': 'SOMEDATA2_22_20', 'age': '22:20', 'values':3, 'hasdata': False, 'name': u'SOMEDATA2'},{'id': 'SOMEDATA2_28_27', 'age': '28:27', 'values':4, 'hasdata': False, 'name': u'SOMEDATA2'}]
]
So when I pass the whole(lst) to compare(function?!) then it should give me a list of dict ordered in desc order based on the age.
lst = [
{'id': 'SOMEDATA1_32_29', 'age': '32:29', 'values':1, 'hasdata': False, 'name': u'SOMEDATA1'}
{'id': 'SOMEDATA_31_30', 'age': '31:30', 'values':0, 'hasdata': False, 'name': u'SOMEDATA'}
{'id': 'SOMEDATA2_28_27', 'age': '28:27', 'values':4, 'hasdata': False, 'name': u'SOMEDATA2'}
{'id': 'SOMEDATA2_22_20', 'age': '22:20', 'values':3, 'hasdata': False, 'name': u'SOMEDATA2'}
]
How should we do this ? Thanks for your time.

You can use sorted().
lst = [
{'id': 'SOMEDATA_31_30', 'age': '31:30', 'values':0, 'hasdata': False, 'name': u'SOMEDATA'},
{'id': 'SOMEDATA1_32_29', 'age': '32:29', 'values':1, 'hasdata': False, 'name': u'SOMEDATA1'},
{'id': 'SOMEDATA2_22_20', 'age': '22:20', 'values':3, 'hasdata': False, 'name': u'SOMEDATA2'},
{'id': 'SOMEDATA2_28_27', 'age': '28:27', 'values':4, 'hasdata': False, 'name': u'SOMEDATA2'},
]
newlist = sorted([x[0] for x in lst], key=lambda k: k['age'], reverse=True)
print(newlist)
Output
[{'age': '32:29', 'hasdata': False, 'id': 'SOMEDATA1_32_29','name': 'SOMEDATA1', 'values': 1},
{'age': '31:30','hasdata': False, 'id': 'SOMEDATA_31_30','name': 'SOMEDATA', 'values': 0},
{'age': '28:27', 'hasdata': False, 'id': 'SOMEDATA2_28_27','name':'SOMEDATA2','values': 4},
{'age': '22:20', 'hasdata': False, 'id': 'SOMEDATA2_22_20','name': 'SOMEDATA2','values': 3}]

Using the key and reverse argument of python built-in function sorted():
lst = [
{'id': 'SOMEDATA_31_30', 'age': '31:30', 'values':0, 'hasdata': False, 'name': u'SOMEDATA'},
{'id': 'SOMEDATA1_32_29', 'age': '32:29', 'values':1, 'hasdata': False, 'name': u'SOMEDATA1'},
{'id': 'SOMEDATA2_22_20', 'age': '22:20', 'values':3, 'hasdata': False, 'name': u'SOMEDATA2'},
{'id': 'SOMEDATA2_28_27', 'age': '28:27', 'values':4, 'hasdata': False, 'name': u'SOMEDATA2'},
]
import re
from pprint import pprint
pprint( sorted(lst, reverse=True, key=lambda v: [(int(i[0]), int(i[1])) for i in re.findall(r'(\d+):(\d+)', v['age'])]), width=120 )
This prints:
[{'age': '32:29', 'hasdata': False, 'id': 'SOMEDATA1_32_29', 'name': 'SOMEDATA1', 'values': 1},
{'age': '31:30', 'hasdata': False, 'id': 'SOMEDATA_31_30', 'name': 'SOMEDATA', 'values': 0},
{'age': '28:27', 'hasdata': False, 'id': 'SOMEDATA2_28_27', 'name': 'SOMEDATA2', 'values': 4},
{'age': '22:20', 'hasdata': False, 'id': 'SOMEDATA2_22_20', 'name': 'SOMEDATA2', 'values': 3}]

Using NumPy you can try this:
ages = [i['age'] for i in lst]
print (np.array(lst)[np.argsort(ages)[::-1]])
Output
[ {'id': 'SOMEDATA1_32_29', 'age': '32:29', 'values': 1, 'hasdata': False, 'name': 'SOMEDATA1'}
{'id': 'SOMEDATA_31_30', 'age': '31:30', 'values': 0, 'hasdata': False, 'name': 'SOMEDATA'}
{'id': 'SOMEDATA2_28_27', 'age': '28:27', 'values': 4, 'hasdata': False, 'name': 'SOMEDATA2'}
{'id': 'SOMEDATA2_22_20', 'age': '22:20', 'values': 3, 'hasdata': False, 'name': 'SOMEDATA2'}]

Python3 asc desc sorting

Lets assume I've got a following python list(this is only example):
my_list = [{'user': 'Joe', 'score': 14},
{'user': 'Foo', 'score': 12},
{'user': 'May', 'score': 12},
{'user': 'Kat', 'score': 12},
{'user': 'Doe', 'score': 13}]
I need to sort this list in ascending order by score and descending order by a username.
Expected sort result:
my_list = [{'user': 'May', 'score': 12},
{'user': 'Kat', 'score': 12},
{'user': 'Foo', 'score': 12},
{'user': 'Doe', 'score': 13},
{'user': 'Joe', 'score': 14}]
So, I could do something like this if I want everything to be in ascending order:
my_list.sort(key=lambda x: (x['score'], x['user']))
For integers it is easy to solve this problem just adding - in front of it:
my_list.sort(key=lambda x: (-x['score'], x['user']))
Unfortunately, strings can not be negative :-|
I need a generic solution that doesn't involve 'reverse=True'. Lambda function is dynamically generated based on a user config.
Thoughts?

Your current solution will work fine if you set the reverse parameter of list.sort to True:
>>> my_list = [{'user': 'Joe', 'score': 14},
... {'user': 'Foo', 'score': 12},
... {'user': 'May', 'score': 12},
... {'user': 'Kat', 'score': 12},
... {'user': 'Doe', 'score': 13}]
>>> my_list.sort(key=lambda x: (-x['score'], x['user']), reverse=True)
>>> pprint(my_list) # pprint makes the nice output
[{'score': 12, 'user': 'May'},
{'score': 12, 'user': 'Kat'},
{'score': 12, 'user': 'Foo'},
{'score': 13, 'user': 'Doe'},
{'score': 14, 'user': 'Joe'}]
>>>
This will sort the list in reverse order.
Edit:
Since the names and scores have two different sort orders, you will need to use two separate sorts to achieve your desired output:
>>> my_list = [{'user': 'Joe', 'score': 14},
... {'user': 'Foo', 'score': 12},
... {'user': 'May', 'score': 12},
... {'user': 'Kat', 'score': 12},
... {'user': 'Doe', 'score': 13}]
>>> my_list.sort(key=lambda x: x['user'], reverse=True)
>>> my_list.sort(key=lambda x: x['score'])
>>> pprint(my_list)
[{'score': 12, 'user': 'May'},
{'score': 12, 'user': 'Kat'},
{'score': 12, 'user': 'Foo'},
{'score': 13, 'user': 'Doe'},
{'score': 14, 'user': 'Joe'}]
>>>

The ord of a string can be negative:
my_list.sort(key=lambda x: (x["score"],[-ord(x) for x in x["user"]])))
In [50]: my_list.sort(key=lambda x: (x["score"],[-ord(x) for x in x["user"]]))
In [51]: my_list
Out[51]:
[{'score': 12, 'user': 'May'},
{'score': 12, 'user': 'Kat'},
{'score': 12, 'user': 'Foo'},
{'score': 13, 'user': 'Doe'},
{'score': 14, 'user': 'Joe'}]

you need to add the reverse=True keyword for descending order.
>>> my_list.sort(key=lambda x: (-x['score'], x['user']),reverse=True)
>>> my_list
[{'score': 12, 'user': 'May'}, {'score': 12, 'user': 'Kat'}, {'score': 12, 'user': 'Foo'}, {'score': 13, 'user': 'Doe'}, {'score': 14, 'user': 'Joe'}]

Find a minimal value in each array in Python

Suppose I have the following data in Python 3.3:
my_array =
[{'man_id': 1, '_id': ObjectId('1234566'), 'type': 'worker', 'value': 11},
{'man_id': 1, '_id': ObjectId('1234577'), 'type': 'worker', 'value': 12}],
[{'man_id': 2, '_id': ObjectId('1234588'), 'type': 'worker', 'value': 11},
{'man_id': 2, '_id': ObjectId('3243'), 'type': 'worker', 'value': 7},
{'man_id': 2, '_id': ObjectId('54'), 'type': 'worker', 'value': 99},
{'man_id': 2, '_id': ObjectId('9879878'), 'type': 'worker', 'value': 135}],
#.............................
[{'man_id': 13, '_id': ObjectId('111'), 'type': 'worker', 'value': 1},
{'man_id': 13, '_id': ObjectId('222'), 'type': 'worker', 'value': 2},
{'man_id': 13, '_id': ObjectId('3333'), 'type': 'worker', 'value': 9}]
There are 3 arrays. How do I find an element in each array with minimal value?

[min(arr, key=lambda s:s['value']) for arr in my_array]

Maybe something like that is acc for you:
for arr in my_array:
minVal = min([row['value'] for row in arr])
print [row for row in arr if row['value'] == minVal]

We Keep Coding

Python is a programming language that lets you work quickly and integrate systems more effectively.

Converting a RESTful API response from list to dictionary - python

This a list of dictionaries. Therefore you can access them using the array syntax: data_dict[0] for the first element for example.

Related

spark.read.json error: (java.lang.ArrayStoreException: java.util.HashMap)

How to separate values in a dictionary to be put into CSV? [duplicate]

Sort a value in python list of dict

Python3 asc desc sorting

Find a minimal value in each array in Python

Categories

Resources