im calling an API in Python and get a JSON response. Im filtering that response for the values I need. Then I want to make a JSON from that values again and print it.
Here's my code:
import requests
import json
url = "https://my.api"
payload={}
headers = {
'Cookie': 'PVEAuthCookie=123'
}
response = requests.request("GET", url, headers=headers, data=payload)
json_object = json.loads(response.text)
json_formatted_str = json.dumps(json_object, indent=2)
vmid_count = json_formatted_str.count("vmid")
i = 0
for i in range(vmid_count):
vmid = json_object['data'][i]['vmid']
cpu = json_object['data'][i]['cpu']
mem = json_object['data'][i]['mem']
data = { "data": [{"vmid": vmid, "name": name, "type": type, "maxcpu": maxcpu, "maxmem": maxmem, "maxdisk": maxdisk, "cpu": cpu, "mem": mem, "disk": disk, "status": status, "uptime": uptime, "node": node}]}
json_dump = json.dumps(data, indent=2)
print(json_dump)
json_formatted_str contains the JSON I receive from the API and looks like that:
{
"data": [
{
"status": "running",
"netin": 44452797,
"maxdisk": 16729894912,
"diskwrite": 649285632,
"node": "pve",
"uptime": 76654,
"vmid": 108,
"id": "lxc/108",
"type": "lxc",
"mem": 111636480,
"cpu": 0.000327262867680765,
"diskread": 456568832,
"name": "container108",
"disk": 2121224192,
"maxmem": 2147483648,
"netout": 25054481,
"maxcpu": 1,
"template": 0
},
more arrays (a lot more)
]
}
json_dump looks like that:
{
"data": [
{
"vmid": 108,
"name": "container108",
"type": "lxc",
"maxcpu": 1,
"maxmem": 2147483648,
"maxdisk": 16729894912,
"cpu": 0.0123243844774696,
"mem": 111116288,
"disk": 2121342976,
"status": "running",
"uptime": 76825,
"node": "pve"
}
]
}
{
"data": [
{
"vmid": 1007,
... more arrays
It starts a whole new object every time it runs through the for-loop. If I remove the print(json_dump) from the loop, I only get the last array.
("data":[ should not be there more then one time at the beginning and the commas at the end of the arrays are missing too.
I want the output to look like this:
{
"data":[
{
"vmid": "100",
"cpu": "4",
"mem": "16384" (more keys and values...)
},
{
"vmid": "101",
"cpu": "2",
"mem": "4096"
},
{
"vmid": "102",
"cpu": "6",
"mem": "32768"
}
]
}
I tried to find examples online and here on Stackoverflow, but I coundn't find anything so I thought I ask here.
You have to append every time new data, not create again.
like this
data["data"].append({"vmid": vmid, "name": name, "type": type, "maxcpu": maxcpu, "maxmem": maxmem, "maxdisk": maxdisk, "cpu": cpu, "mem": mem, "disk": disk, "status": status, "uptime": uptime, "node": node})
And than you can dump and print out of the loop.
Related
I have an API, after calling which I'm getting a very big json in response.
I want to access similar keys which are present inside the nested dict.
I'm using following lines to make a get request and storing the json data : -
p25_st_devices = r'https://url_from_where_im_getting_data.com'
header_events = {
'Authorization': 'Basic random_keys'}
r2 = requests.get(p25_st_devices, headers= header_events)
r2_json = json.loads(r2.content)
The sample of the json is as follows : -
{
"next": "value",
"self": "value",
"managedObjects": [
{
"creationTime": "2021-08-02T10:48:15.120Z",
"type": " c8y_MQTTdevice",
"lastUpdated": "2022-03-24T17:09:01.240+03:00",
"childAdditions": {
"self": "value",
"references": []
},
"name": "PS_MQTT1",
"assetParents": {
"self": "value",
"references": []
},
"self": "value",
"id": "338",
"Building": "value"
},
{
"creationTime": "2021-08-02T13:06:09.834Z",
"type": " c8y_MQTTdevice",
"lastUpdated": "2021-12-27T12:08:20.186+03:00",
"childAdditions": {
"self": "value",
"references": []
},
"name": "FS_MQTT2",
"assetParents": {
"self": "value",
"references": []
},
"self": "value",
"id": "339",
"c8y_IsDevice": {}
},
{
"creationTime": "2021-08-02T13:06:39.602Z",
"type": " c8y_MQTTdevice",
"lastUpdated": "2021-12-27T12:08:20.433+03:00",
"childAdditions": {
"self": "value",
"references": []
},
"name": "PS_MQTT3",
"assetParents": {
"self": "value",
"references": []
},
"self": "value",
"id": "340",
"c8y_IsDevice": {}
}
],
"statistics": {
"totalPages": 423,
"currentPage": 1,
"pageSize": 3
}
}
As per my understanding I can access name key using r2_json['managedObjects'][0]['name']
But how do I iterate over this json and store all values of name inside an array?
EDIT 1 :
Another thing which I'm trying to achieve is get all id from the JSON data and store in an array where the nested dict managedObjects contains name starting with PS_ only.
Therefore, the expected output would be device_id = ['338','340']
You should not just call the [0] index of the list, but loop over it:
all_names = []
for object in r2_json['managedObjects']:
all_names.append(object['name'])
print(all_names)
edit: Updated answer after OP updated theirs.
For your second question you can use startswith(). The code is almost the same.
PS_names = []
for object in r2_json['managedObjects']:
if object['name'].startswith("PS_"):
PS_names.append(object['id']) # we append with the id, if startswith("PS_") returns True.
print(PS_names)
I'm pretty new to Python so forgive me here.
I'm trying to call/print/log JSON from a GET request. I need to get the id's following this tree: items > track > artists > id.
Basically im trying to just get the ID's from each track on this JSON file/list. I keep getting errors no matter how I format them. Again im new so im not sure how to call it let also run it in a loop to get all the ID's as an array. I have done it with other JSON files in the past but this one is farther in and I get the error : TypeError: list indices must be integers or slices, not str
I figured out it works when you do it by adding the [0] like this:
artists = response["items"][0]["track"]["artists"] BUT I want to loop it and get all of the Id's for each one and that option picks just one.
here is the beginning of the json so you can see the layout.
{
"href": "https://api.spotify.com/v1/me/tracks?offset=0&limit=15&market=US",
"items": [
{
"added_at": "2021-12-15T22:26:25Z",
"track": {
"album": {
"album_type": "single",
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/6MlPT0WxdWnrYcpXT8GZF8"
},
"href": "https://api.spotify.com/v1/artists/6MlPT0WxdWnrYcpXT8GZF8",
"id": "6MlPT0WxdWnrYcpXT8GZF8",
"name": "PARKFORD",
"type": "artist",
"uri": "spotify:artist:6MlPT0WxdWnrYcpXT8GZF8"
}
],
"external_urls": {
"spotify": "https://open.spotify.com/album/4o2d8uBTyMfJeaJqSXn9tP"
},
"href": "https://api.spotify.com/v1/albums/4o2d8uBTyMfJeaJqSXn9tP",
"id": "4o2d8uBTyMfJeaJqSXn9tP",
"images": [
{
"height": 640,
"url": "https://i.scdn.co/image/ab67616d0000b27332bcd9e1b2234c6cd6b2b2ec",
"width": 640
},
{
"height": 300,
"url": "https://i.scdn.co/image/ab67616d00001e0232bcd9e1b2234c6cd6b2b2ec",
"width": 300
},
{
"height": 64,
"url": "https://i.scdn.co/image/ab67616d0000485132bcd9e1b2234c6cd6b2b2ec",
"width": 64
}
],
"name": "There's Nothing in the Rain",
"release_date": "2021-11-25",
"release_date_precision": "day",
"total_tracks": 1,
"type": "album",
"uri": "spotify:album:4o2d8uBTyMfJeaJqSXn9tP"
},
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/6MlPT0WxdWnrYcpXT8GZF8"
},
"href": "https://api.spotify.com/v1/artists/6MlPT0WxdWnrYcpXT8GZF8",
"id": "6MlPT0WxdWnrYcpXT8GZF8",
"name": "PARKFORD",
"type": "artist",
"uri": "spotify:artist:6MlPT0WxdWnrYcpXT8GZF8"
here is the code I have written out
uri = MY_LIKED_SONGS
headers = {
"Authorization": f'Bearer {tokens["access_token"]}',
"Content-Type": "application/json",
}
r = requests.get(uri, headers=headers)
response = r.json()
# return response
artist_ids = {}
data = json.dumps(response)
for artist_ids in data["items"]["track"]:
logger.info(artist_ids["items"]["track"]["artists"]["id"])
print(artist_ids["_source"]["items"][0]["track"]["id"])
tracks = response["items"]["track"][0]["artists"]["id"]
Here when you do json.dumps(), it converts a Python object into a json string. In order to iterate through json object you need to convert it into json object. For that you have to use json.loads() then you can get artist id from that.
data = json.loads(json.dumps(response))
for item in data["items"]:
for artist in item['track']['album']['artists']:
print(artist['id'])
This might be helpful.
Hi im am trying to parse json data and gets this error every time the element
if ['fields']['assignee'] in each:
TypeError: list indices must be integers or slices, not str
>>>
My json is this
{
"expand": "schema,names",
"startAt": 1,
"maxResults": 50,
"total": 7363,
"issues": [
{
"expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id": "591838",
"self": "https://jira.mynet.com/rest/api/2/issue/591838",
"key": "TEST-8564",
"fields": {
"summary": "delete tables 31-03-2020 ",
"customfield_10006": 2.0,
"created": "2020-02-27T10:29:12.000+0100",
"description": "A LOT OF TEXT",
"assignee": null,
"labels": [
"DATA",
"Refined"
],
"status": {
"self": "https://jira.mynet.com/rest/api/2/status/10000",
"description": "",
"iconUrl": "https://jira.mynet.com/",
"name": "To Do",
"id": "10000",
"statusCategory": {
"self": "https://jira.mynet.com/rest/api/2/statuscategory/2",
"id": 2,
"key": "new",
"colorName": "blue-gray",
"name": "To Do"
}
}
}
}
]
}
The element in ['fields']['assignee'] is NULL in this example
sometimes it is like this
"assignee": : {
"self": "https://mynet.com/rest/api/2/user?username=xxxxxx",
"name": "sij",
"key": "x",
"emailAddress": xx#mynet.com",
"avatarUrls": {
"48x48": "https://mynet.com/secure/useravatar?ownerId=bdysdh&avatarId=16743",
"24x24": "https://mynet.com/secure/useravatar?size=small&ownerId=bdysdh&avatarId=16743",
"16x16": "https://mynet.com/secure/useravatar?size=xsmall&ownerId=bdysdh&avatarId=16743",
"32x32": "https://mynet.com/secure/useravatar?size=medium&ownerId=bdysdh&avatarId=16743"
},
"displayName": "Bruce Springsteen",
"active": true,
"timeZone": "Arctic/Longyearbyen"
},
I am trying to check of assignee is null and if so print null
my code looks like this
with open('C:\\TEMP\\testdata.json') as json_file:
data = json.load(json_file)
for each in data['issues']:
if ['fields']['assignee'] in each:
print (['fields']['assignee']['name'])
else:
print ('null')
I have tried to put in [0] between ['fields']['assignee']['name'] but nothing seems to help.
Try with
if 'fields' in each and 'assignee' in each['fields']:
Note that you need the name of the key, not surrounded by square brackets.
Perhaps better:
for each in data['issues']:
print(each.get('fields', {}).get('assignee', {}).get('name', 'null'))
and if you can't guarantee that 'issues' exists in data either:
for each in data.get('issues', []):
<as before>
data.get('issues', []) returns an empty list if data['issuess'] doesn't exist.
I have a JSON file with lots of data, and I want to keep only specific data.
I thought reading the file, get all the data I want and save as a new JSON.
The JSON is like this:
{
"event": [
{
"date": "2019-01-01",
"location": "world",
"url": "www.com",
"comments": "null",
"country": "china",
"genre": "blues"
},
{
"date": "2000-01-01",
"location": "street x",
"url": "www.cn",
"comments": "null",
"country":"turkey",
"genre": "reds"
},
{...
and I want it to be like this (with just date and url from each event.
{
"event": [
{
"date": "2019-01-01",
"url": "www.com"
},
{
"date": "2000-01-01",
"url": "www.cn"
},
{...
I can open the JSON and read from it using
with open('xx.json') as f:
data = json.load(f)
data2=data["events"]["date"]
But I still need to understand how to save the data I want in a new JSON keeping it's structure
You can use loop comprehension to loop over the events in and return a dictionary containing only the keys that you want.
data = { "event": [
{
"date": "2019-01-01",
"location": "world",
"url": "www.com",
"comments": None,
"country": "china",
"genre": "blues",
},
{
"date": "2000-01-01",
"location": "street x",
"url": "www.cn",
"comments": None,
"country" :"turkey",
"genre":"reds",
}
]}
# List comprehension
data["event"] = [{"date": x["date"], "url": x["url"]} for x in data["event"]]
Alternatively, you can map a function over the events list
keys_to_keep = ["date", "url"]
def subset_dict(d):
return {x: d[x] for x in keys_to_keep}
data["event"] = list(map(subset_dict, data["event"]))
I am trying to interact with an API and running into issues accessing nested objects. Below is sample json output that I am working with.
{
"results": [
{
"task_id": "22774853-2b2c-49f4-b044-2d053141b635",
"params": {
"type": "host",
"target": "54.243.80.16",
"source": "malware_analysis"
},
"v": "2.0.2",
"status": "success",
"time": 227,
"data": {
"details": {
"as_owner": "Amazon.com, Inc.",
"asn": "14618",
"country": "US",
"detected_urls": [],
"resolutions": [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
],
"response_code": 1,
"verbose_msg": "IP address in dataset"
},
"match": true
}
}
]
}
The deepest I am able to access is the data portion which returns too much.... ideally I am just trying access as_owner,asn,country,detected_urls,resolutions
When I try to access details / response code ... etc I will get a KeyError. My nested json goes deeper then other Q's mentioned and I have tried that logic.
Below is my current code snippet and any help is appreciated!
import requests
import json
headers = {
'Content-Type': 'application/json',
}
params = (
('wait', 'true'),
)
data = '{"target":{"one":{"type": "ip","target": "54.243.80.16", "sources": ["xxx","xxxxx"]}}}'
r=requests.post('https://fakewebsite:8000/api/services/intel/lookup/jobs', headers=headers, params=params, data=data, auth=('apikey', ''))
parsed_json = json.loads(r.text)
#results = parsed_json["results"]
for item in parsed_json["results"]:
print(item['data'])
You just need to index correctly into the converted JSON. Then you can easily loop over a list of the keys you want to fetch, since they are all in the "details" dictionary.
import json
raw = '''\
{
"results": [
{
"task_id": "22774853-2b2c-49f4-b044-2d053141b635",
"params": {
"type": "host",
"target": "54.243.80.16",
"source": "malware_analysis"
},
"v": "2.0.2",
"status": "success",
"time": 227,
"data": {
"details": {
"as_owner": "Amazon.com, Inc.",
"asn": "14618",
"country": "US",
"detected_urls": [],
"resolutions": [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
],
"response_code": 1,
"verbose_msg": "IP address in dataset"
},
"match": true
}
}
]
}
'''
parsed_json = json.loads(raw)
wanted = ['as_owner', 'asn', 'country', 'detected_urls', 'resolutions']
for item in parsed_json["results"]:
details = item['data']['details']
for key in wanted:
print(key, ':', json.dumps(details[key], indent=4))
# Put a blank line at the end of the details for each item
print()
output
as_owner : "Amazon.com, Inc."
asn : "14618"
country : "US"
detected_urls : []
resolutions : [
{
"hostname": "bumbleride.com",
"last_resolved": "2016-09-15 00:00:00"
},
{
"hostname": "chilitechnology.com",
"last_resolved": "2016-09-16 00:00:00"
}
]
BTW, when you fetch JSON data using requests there's no need to use json.loads: you can access the converted JSON using the .json method of the returned request object instead of using its .text attribute.
Here's a more robust version of the main loop of the above code. It simply ignores any missing keys. I didn't post this code earlier because the extra if tests make it slightly less efficient, and I didn't know that keys could be missing.
for item in parsed_json["results"]:
if not 'data' in item:
continue
data = item['data']
if not 'details' in data:
continue
details = data['details']
for key in wanted:
if key in details:
print(key, ':', json.dumps(details[key], indent=4))
# Put a blank line at the end of the details for each item
print()