We Keep Coding

How do I do multiple JSON entries with Python?

I'm trying to pull some data from a flight simulation JSON table. It's updated every 15 seconds and I've been trying to pull print(obj['pilots']['flight_plans']['cid']). However im getting the error
Traceback (most recent call last):
File "main.py", line 18, in <module>
print(obj['pilots']['flight_plans']['cid'])
TypeError: list indices must be integers or slices, not str
My code is below
import json
from urllib.request import urlopen
import urllib
# initial setup
URL = "https://data.vatsim.net/v3/vatsim-data.json"
# json entries
response = urllib.request.urlopen(URL)
str_response = response.read().decode('utf-8')
obj = json.loads(str_response)
# result is connections
# print(obj["general"]["connected_clients"])
print(obj['pilots']['flight_plans']['cid'])
The print(obj["general"]["connected_clients"]) does work.

Investigate your obj with print(json.dumps(obj,indent=2). You'll find that the pilots key is a list of dictionaries containing flight_plan (not plural) and cid keys. Here's the first few lines:
{
"general": {
"version": 3,
"reload": 1,
"update": "20220301062202",
"update_timestamp": "2022-03-01T06:22:02.245318Z",
"connected_clients": 292,
"unique_users": 282
},
"pilots": [
{
"cid": 1149936,
"name": "1149936",
"callsign": "URO504",
"server": "UK",
"pilot_rating": 0,
"latitude": -23.39706,
"longitude": -46.3709,
"altitude": 9061,
"groundspeed": 327,
"transponder": "0507",
"heading": 305,
"qnh_i_hg": 29.97,
"qnh_mb": 1015,
"flight_plan": {
"flight_rules": "I",
"aircraft": "A346",
...
For example, iterate over the list of pilots and print name/cid:
for pilot in obj['pilots']:
print(pilot['name'],pilot['cid'])
Output:
1149936 1149936
Nick Aydin OTHH 1534423
Oguz Aydin 1429318
Marvin Steglich LSZR 1482019
Daniel Krol EPKK 1279199
... etc ...

Extract data from json, append using for loop and save as CSV

I have extracted id, username, and name for 100 followers for 102 politicians using Tweepy. The data is stored in a JSON file named pol_followers. Now I wish to append id and username and save it as a CSV file using the function below. However, when using the function in the last line append_followers_to_csv(pol_followers, "pol_followers.csv") I get the error seen at the bottom.
# Structure of pol_followers. The full pol_followers is much longer...
print(json.dumps(pol_followers, indent=4, sort_keys=True)) # see json data structure
[
{
"data": [
{
"id": "1464206217807601666",
"name": "terry alex",
"username": "terryal51850644"
},
{
"id": "1479032154394968064",
"name": "Charles Williams",
"username": "Charles99924770"
},
{
"id": "2526015770",
"name": "LISA P",
"username": "LISAP0910"
},
{
"id": "2957692520",
"name": "fayaz ahmad",
"username": "ahmadfayaz202"
}
],
"meta": {
"next_token": "F6HS7IU5SRGHEZZZ",
"result_count": 100
}
},
{
"data": [
{
"id": "2482703136",
"name": "HieuVu",
"username": "sachieuhaihanh"
},
{
"id": "580882148",
"name": "Maxine D. Harmon",
"username": "maxxximd"
},
{
"id": "1478867472841334787",
"name": "RBPsych1",
"username": "RBPsych1"
# Create file
csv_follower_file = open("pol_followers.csv", "a", newline="", encoding='utf-8')
csv_follower_writer = csv.writer(csv_follower_file)
# Create headers for the data I want to save. I only want to save these columns in my dataset
csv_follower_writer.writerow(
['id', 'username'])
csv_follower_file.close()\
def append_followers_to_csv(pol_followers, csv_follower_file):
# A counter variable
global follower_id, username
counter = 0
# Open OR create the target CSV file
csv_follower_file = open(csv_follower_file, "a", newline="", encoding='utf-8')
csv_follower_writer = csv.writer(csv_follower_file)
for ids in pol_followers['data']:
# 1. follower ID
follower_id = ids['id']
# 2. follower username
username = ids['username']
# Assemble all data in a list
ress = [follower_id, username]
# Append the result to the CSV file
csv_follower_writer.writerow(ress)
counter += 1
# When done, close the CSV file
csvFile.close()
# Print the number of tweets for this iteration
print("# of Tweets added from this response: ", counter)
append_followers_to_csv(pol_followers, "pol_followers.csv") # Save tweet data in a csv file
File "<input>", line 1, in <module>
File "<input>", line 11, in append_followers_to_csv
TypeError: list indices must be integers or slices, not str

You are just missing additional loop, like so:
for each_dict in pol_followers:
for ids in each_dict['data']:
follower_id = ids['id']
username = ids['username']

You seem to have wrapped your JSON object in a list, so instead of getting the 'data' bit of the JSON, you are getting the 'data'th element of a list when you are iterating in your append_followers_to_csv function, which you can't do in python. Try removing the square brackets around the JSON or making it for ids in pol_followers[0]['data'].

TypeError: string indices must be integers, while reading a json file with python3

I'm trying to read a json file and put its content into a list. When I try to do a for loop to get the contents, I am receiving the error: TypeError: string indices must be integers
Weirdly, if I try to print the file, no problem at all. Does someone know what am I doing wrong? Is the json file ok? 
Here's the json file
{
"Username": "Final test",
"Email": "teste#email.com",
"Risks": {
"Risk1": {
"Name": "Risco 1",
"Min": "3000",
"Likely": "5000"
},
"Risk2": {
"Name": "risco2",
"Min": "4500",
"Likely": "6000"
},
"Risk3": {
"Name": "risco3",
"Min": "1500",
"Likely": "7000"
}
}
}
And here's the python code:
json_file = open('path to file..')
data = json.load(json_file)
lista_json = [data['Username'], data['Email']]
for item in data['Risks']:
print("item= ", item)
print("Name, ", data['Risks'][item]['Name'])
print("Min, ", data['Risks'][item]['Min'])
lista_json.append(item['Risks'][item]['Name'])
lista_json.append(item[‘'Risks'][item]['Min'])

You don't seem to be using a proper quote. ‘ != '. Also, you're accessing item in the append, not data
This is what you want:
json_file = open('path...')
data = json.load(json_file)
lista_json = [data['Username'], data['Email']]
for item in data['Risks']:
print("item= ", item)
print("Name, ", data['Risks'][item]['Name'])
print("Min, ", data['Risks'][item]['Min'])
lista_json.append(data['Risks'][item]['Name'])
lista_json.append(data['Risks'][item]['Min'])

How to print out the exact field/string of the JSON output?

I'm trying to filter all the result that I got in the GET Request.
The Output that I want is just to get the summary: , key: and self:.
But I'm getting a lot of Json data.
I've tried googling on how to do this and I'm going to nowhere.
Here is my code:
The commented lines are the codes that I have tried.
import requests
import json
import re
import sys
url ="--------"
auth='i.g--t----------', 'X4------'
r = requests.get(url, auth=(auth))
data = r.json()
#print( json.dumps(data, indent=2) )
#res1 = " ".join(re.split("summary", data))
#print ("first string result: ", str(res1))
#json_str = json.dumps(data)
#resp = json.loads(json_str)
#print (resp['id'])
#resp_dict = json.loads(resp_str)
#resp_dict.get('name')
#print('dasdasd', json_str["summary"])
Example of the Get Api Output that I'm getting using this code. print( json.dumps(data, indent=2) )
{
"id": "65621",
"self": "https://bboxxltd.atlassian.net/rest/api/2/issue/65621",
"key": "CMS-5901",
"fields": {
"summary": "new starter: Edoardo Bologna",
"customfield_10700": [
{
"id": "2",
"name": "BBOXX Rwanda HQ",
"_links": {
"self": "https://bboxxltd.atlassian.net/rest/servicedeskapi/organization/2"
}
}
},
"inwardIssue": {
"id": "65862",
"key": "BMT-2890",
"self": "https://bboxxltd.atlassian.net/rest/api/2/issue/65862",
"fields": {
"summary": "ERP Databases access with Read Only",
"status": {
"self": "https://bboxxltd.atlassian.net/rest/api/2/status/10000",
"description": "",
"iconUrl": "https://bboxxltd.atlassian.net/",
"name": "To Do",
"id": "10000",
"statusCategory": {
"self": "https://bboxxltd.atlassian.net/rest/api/2/statuscategory/2",
"id": 2,
"key": "new",
"colorName": "blue-gray",
"name": "To Do"
}
},
"priority": {
"self": "https://bboxxltd.atlassian.net/rest/api/2/priority/4",
"iconUrl": "https://bboxxltd.atlassian.net/images/icons/priorities/low.svg",
"name": "Low",
My error is:
Traceback (most recent call last):
File "c:/Users/IanJayloG/Desktop/Python Files/Ex_Files_Learning_Python/Exercise Files/Test/Untitled-1.py", line 17, in <module>
print('dasdasd', data["summary"])
KeyError: 'summary'
PS C:\Users\IanJayloG\Desktop\Python Files\Ex_Files_Learning_Python\Exercise Files> & C:/Users/IanJayloG/AppData/Local/Programs/Python/Python37-32/python.exe "c:/Users/IanJayloG/Desktop/Python Files/Ex_Files_Learning_Python/Exercise Files/Test/Untitled-1.py"
Traceback (most recent call last):
File "c:/Users/IanJayloG/Desktop/Python Files/Ex_Files_Learning_Python/Exercise Files/Test/Untitled-1.py", line 17, in <module>
print('dasdasd', json_str["summary"])
TypeError: string indices must be integers

The problem about your error message
print('dasdasd', json_str["summary"])
TypeError: string indices must be integers
is that you try to access the named field summary on a string (variable json_str), which does not work because strings don't have fields to access by name. If you use the indexing [] operator on a string, you can only provide integers or ranges to extract single characters or sequences from that string. This is obviously not what you're intending.
The keys self and key are on top level of your JSON document, whereas summary is under fields. This should do it, without any extra transformation applied:
import requests
r = requests.get(url, auth=(auth))
data = r.json()
data_summary = data['fields']['summary']
data_self = data['self']
data_key = data['key']

How to convert nested JSON data to CSV using python?

I have a file consisting of an array containing over 5000 objects. However, I am having trouble converting one particular part of my JSON file into the appropriate columns in CSV format.
Below is an example version of my data file:
{
"Result": {
"Example 1": {
"Type1": [
{
"Owner": "Name1 Example",
"Description": "Description1 Example",
"Email": "example1_email#email.com",
"Phone": "(123) 456-7890"
}
]
},
"Example 2": {
"Type1": [
{
"Owner": "Name2 Example",
"Description": "Description2 Example",
"Email": "example2_email#email.com",
"Phone": "(111) 222-3333"
}
]
}
}
}
Here is my current code:
import csv
import json
json_file='example.json'
with open(json_file, 'r') as json_data:
x = json.load(json_data)
f = csv.writer(open("example.csv", "w"))
f.writerow(["Address","Type","Owner","Description","Email","Phone"])
for key in x["Result"]:
type = "Type1"
f.writerow([key,
type,
x["Result"][key]["Type1"]["Owner"],
x["Result"][key]["Type1"]["Description"],
x["Result"][key]["Type1"]["Email"],
x["Result"][key]["Type1"]["Phone"]])
My problem is that I'm encountering this issue:
Traceback (most recent call last):
File "./convert.py", line 18, in <module>
x["Result"][key]["Type1"]["Owner"],
TypeError: list indices must be integers or slices, not str
When I try to substitute the last array such as "Owner" to an integer value, I receive this error: IndexError: list index out of range.
When I strictly change the f.writerow function to
f.writerow([key,
type,
x["Result"][key]["Type1"]])
I receive the results in a column, but it merges everything into one column, which makes sense. Picture of the output: https://imgur.com/a/JpDkaAT
I would like the results to be separated based on the label into individual columns instead of being merged into one. Could anyone assist?
Thank you!

Type1 in your data structure is a list, not a dict. So you need to iterate over it instead of referencing by key.
for key in x["Result"]:
# key is now "Example 1" etc.
type1 = x["Result"][key]["Type1"]
# type1 is a list, not a dict
for i in type1:
f.writerow([key,
"Type1",
type1["Owner"],
type1["Description"],
type1["Email"],
type1["Phone"]])
The inner for loop ensure that you're protected from the assumption that "Type1" only ever has one item in the list.

It's definately not the best example, but I'm to sleepy to optimize it.
import csv
def json_to_csv(obj, res):
for k, v in obj.items():
if isinstance(v, dict):
res.append(k)
json_to_csv(v, res)
elif isinstance(v, list):
res.append(k)
for el in v:
json_to_csv(el, res)
else:
res.append(v)
obj = {
"Result": {
"Example 1": {
"Type1": [
{
"Owner": "Name1 Example",
"Description": "Description1 Example",
"Email": "example1_email#email.com",
"Phone": "(123) 456-7890"
}
]
},
"Example 2": {
"Type1": [
{
"Owner": "Name2 Example",
"Description": "Description2 Example",
"Email": "example2_email#email.com",
"Phone": "(111) 222-3333"
}
]
}
}
}
with open("out.csv", "w+") as f:
writer = csv.writer(f)
writer.writerow(["Address","Type","Owner","Description","Email","Phone"])
for k, v in obj["Result"].items():
row = [k]
json_to_csv(v, row)
writer.writerow(row)

Figured it out!
I changed the f.writerow function to the following:
for key in x["Result"]:
type = "Type1"
f.writerow([key,
type,
x["Result"][key]["Type1"][0]["Owner"],
x["Result"][key]["Type1"][0]["Email"]])
...
This allowed me reference the keys within the object. Hopefully this helps someone down the line!