Python json to csv: "AttributeError: 'str' object has no attribute 'keys'" - python

So I have been attempting to create a file to convert json files to csv files using a template i found online. However, I keep receiving the following error:
header = User.keys()
AttributeError: 'str' object has no attribute 'keys'
Here is the code I'm using:
# Python program to convert
# JSON file to CSV
import json
import csv
# Opening JSON file and loading the data
# into the variable data
with open('/Users/jhillier5/Downloads/mentalhealthcheck-in-default-rtdb-export (1).json') as json_file:
data = json.load(json_file)
User_data = data['User']
# now we will open a file for writing
data_file = open('data_file.csv', 'w')
# create the csv writer object
csv_writer = csv.writer(data_file)
# Counter variable used for writing
# headers to the CSV file
count = 0
print(User_data)
print(type(User_data))
for User in User_data:
if count == 0:
# Writing headers of CSV file
header = User.keys()
csv_writer.writerow(header)
count += 1
# Writing data of CSV file
csv_writer.writerow(User.values())
data_file.close()
I simply don't understand why there is a string object and not a dict. I double checked and User_data is stored as a dict but i still get the error.
The json is organised in the following format:
{
"User" : {
"-MlIoVUgATqwtD_3AeCb" : {
"Aid" : "1",
"Gender" : "1",
"Grade" : "11",
}
}
}


In this for loop
for User in User_data:
if count == 0:
# Writing headers of CSV file
header = User.keys()
csv_writer.writerow(header)
count += 1
# Writing data of CSV file
csv_writer.writerow(User.values())
You are iterating through User_data, so User will be ["MlIoVUgATqwtD_3AeCb", ...]. In order to get the keys of that user data you should index it this way -
header = User_data[User].keys()

Related

How do I assign a value to a JSON key using a variable in Python?

I'm trying to make a feature with a Discord bot where it counts how many commands have been run and it stores it on a JSON file named data.json. Here is the code:
import json
# Read 'data.json'
with open("data.json", "r") as d:
datar = json.load(d)
com = datar.get("commands-run")
# Functions
def command_increase():
commands_run = com + 1
dataw = open("data.json", "w")
dataw["commands-run"] = commands_run
And here is the JSON file:
{
"commands-run": 0
}
And this is the error I get when I run a command and it tries to increase the value in the JSON file:
TypeError: '_io.TextIOWrapper' object does not support item assignment
On top of that, it also completely wipes the JSON file. By that, I mean that it just clears everything. Even the brackets.

When you do a json.load, it loads your json data into a dict.
You can increase your command counter in this dict and re-write the dict back into your json file at the end of it.
import json
# Read 'data.json'
with open("data.json", "r") as d:
datar = json.load(d)
com = datar.get("commands-run")
# Functions
def command_increase():
commands_run = com + 1
datar["commands-run"] = commands_run
with open("data.json", "w") as dataw:
dataw.write(json.dumps(datar, indent=4))
command_increase()

how to convert json file to csv with "success":true

I have problem with convert json file to csv file on python
and i think it will be the nested json file but i don't know how to handle it!
import json, requests
url = requests.get("https://####/api/food_orders")
text = url.text
data = json.load(text)
order_data = data['data']
# now we will open a file for writing
data_file = open('ordersJsonToCsv.csv', 'w', newline='')
# create the csv writer object
csv_writer = csv.writer(data_file)
# Counter variable used for writing
# headers to the CSV file
count = 0
for ord in order_data:
if count == 0:
# Writing headers of CSV file
header = ord.keys()
csv_writer.writerow(header)
count += 1
# Writing data of CSV file
csv_writer.writerow(ord.values())
data_file.close()
And Json file look like

This code will solve the problem to get data only
import pandas as pd
import json, requests
url = requests.get("https://##/api/orders?
text = url.text
info = json.loads(text)
df = pd.json_normalize(info['data'])
df.to_csv("samplecsv.csv")

Parse JSON to CSV + additional columns

I'm attempting to parse a JSON file with the following syntax into CSV:
{"code":2000,"message":"SUCCESS","data":
{"1":
{"id":1,
"name":"first_name",
"icon":"url.png",
"attribute1":"value",
"attribute2":"value" ...},
"2":
{"id":2,
"name":"first_name",
"icon":"url.png",
"attribute1":"value",
"attribute2":"value" ...},
"3":
{"id":3,
"name":"first_name",
"icon":"url.png",
"attribute1":"value",
"attribute2":"value" ...}, and so forth
}}}
I have found similar questions (e.g. here and here and I am working with the following method:
import requests
import json
import csv
import os
jsonfile = "/path/to.json"
csvfile = "/path/to.csv"
with open(jsonfile) as json_file:
data=json.load(json_file)
data_file = open(csvfile,'w')
csvwriter = csv.writer(data_file)
csvwriter.writerow(data["data"].keys())
for row in data:
csvwriter.writerow(row["data"].values())
data_file.close()
but I am missing something.
I get this error when I try to run:
TypeError: string indices must be integers
and my csv output is:
1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,96
At the end of the day, I am trying to convert the following function (from PowerShell) to Python. This converted the JSON to CSV and added 3 additional custom columns to the end:
$json = wget $lvl | ConvertFrom-Json
$json.data | %{$_.psobject.properties.value} `
| select-object *,#{Name='Custom1';Expression={$m}},#{Name='Level';Expression={$l}},#{Name='Custom2';Expression={$a}},#{Name='Custom3';Expression={$r}} `
| Export-CSV -path $outfile
The output looks like:
"id","name","icon","attribute1","attribute2",..."Custom1","Custom2","Custom3"
"1","first_name","url.png","value","value",..."a","b","c"
"2","first_name","url.png","value","value",..."a","b","c"
"3","first_name","url.png","value","value",..."a","b","c"

As suggested by martineau in a now-deleted answer, my key name was incorrect.
I ended up with this:
import json
import csv
jsonfile = "/path/to.json"
csvfile = "/path/to.csv"
with open(jsonfile) as json_file:
data=json.load(json_file)
data_file = open(csvfile,'w')
csvwriter = csv.writer(data_file)
#get sample keys
header=data["data"]["1"].keys()
#add new fields to dict
keys = list(header)
keys.append("field2")
keys.append("field3")
#write header
csvwriter.writerow(keys)
#for each entry
total = data["data"]
for row in total:
rowdefault = data["data"][str(row)].values()
rowdata = list(rowdefault)
rowdata.append("value1")
rowdata.append("value2")
csvwriter.writerow(rowdata)
Here, I'm grabbing each row by its name id via str(row).

Convert json file to csv

I have this simple python code that converts a json file into a csv.
I would like to convert only the first four values of each key, but i couldn't figure out how to do it.
import json
import csv
# Opening JSON file and loading the data
# into the variable data
with open('personas.json') as json_file:
data = json.load(json_file)
employee_data = data['emp_details']
# now we will open a file for writing
data_file = open('data_file.csv', 'w')
# create the csv writer object
csv_writer = csv.writer(data_file)
# Counter variable used for writing
# headers to the CSV file
count = 0
for emp in employee_data:
if count == 0:
# Writing headers of CSV file
header = emp.keys()
csv_writer.writerow(header)
count += 1
# Writing data of CSV file
csv_writer.writerow(emp.values())
data_file.close()
Here is an example of json file's format
{"emp_details":[
{
"DATAID":"6908443",
"FIRST_NAME":"Fernando",
"SECOND_NAME":"Fabbiano",
"THIRD_NAME":"Agustin",
"FOURTH_NAME":"",
"AGE": "21",
"EMAIL": "fer.fab#gmail.com"
}
]}
And as i said, i would like to convert only the fields DATAID, FIRSTNAME, SECONDNAME, THIRD NAME.

Loop over list of file paths, read in JSON files, insert into MongoDB using Pymongo

I have a file folder of 1000+ json metadata files. I have created a list of the file paths and I'm trying to:
for each file path, read json file
pull in only the key value pairs I'm interested in
store it in a variable or save it in a way that I can insert into
mongodb using pymongo
I have been successful listing the file paths to a variable and loading ONE json doc (from one file path). The problem is I need to do over a thousand and I get an error when trying to incorporate list of file paths and loop.
Here's what I've tried so far:
import pymongo
import json
filename = r"C:\Users\Documents\FileFolder\randomFile.docx.json"
with open(filename, "r", encoding = "utf8") as f:
json_doc = json.load(f)
new_jsonDoc = dict()
for key in {'Application-Name', 'Author', 'resourceName', 'Line-Count', 'Page-Count', 'Paragraph-Count', 'Word-Count'}:
new_jsonDoc[key] = json_doc[0][key]
Sample output:
{'Application-Name': 'Microsoft Office Word',
'Author': 'Sample, John Q.',
'Character Count': '166964',
'Line-Count': '1391',
'Page-Count': '103',
'Paragraph-Count': '391',
'Word-Count': '29291',
'resourceName': 'randomFile.docx'}
Now when I add the loop:
for file in list_JsonFiles: # this is list of file paths created by os.walk
# if I do a print(type(file)) here, file type is a string
with open(file, "r") as f:
# print(type(file)) = string, print(type(f)) = TextIOWrapper
json_doc = json.loads(f)
### TypeError: the JSON object must be str, bytes or bytearray, not TextIOWrapper ###
How can I get my loop working? Is my approach wrong?

Figured the TypeError out:
for file in list_JsonFiles:
with open(file, "r", encoding = "utf8") as f:
json_doc = json.load(f)
new_jsonDoc = dict()
for key in {'Application-Name', 'Author', 'resourceName', 'Line-Count', 'Page-Count', 'Paragraph-Count', 'Word-Count'}:
if key in json_doc[0]:
new_jsonDoc[key] = json_doc[0][key]

Categories