We Keep Coding

CSV to JSON in Python takes only first and last rows

I made a function that converts data from given csv file to a json object, and the weird thing is that it only gets the first and last element of the CSV.
My csv structure is 2 columns: name,days
Example:
name,days
John,17
Fred,2
Michelle,22
When I get the json object, and print it, it gives me:
jsondata is: {
"0": {
"name": "John",
"days": "17"
},
"1": {
"name": "Michelle",
"days": "22"
}
}
Here is my code:
data = {}
with open(file, "rt") as csvf:
csvReader = csv.DictReader(csvf)
i = 0
for rows in csvReader:
data[i] = rows
i =+ 1
jsondata = json.dumps(data, indent=4)
print("jsondata is: ", jsondata)

I do not understand why would you need to use i as a counter but here a two suggestions:
data = []
with open(file, "rt") as csvf:
csvReader = csv.DictReader(csvf)
for rows in csvReader:
data.append(rows)
jsondata = json.dumps(data, indent=4)
print("jsondata is: ", jsondata)
or
data = {}
with open(file, "rt") as csvf:
csvReader = csv.DictReader(csvf)
for i, rows in enumerate(csvReader):
data[i] = rows
jsondata = json.dumps(data, indent=4)
print("jsondata is: ", jsondata)
In both cases you will be able to access by index e.g.: data[0], data[1]...

Converting CSV to jSON - Keep getting "End of File expected" error

I'm trying to convert a CSV file into a jSON file in order to then inject it into a Firebase database.
csvfile = open('final_df_2.csv', 'r')
jsonfile = open('final_df_5.json', 'w')
reader = csv.DictReader(csvfile)
for row in reader:
json.dump({row['ballID']: [{"colour": row['colour'], "radius":row['radius']}]}, jsonfile)
jsonfile.write('\n')
Unfortunately, I keep getting an "End of File expected" error
Here's my JSON's output
{
"001": [
{
"colour": "green",
"radius": "199405.0"
}
]
}
{
"002": [
{
"colour": "blue",
"radius": "199612.0"
}
]
}
In addition, Firebase sends back an error when I try to import the JSON file saying "Invalid JSON files"

You could collect all the data into a python list and dump that list to the json file:
csvfile = open('final_df_2.csv', 'r')
reader = csv.DictReader(csvfile)
jsoncontent = []
for row in reader:
jsoncontent.append({row['ballID']: [{"colour": row['colour'], "radius":row['radius']}]})
with open('final_df_5.json', 'w') as jsonfile:
json.dump(jsoncontent, jsonfile)
However, I'm not sure what your firebase database is expecting.

Convert CSV file to JSON with python

I am trying to covert my CSV email list to a JSON format to mass email via API. This is my code thus far but am having trouble with the output. Nothing is outputting on my VS code editor.
import csv
import json
def make_json(csvFilePath, jsonFilePath):
data = {}
with open(csvFilePath, encoding='utf-8') as csvf:
csvReader = csv.DictReader(csvf)
for rows in csvReader:
key = rows['No']
data[key] = rows
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
jsonf.write(json.dumps(data, indent=4))
csvFilePath = r'/data/csv-leads.csv'
jsonFilePath = r'Names.json'
make_json(csvFilePath, jsonFilePath)
Here is my desired JSON format
{
"EmailAddress": "hello#youngstowncoffeeseattle.com",
"Name": "Youngstown Coffee",
"ConsentToTrack": "Yes"
},
Heres my CSV list
No,EmailAddress,ConsentToTrack
Zylberschtein's Delicatessen & Bakery,catering#zylberschtein.com,Yes
Youngstown Coffee,hello#youngstowncoffeeseattle.com,Yes

It looks like you could use a csv.DictReader to make this easier.
If I have data.csv that looks like this:
Name,EmailAddress,ConsentToTrack
Zylberschtein's Delicatessen,catering#zylberschtein.com,yes
Youngstown Coffee,hello#youngstowncoffeeseattle.com,yes
I can convert it into JSON like this:
>>> import csv
>>> import json
>>> fd = open('data.csv')
>>> reader = csv.DictReader(fd)
>>> print(json.dumps(list(reader), indent=2))
[
{
"Name": "Zylberschtein's Delicatessen",
"EmailAddress": "catering#zylberschtein.com",
"ConsentToTrack": "yes"
},
{
"Name": "Youngstown Coffee",
"EmailAddress": "hello#youngstowncoffeeseattle.com",
"ConsentToTrack": "yes"
}
]
Here I've assumed the headers in the CSV can be used verbatim. I'll update this with an exmaple if you need to modify key names (e.g. convert "No" to "Name"),.
If you need to rename a column, it might look more like this:
import csv
import json
with open('data.csv') as fd:
reader = csv.DictReader(fd)
data = []
for row in reader:
row['Name'] = row.pop('No')
data.append(row)
print(json.dumps(data, indent=2))
Given this input:
No,EmailAddress,ConsentToTrack
Zylberschtein's Delicatessen,catering#zylberschtein.com,yes
Youngstown Coffee,hello#youngstowncoffeeseattle.com,yes
This will output:
[
{
"EmailAddress": "catering#zylberschtein.com",
"ConsentToTrack": "yes",
"Name": "Zylberschtein's Delicatessen"
},
{
"EmailAddress": "hello#youngstowncoffeeseattle.com",
"ConsentToTrack": "yes",
"Name": "Youngstown Coffee"
}
]
and to print on my editor is it simply print(json.dumps(list(reader), indent=2))?
I'm not really familiar with your editor; print is how you generate console output in Python.

Convert csv to json - column1 as a key (nested dict)

I need column1 to be used as a KEY and its value to be a dict of column2 (as key) & column3 (as value) so that output will become a nested dictonary.
For example:
I have a csv file as shown below:
customer1,subkey1,val1
customer2,subkey2,val2
customer2,subkey3,val3
customer2,subkey4,val4
customer3,subkey5,val5
customer3,subkey6,val6
expecting output to be:
{
customer1: {
subkey1:val1
},
customer2: {
subkey2:val2,
subkey3:val3,
subkey4:val4
},
customer3: {
subkey5:val5,
subkey6:val6
}
}
I have tried to convert below sample code as per my requirement, but no luck:
import csv
import json
def csv_to_json(csvFilePath, jsonFilePath):
jsonArray = []
#read csv file
with open(csvFilePath, encoding='utf-8') as csvf:
#load csv file data using csv library's dictionary reader
csvReader = csv.DictReader(csvf)
#convert each csv row into python dict
for row in csvReader:
#add this python dict to json array
jsonArray.append(row)
#convert python jsonArray to JSON String and write to file
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
jsonString = json.dumps(jsonArray, indent=4)
jsonf.write(jsonString)
csvFilePath = r'data.csv'
jsonFilePath = r'data.json'
csv_to_json(csvFilePath, jsonFilePath)

Try this
worked for me I used the csv.reader for this and changed the variable into a dict object
import csv
import json
def csv_to_json(csvFilePath, jsonFilePath):
jsonDict = {}
# read csv file
with open(csvFilePath, encoding='utf-8') as csvf:
# load csv file data using csv library's dictionary reader
csvReader = csv.reader(csvf, delimiter=',')
# convert each csv row into python dict
for row in csvReader:
# add this python dict to json array
if row[0] in jsonDict:
jsonDict[row[0]][row[1]] = row[2]
else:
jsonDict[row[0]] = {row[1]: row[2]}
# convert python jsonArray to JSON String and write to file
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
jsonString = json.dumps(jsonDict, indent=4)
jsonf.write(jsonString)
csvFilePath = r'data.csv'
jsonFilePath = r'data.json'
csv_to_json(csvFilePath, jsonFilePath)
output
{
"customer1": {
"subkey1": "val1"
},
"customer2": {
"subkey2": "val2",
"subkey3": "val3",
"subkey4": "val4"
},
"customer3": {
"subkey5": "val5",
"subkey6": "val6"
}
}

import collections
import csv
with open('file.csv') as f:
reader = csv.reader(f, delimiter=',')
dict_1 = collections.defaultdict(dict)
for row in reader:
dict_1[row[0]][row[1]] = row[2]
print(dict(dict_1))
# Output
{
customer1: {
subkey1:val1
},
customer2: {
subkey2:val2,
subkey3:val3,
subkey4:val4
},
customer3: {
subkey5:val5,
subkey6:val6
}
}

In this case, DictReader doesn't really help you because it gives each row as a dict in the form:
{"column1": "customer1", "column2": "subkey1", "column3": "val1"}
So it will actually be simpler to use a regular reader and parse manually. You just need to expand existing customers, so it will be helpful to use a defaultdict:
import csv
import json
from collections import defaultdict
def csv_to_json(csvFilePath, jsonFilePath):
jsonDict = defaultdict(dict)
# read csv file
with open(csvFilePath, encoding='utf-8') as csvf:
# load csv file data using csv library's reader
csvReader = csv.reader(csvf)
# convert each csv row into a list
for row in csvReader:
# add this list to json dict
jsonDict[row[0]][row[1]] = row[2]
# write python jsonDict to JSON file
with open(jsonFilePath, 'w', encoding='utf-8') as jsonf:
json.dump(jsonDict, jsonf, indent=4)
Note that the json file can be written simpler by using dump instead of dumps.

Python CSV to JSON W/ Array Output

I'm trying to take data from a CSV and put it in a top-level array in JSON format.
Currently I am running this code:
import csv
import json
csvfile = open('music.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("ID","Artist","Song", "Artist")
reader = csv.DictReader( csvfile, fieldnames)
for row in reader:
json.dump(row, jsonfile)
jsonfile.write('\n')
The CSV file is formatted as so:
| 1 | Empire of the Sun | We Are The People | Walking on a Dream |
| 2 | M83 | Steve McQueen | Hurry Up We're Dreaming |
Where = Column 1: ID | Column 2: Artist | Column 3: Song | Column 4: Album
And getting this output:
{"Song": "Empire of the Sun", "ID": "1", "Artist": "Walking on a Dream"}
{"Song": "M83", "ID": "2", "Artist": "Hurry Up We're Dreaming"}
I'm trying to get it to look like this though:
{
"Music": [
{
"id": 1,
"Artist": "Empire of the Sun",
"Name": "We are the People",
"Album": "Walking on a Dream"
},
{
"id": 2,
"Artist": "M83",
"Name": "Steve McQueen",
"Album": "Hurry Up We're Dreaming"
},
]
}

Pandas solves this really simply. First to read the file
import pandas
df = pandas.read_csv('music.csv', names=("id","Artist","Song", "Album"))
Now you have some options. The quickest way to get a proper json file out of this is simply
df.to_json('file.json', orient='records')
Output:
[{"id":1,"Artist":"Empire of the Sun","Song":"We Are The People","Album":"Walking on a Dream"},{"id":2,"Artist":"M83","Song":"Steve McQueen","Album":"Hurry Up We're Dreaming"}]
This doesn't handle the requirement that you want it all in a "Music" object or the order of the fields, but it does have the benefit of brevity.
To wrap the output in a Music object, we can use to_dict:
import json
with open('file.json', 'w') as f:
json.dump({'Music': df.to_dict(orient='records')}, f, indent=4)
Output:
{
"Music": [
{
"id": 1,
"Album": "Walking on a Dream",
"Artist": "Empire of the Sun",
"Song": "We Are The People"
},
{
"id": 2,
"Album": "Hurry Up We're Dreaming",
"Artist": "M83",
"Song": "Steve McQueen"
}
]
}
I would advise you to reconsider insisting on a particular order for the fields since the JSON specification clearly states "An object is an unordered set of name/value pairs" (emphasis mine).

Alright this is untested, but try the following:
import csv
import json
from collections import OrderedDict
fieldnames = ("ID","Artist","Song", "Artist")
entries = []
#the with statement is better since it handles closing your file properly after usage.
with open('music.csv', 'r') as csvfile:
#python's standard dict is not guaranteeing any order,
#but if you write into an OrderedDict, order of write operations will be kept in output.
reader = csv.DictReader(csvfile, fieldnames)
for row in reader:
entry = OrderedDict()
for field in fieldnames:
entry[field] = row[field]
entries.append(entry)
output = {
"Music": entries
}
with open('file.json', 'w') as jsonfile:
json.dump(output, jsonfile)
jsonfile.write('\n')

Your logic is in the wrong order. json is designed to convert a single object into JSON, recursively. So you should always be thinking in terms of building up a single object before calling dump or dumps.
First collect it into an array:
music = [r for r in reader]
Then put it in a dict:
result = {'Music': music}
Then dump to JSON:
json.dump(result, jsonfile)
Or all in one line:
json.dump({'Music': [r for r in reader]}, jsonfile)
"Ordered" JSON
If you really care about the order of object properties in the JSON (even though you shouldn't), you shouldn't use the DictReader. Instead, use the regular reader and create OrderedDicts yourself:
from collections import OrderedDict
...
reader = csv.Reader(csvfile)
music = [OrderedDict(zip(fieldnames, r)) for r in reader]
Or in a single line again:
json.dump({'Music': [OrderedDict(zip(fieldnames, r)) for r in reader]}, jsonfile)
Other
Also, use context managers for your files to ensure they're closed properly:
with open('music.csv', 'r') as csvfile, open('file.json', 'w') as jsonfile:
# Rest of your code inside this block

It didn't write to the JSON file in the order I would have liked
The csv.DictReader classes return Python dict objects. Python dictionaries are unordered collections. You have no control over their presentation order.
Python does provide an OrderedDict, which you can use if you avoid using csv.DictReader().
and it skipped the song name altogether.
This is because the file is not really a CSV file. In particular, each line begins and ends with the field separator. We can use .strip("|") to fix this.
I need all this data to be output into an array named "Music"
Then the program needs to create a dict with "Music" as a key.
I need it to have commas after each artist info. In the output I get I get
This problem is because you call json.dumps() multiple times. You should only call it once if you want a valid JSON file.
Try this:
import csv
import json
from collections import OrderedDict
def MyDictReader(fp, fieldnames):
fp = (x.strip().strip('|').strip() for x in fp)
reader = csv.reader(fp, delimiter="|")
reader = ([field.strip() for field in row] for row in reader)
dict_reader = (OrderedDict(zip(fieldnames, row)) for row in reader)
return dict_reader
csvfile = open('music.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("ID","Artist","Song", "Album")
reader = MyDictReader(csvfile, fieldnames)
json.dump({"Music": list(reader)}, jsonfile, indent=2)