Related
I read a string containing a json document.
d2 = json.loads(s1)
I am getting data in this format, a list of dictionnaries.
[{'creati_id': 123,
'creativ_id': 234,
'status': 'adsc',
'name': 'seded',
…
'video_75_views': None,
'video_100_views': None,
'estimated': None,
'creative1': 1.0,
'creative': 'Excellent',
'value': 1.023424324}]}
How can I save this data in CSV format?
This can easily be achieved with the csv module:
import csv
data = [
{
"creati_id": 123,
"creativ_id": 234,
"status": "adsc",
"name": "seded",
}
]
with open("data_file.csv", "w") as data_file:
csv_writer = csv.writer(data_file)
header = data[0].keys()
csv_writer.writerow(header)
for line in data:
csv_writer.writerow(line.values())
You can use the standard csv library in Python to write CSV files. From your question, I'm assuming that you have multiple rows, each having the structure you shared. If that's the case, then something like this should do the trick:
import csv
json1 = [
{'creati_id': 123, 'creativ_id': 234, 'status': 'adsc', 'name': 'seded', 'email': None, 'brand': 'adc', 'market': 'dcassca', 'channel': 'dAD'},
{'creati_id': 123, 'creativ_id': 234, 'status': 'adsc', 'name': 'seded', 'email': None, 'brand': 'adc', 'market': 'dcassca', 'channel': 'dAD'}
]
header_names = json1[0].keys() # Extract the header names
data_rows = [row.values() for row in json1] # Extract the values for each
with open('output.csv', 'w', encoding='UTF8', newline='') as file:
writer = csv.writer(file)
writer.writerow(header_names) # Writes the header
writer.writerows(data_rows) # Writes the rows
I am a beginner and I've tried searching online everywhere, but I'm not sure I'm searching the right terms.
My CSV file looks this:
https://drive.google.com/file/d/0B74bmJNIxxW-dWl0Y0dsV1E4bjA/view?usp=sharing
I want to know how to use the CSV file to do something like this,
Email
driver.find_element_by_name('emailAddress').send_keys("johndoe#example.com")
print "Successfully Entered Email..."
There are lots of ways that you could do this. One would be to use the csv module.
with open("foo.csv", "r") as fh:
lines = csv.reader(fh)
for line in lines:
address = line[0]
driver.find_element_by_name('emailAddress').send_keys(address)
It really helps to post the data here so that we see what the format really is and run code ourselves. So, I invented some sample data
emails.csv
Email,Password,First Name,Last Name,City
foo1#example.com,frobinate,John,Doe,District Heights
foo2#example.com,frobinate,John,Doe,District Heights
foo3#example.com,frobinate,John,Doe,District Heights
foo4#example.com,frobinate,John,Doe,District Heights
I can use the csv module to read that. csv.DictReader reads each row into its own dict that lets me reference cells by the name given in the header. Since I'll be looking up records by email name later, I'll read it into another dict that will act as an index into the records. If the same user is in there multiple times, only the last one will be remembered.
With the index in place, I can grab the row by email name.
>>> import csv
>>> with open('emails.csv', newline='') as fp:
... reader = csv.DictReader(fp) # auto-reads header
... for row in reader:
... email_index[row['Email']] = row
...
>>> for item in email_index.items():
... print(item)
...
('foo3#example.com', {'Email': 'foo3#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo2#example.com', {'Email': 'foo2#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo4#example.com', {'Email': 'foo4#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo1#example.com', {'Email': 'foo1#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
>>>
>>> user = 'foo1#example.com'
>>> record = email_index[user]
>>> print("{Email} is {First Name} {Last Name} and lives in {City}".format(**record))
foo4#example.com is John Doe and lives in District Heights
>>>
This question already has answers here:
List of dicts to/from dict of lists
(14 answers)
Closed 6 years ago.
I have a dictionary that stores a list of items for each key as shown:
name_dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
I want to print the data in the dictionary as table format:
FirstName,MiddleName,LastName # the keys of the dictionary
Elizabeth,H.,Perkins #the values of the keys in one line (the first item in the list)
Scott,T.,Joseph #the values of the keys in new line (the following item in the list)
How to solve this problem?
I have tried doing the suggested solution by Gareth Latty, but that did not work.
with open('C:/Output.csv',"w+") as file:
w = csv.DictWriter(file,name_dict.keys())
w.writeheader()
w.writerow(name_dict)
It outputs the following:
MiddleName,LastName,FirstName
"['H.', 'T.']","['Perkins', 'Joseph']","['Perkins', 'Joseph']"
Any idea how to output the values (the item in the list) of each of the keys in new row?
csv.DictWriter expects a dictionary with the field:single_line pairs for each row which is unfortunately not what you have, you basically need to convert your data structure to be a list of dicts for the single lines:
[{'MiddleName': 'H.', 'FirstName': 'Elizabeth ', 'LastName': 'Perkins'}, {'MiddleName': 'T.', 'FirstName': 'Scott ', 'LastName': 'Joseph'}]
You can convert it with something like this:
import csv
def seperate_entries(dict_of_lists):
iters = [(k,iter(v)) for k,v in dict_of_lists.items()]
try:
while True:
yield {k:next(v) for k,v in iters}
except StopIteration:
return
name_dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
with open('sample.csv',"w+") as file:
w = csv.DictWriter(file,name_dict.keys())
w.writeheader()
w.writerows(seperate_entries(name_dict))
I think you misuse dict. When you have multiple values, you should use a list of dicts and not a dict which the values are lists. Instead of
Dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
You should do:
Dict = [{'FirstName': 'Elizabeth', 'MiddleName': 'H.', 'LastName': 'Perkins'}, {'FirstName': 'Joseph', 'MiddleName': 'T. ', 'LastName': 'Scott'}]
or in a more readable version:
Dict = [
{'FirstName': 'Elizabeth', 'MiddleName': 'H.', 'LastName': 'Perkins'},
{'FirstName': 'Joseph', 'MiddleName': 'T. ', 'LastName': 'Scott' }
]
If you want to print one line (one dictionary of the list), you can do something like that:
def printDict(d):
print d["FirstName"] + "," + d["MiddleName"] + "," + d["LastName"]
And if you want to print each of the elements in the list you have:
def printList(l):
for i in l:
printDict(i)
And just use it like that:
printList(Dict)
With your first (original) Dict, accessing Dict["FirstName"] would return a list, and when printed it would print as:
["Elizabeth", "Joesph"]
But with the second (new way I suggested) Dict, accessing Dict[0]["FirstName"] would return a string, and will print like:
Elizabeth
To access the keys in the Dictionary, you just need to do the following:
middleNames=Dict['Middlename']
firstNames=Dict['FirstName']
lastNames=Dict['LastName']
You now have access to the values stored in the inner list information, this can similarly be accessed by the following:
# Find how many items read (optional)
len(middleNames)
# Then iterate through the items
for mName in middleName:
print mName # this will print each list item e.g.
H.
T.
# for brevity, this is the same as doing this...
middleName[0] etc
Hope this helps.
You need define how many row you have.
Just flat it into rows with all keys in Dict.
import csv
Dict = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
len_row = 2
with open('Output.csv', "w+") as file:
w = csv.DictWriter(file, Dict.keys())
w.writeheader()
for i in range(len_row):
row = {}
for k in Dict.keys():
for v in Dict.values():
row[k] = Dict[k][i]
w.writerow(row)
I'm sure there are more efficient ways of achieving what you want but this lays down a simple outline of what you want and shows how you can achieve it.
names = {'MiddleName': ['H.', 'T.'], 'LastName': ['Perkins', 'Joseph'], 'FirstName': ['Elizabeth ', 'Scott ']}
output = open('output.txt','w')
#NOTE: this will write headers in alphabetical format. You will need to change that so it follows the FirstName, MiddleName, LastName pattern
for key in sorted(names.keys()):
output.write(key + ',')
output.write('\n')
#assuming we always have same amount of middle, first, and last names
for i in range(len(names['FirstName'])):
personFname = ''
personLname = ''
personMname = ''
for key in names.keys():
if key == 'MiddleName':
personMname = names[key][i]
elif key == 'FirstName':
personFname = names[key][i]
elif key == 'LastName':
personLname = names[key][i]
output.write(personFname + ',' + personMname + ',' + personLname)
output.write('\n')
output.close()
So here is my issue, I have created a Dictionary for my lists and am trying to add the data I find and append it to each row but instead it is just appending to the same column with the same data type.
How could I get it so each new append add to new row.
data_dict = {'contact name': [], 'name': [], 'telephone': [], 'email': [],
'mobile': [], 'feedback average': []}
try:
data_dict['telephone'].append(soup.find('span',itemprop='telephone').text)
except AttributeError:
data_dict['telephone'].append('No telephone')
print data_dict
field_names = fn = data_dict.keys()
with open('./file.csv','w') as csvfile:
f = csv.DictWriter(csvfile, fieldnames=fn)
f.writeheader()
f.writerow(data_dict)
Try something like this:
data_dict = {'contact name': [], 'name': [], 'telephone': [], 'email': [],
'mobile': [], 'feedback average': []}
try:
data_dict['telephone'].append(soup.find('span',itemprop='telephone').text)
except AttributeError:
data_dict['telephone'].append('No telephone')
print data_dict
fn = data_dict.keys()
with open('./file.csv','w') as csvfile:
f = csv.reader(csvfile)
for row in f:
for i in len(fn):
data_dict[fn[i]].append(row[i])
This should work for you, if I got you right.
But care, this requires that one row in the csv contains exactly the elements of your dictionary, in the correct order.
If this isn't the case, you will need to find out which value is written in which column, and then add the value of this column to the list in your dictionary.
So you would need to replace
for i in len(fn):
data_dict[fn[i]].append(row[i])
by
for k in fn:
data_dict[k].append(row[columns[k]])
where columns is a dictionary that contains the same keys as data_dict, and as the values the columns in which the data of the specific key is stored in the csv-file. For an example, columns could look like this:
columns = {'contact name': 1, 'name': 3, 'telephone' : 6, 'email': 7, 'mobile':8, 'feedback average': 2}
This question already has answers here:
How do I read and write CSV files with Python?
(7 answers)
Closed 3 months ago.
"Type","Name","Description","Designation","First-term assessment","Second-term assessment","Total"
"Subject","Nick","D1234","F4321",10,19,29
"Unit","HTML","D1234-1","F4321",18,,
"Topic","Tags","First Term","F4321",18,,
"Subtopic","Review of representation of HTML",,,,,
All the above are the value from an excel sheet , which is converted to csv and that is the one shown above
The header as you notice contains seven coulmns,the data below them vary,
I have this script to generate these from python script,the script is below
from django.db import transaction
import sys
import csv
import StringIO
file = sys.argv[1]
no_cols_flag=0
flag=0
header_arr=[]
print file
f = open(file, 'r')
while (f.readline() != ""):
for i in [line.split(',') for line in open(file)]: # split on the separator
print "==========================================================="
row_flag=0
row_d=""
for j in i: # for each token in the split string
row_flag=1
print j
if j:
no_cols_flag=no_cols_flag+1
data=j.strip()
print j
break
How to modify the above script to say that this data belongs to a particular column header..
thanks..
You're importing the csv module but never use it. Why?
If you do
import csv
reader = csv.reader(open(file, "rb"), dialect="excel") # Python 2.x
# Python 3: reader = csv.reader(open(file, newline=""), dialect="excel")
you get a reader object that will contain all you need; the first row will contain the headers, and the subsequent rows will contain the data in the corresponding places.
Even better might be (if I understand you correctly):
import csv
reader = csv.DictReader(open(file, "rb"), dialect="excel") # Python 2.x
# Python 3: reader = csv.DictReader(open(file, newline=""), dialect="excel")
This DictReader can be iterated over, returning a sequence of dicts that use the column header as keys and the following data as values, so
for row in reader:
print(row)
will output
{'Name': 'Nick', 'Designation': 'F4321', 'Type': 'Subject', 'Total': '29', 'First-term assessment': '10', 'Second-term assessment': '19', 'Description': 'D1234'}
{'Name': 'HTML', 'Designation': 'F4321', 'Type': 'Unit', 'Total': '', 'First-term assessment': '18', 'Second-term assessment': '', 'Description': 'D1234-1'}
{'Name': 'Tags', 'Designation': 'F4321', 'Type': 'Topic', 'Total': '', 'First-term assessment': '18', 'Second-term assessment': '', 'Description': 'First Term'}
{'Name': 'Review of representation of HTML', 'Designation': '', 'Type': 'Subtopic', 'Total': '', 'First-term assessment': '', 'Second-term assessment': '', 'Description': ''}