This question already has answers here:
How do I read and write CSV files with Python?
(7 answers)
Closed 3 months ago.
"Type","Name","Description","Designation","First-term assessment","Second-term assessment","Total"
"Subject","Nick","D1234","F4321",10,19,29
"Unit","HTML","D1234-1","F4321",18,,
"Topic","Tags","First Term","F4321",18,,
"Subtopic","Review of representation of HTML",,,,,
All the above are the value from an excel sheet , which is converted to csv and that is the one shown above
The header as you notice contains seven coulmns,the data below them vary,
I have this script to generate these from python script,the script is below
from django.db import transaction
import sys
import csv
import StringIO
file = sys.argv[1]
no_cols_flag=0
flag=0
header_arr=[]
print file
f = open(file, 'r')
while (f.readline() != ""):
for i in [line.split(',') for line in open(file)]: # split on the separator
print "==========================================================="
row_flag=0
row_d=""
for j in i: # for each token in the split string
row_flag=1
print j
if j:
no_cols_flag=no_cols_flag+1
data=j.strip()
print j
break
How to modify the above script to say that this data belongs to a particular column header..
thanks..
You're importing the csv module but never use it. Why?
If you do
import csv
reader = csv.reader(open(file, "rb"), dialect="excel") # Python 2.x
# Python 3: reader = csv.reader(open(file, newline=""), dialect="excel")
you get a reader object that will contain all you need; the first row will contain the headers, and the subsequent rows will contain the data in the corresponding places.
Even better might be (if I understand you correctly):
import csv
reader = csv.DictReader(open(file, "rb"), dialect="excel") # Python 2.x
# Python 3: reader = csv.DictReader(open(file, newline=""), dialect="excel")
This DictReader can be iterated over, returning a sequence of dicts that use the column header as keys and the following data as values, so
for row in reader:
print(row)
will output
{'Name': 'Nick', 'Designation': 'F4321', 'Type': 'Subject', 'Total': '29', 'First-term assessment': '10', 'Second-term assessment': '19', 'Description': 'D1234'}
{'Name': 'HTML', 'Designation': 'F4321', 'Type': 'Unit', 'Total': '', 'First-term assessment': '18', 'Second-term assessment': '', 'Description': 'D1234-1'}
{'Name': 'Tags', 'Designation': 'F4321', 'Type': 'Topic', 'Total': '', 'First-term assessment': '18', 'Second-term assessment': '', 'Description': 'First Term'}
{'Name': 'Review of representation of HTML', 'Designation': '', 'Type': 'Subtopic', 'Total': '', 'First-term assessment': '', 'Second-term assessment': '', 'Description': ''}
Related
I have two files where I want to compare value of firstname, lastname and account_id between the two files in Python.
Content of the two files are as below
users.csv contains below content where column names are first_name,last_name,id respectively
Test1,Last1,1101011
Test2,Last2,1231231
user1.txt contains
[{'firstname': 'Test1',
'time_utc': 1672889600.0,
'lastname': 'Last1',
'name': 'Test Last1',
'account_id': 1101011},
{'firstname': 'Test2',
'time_utc': None,
'lastname': 'Last2',
'name': 'Test2 Last2',
'account_id': 1231231}]
I tried with below code but it returns false result and returns all the users even though they are present in both the files.
import csv
with open ("users.csv", 'r') as read_csv:
f = csv.reader(read_csv)
with open ("user1.txt", 'r') as file:
x = file.readlines()
print (x)
for row in f:
print(row)
if row not in x:
print (row)
I read a string containing a json document.
d2 = json.loads(s1)
I am getting data in this format, a list of dictionnaries.
[{'creati_id': 123,
'creativ_id': 234,
'status': 'adsc',
'name': 'seded',
…
'video_75_views': None,
'video_100_views': None,
'estimated': None,
'creative1': 1.0,
'creative': 'Excellent',
'value': 1.023424324}]}
How can I save this data in CSV format?
This can easily be achieved with the csv module:
import csv
data = [
{
"creati_id": 123,
"creativ_id": 234,
"status": "adsc",
"name": "seded",
}
]
with open("data_file.csv", "w") as data_file:
csv_writer = csv.writer(data_file)
header = data[0].keys()
csv_writer.writerow(header)
for line in data:
csv_writer.writerow(line.values())
You can use the standard csv library in Python to write CSV files. From your question, I'm assuming that you have multiple rows, each having the structure you shared. If that's the case, then something like this should do the trick:
import csv
json1 = [
{'creati_id': 123, 'creativ_id': 234, 'status': 'adsc', 'name': 'seded', 'email': None, 'brand': 'adc', 'market': 'dcassca', 'channel': 'dAD'},
{'creati_id': 123, 'creativ_id': 234, 'status': 'adsc', 'name': 'seded', 'email': None, 'brand': 'adc', 'market': 'dcassca', 'channel': 'dAD'}
]
header_names = json1[0].keys() # Extract the header names
data_rows = [row.values() for row in json1] # Extract the values for each
with open('output.csv', 'w', encoding='UTF8', newline='') as file:
writer = csv.writer(file)
writer.writerow(header_names) # Writes the header
writer.writerows(data_rows) # Writes the rows
import csv
keys = ["id", "name", "age", "height", "weight"]
with open('temp.csv', 'w') as temp_file:
dict_writer_obj = csv.DictWriter(temp_file, fieldnames = keys)
with open('dictReader.csv','r') as file:
dict_reader_obj = csv.DictReader(file)
dict_writer_obj.writeheader()
dict_writer_obj.writerows(file)
I want to convert a csv file called dictReader.csv file into dictionary based file:
However I am getting the following error. Any ideas?
AttributeError: 'str' object has no attribute 'keys'
My dictReader.csv file content:
id,name,age,height,weight
1,Alice,20,62,120.6
2,Freddie,21,74,190.6
3,Bob,17,68,120.0
Desired output file called temp.csv with this format
{'id': '1', 'name': 'Alice', 'age': '20', 'height': '62', 'weight': '120.6'}
{'id': '2', 'name': 'Freddie', 'age': '21', 'height': '74', 'weight': '190.6'}
{'id': '3', 'name': 'Bob', 'age': '17', 'height': '68', 'weight': '120.0'}
To improve on the other user's answer a bit, you can still use writerows like this.
import csv
keys = ["id", "name", "age", "height", "weight"]
with open('temp.csv', 'w') as temp_file:
dict_writer_obj = csv.DictWriter(temp_file, fieldnames = keys)
with open('dictReader.csv','r') as file:
dict_reader_obj = csv.DictReader(file)
dict_writer_obj.writeheader()
# Here:
dict_writer_obj.writerows(row for row in dict_reader_obj)
Just change:
dict_writer_obj.writerows(file)
to:
dict_writer_obj.writerows(row for row in dict_reader_obj)
Or row by row using .writerow():
for row in dict_reader_obj:
dict_writer_obj.writerow(row)
I am looping through an API to retrieve data for multiple ICO tokens. Now, I would like to save the data to a csv with variables in columns and 1 row for each ICO token. The basic code works, I have 2 problems:
- entries are written only in every second line, which is quite unpractical. How can I specify not to leave rows blank?
- the variable price is a list itself and thus saved in as a single item (with > 1 variables inside). How can I decompose the list to write one variable per column?
See my code here:
ICO_Wallet = '0xe8ff5c9c75deb346acac493c463c8950be03dfba',
'0x7654915a1b82d6d2d0afc37c52af556ea8983c7e',
'0x4DF812F6064def1e5e029f1ca858777CC98D2D81'
for index, Wallet in enumerate(ICO_Wallet) :
Name = ICO_name[index]
Number = ICO_No[index]
try:
URL = 'http://api.ethplorer.io/getTokenInfo/' + Wallet + '?apiKey=freekey'
except:
print(Wallet)
json_obj = urlopen(URL)
data = json.load(json_obj)
with open('token_data_test.csv','a') as f:
w = csv.writer(f, delimiter=";")
w.writerow(data.values())
time.sleep(1)
Sample output:
data Out[59]:
{'address': '0x8a854288a5976036a725879164ca3e91d30c6a1b',
'countOps': 24207,
'decimals': '18',
'ethTransfersCount': 0,
'holdersCount': 10005,
'issuancesCount': 0,
'lastUpdated': 1542599890,
'name': 'GET',
'owner': '0x9a417e4db28778b6d9a4f42a5d7d01252a3af849',
'price': {'availableSupply': '11388258.0',
'currency': 'USD',
'diff': -20.71,
'diff30d': -14.155971452386,
'diff7d': -22.52,
'marketCapUsd': '2814942.0',
'rate': '0.2471792958',
'ts': '1542641433',
'volume24h': '2371.62380719'},
'symbol': 'GET',
'totalSupply': '33368773400000170376363910',
'transfersCount': 24207}
As mentioned, it's an easy fix for the first problem, just modify the csv.writer line like this:
w = csv.writer(f, delimiter=";", lineterminator='\n')
For your second problem, you can flatten your json before passing into csv:
for k, v in data.pop('price').items():
data['price_{}'.format(k)] = v
This changes all items under price into price_itemname as a flattened key. The .pop() method also helps remove the 'price' key at the same time.
Result:
{'address': '0x8a854288a5976036a725879164ca3e91d30c6a1b',
'countOps': 24207,
'decimals': '18',
'ethTransfersCount': 0,
'holdersCount': 10005,
'issuancesCount': 0,
'lastUpdated': 1542599890,
'name': 'GET',
'owner': '0x9a417e4db28778b6d9a4f42a5d7d01252a3af849',
'price_availableSupply': '11388258.0',
'price_currency': 'USD',
'price_diff': -20.71,
'price_diff30d': -14.155971452386,
'price_diff7d': -22.52,
'price_marketCapUsd': '2814942.0',
'price_rate': '0.2471792958',
'price_ts': '1542641433',
'price_volume24h': '2371.62380719',
'symbol': 'GET',
'totalSupply': '33368773400000170376363910',
'transfersCount': 24207}
Now you can just pass that into your csv.writer().
I am a beginner and I've tried searching online everywhere, but I'm not sure I'm searching the right terms.
My CSV file looks this:
https://drive.google.com/file/d/0B74bmJNIxxW-dWl0Y0dsV1E4bjA/view?usp=sharing
I want to know how to use the CSV file to do something like this,
Email
driver.find_element_by_name('emailAddress').send_keys("johndoe#example.com")
print "Successfully Entered Email..."
There are lots of ways that you could do this. One would be to use the csv module.
with open("foo.csv", "r") as fh:
lines = csv.reader(fh)
for line in lines:
address = line[0]
driver.find_element_by_name('emailAddress').send_keys(address)
It really helps to post the data here so that we see what the format really is and run code ourselves. So, I invented some sample data
emails.csv
Email,Password,First Name,Last Name,City
foo1#example.com,frobinate,John,Doe,District Heights
foo2#example.com,frobinate,John,Doe,District Heights
foo3#example.com,frobinate,John,Doe,District Heights
foo4#example.com,frobinate,John,Doe,District Heights
I can use the csv module to read that. csv.DictReader reads each row into its own dict that lets me reference cells by the name given in the header. Since I'll be looking up records by email name later, I'll read it into another dict that will act as an index into the records. If the same user is in there multiple times, only the last one will be remembered.
With the index in place, I can grab the row by email name.
>>> import csv
>>> with open('emails.csv', newline='') as fp:
... reader = csv.DictReader(fp) # auto-reads header
... for row in reader:
... email_index[row['Email']] = row
...
>>> for item in email_index.items():
... print(item)
...
('foo3#example.com', {'Email': 'foo3#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo2#example.com', {'Email': 'foo2#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo4#example.com', {'Email': 'foo4#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
('foo1#example.com', {'Email': 'foo1#example.com', 'City': 'District Heights', 'First Name': 'John', 'Password': 'frobinate', 'Last Name': 'Doe'})
>>>
>>> user = 'foo1#example.com'
>>> record = email_index[user]
>>> print("{Email} is {First Name} {Last Name} and lives in {City}".format(**record))
foo4#example.com is John Doe and lives in District Heights
>>>