I've a script that gets data in a json format through an API. I need to write headers to the file, then append with some of the json data.
import requests, csv, json
def test(limit="", api_token=""):
url_rack = "url"
api_token = "Token " + api_token
headers = {'Authorization': api_token}
params = {'limit': limit}
session = requests.Session()
json_data = session.get(url_rack, headers=headers, params=params)
with open('my_file.csv', 'wb') as csvfile:
filewriter = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
filewriter.writerow(['Header1', 'Header2'])
json_data = json_data.json()['results']
for i in json_data:
bunit = i['tenant']
open(csvfile)
if not bunit:
print 'Placeholder'
else:
bunit = i['tenant']['name']
print bunit
for i in json_data:
owner = i['tenant']
if not owner:
print 'Placeholder Owner'
else:
owner = i['tenant']['name']
print owner
def main():
api_token = "mytoken"
test(api_token=api_token)
if __name__ == "__main__":
main()
I tried to write the data taken from for i in tenants: in the column under Header1, and the data from for i in owners under Header2. I have two trivial problems. When I tried to open the csv file inside the for loop I got the error TypeError: coercing to Unicode: need string or buffer, file found. I got that the file is open so I can't reopen it, but don't want to create it in the loop as that would rewrite the file each iteration?
Secondly filewriter.writerow obviously writes in rows, but I need to write the data in each column under the header. I printed it out to test that I can actually call the api and get the data but can't work out how to append it under Header1 and Header2.
EDIT
My data that prints:
Placeholder tenant
Placeholder tenant
Tenant1
Placeholder tenant
Placeholder tenant
Placeholder tenant
Placeholder tenant
Tenant2
Placeholder tenant
Tenant3
Tenant4
and
Placeholder Owner
Placeholder Owner
Owner1
Placeholder Owner
Placeholder Owner
Placeholder Owner
Placeholder Owner
Owner2
Placeholder Owner
Owner3
Owner4
Why not to rewrite your code like this:
...
with open('my_file.csv', 'wb') as csvfile:
filewriter = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
filewriter.writerow(['Header1', 'Header2'])
json_data = json_data.json()['results']
for i in json_data:
bunit = i['tenant']
if not bunit:
print 'Placeholder'
else:
bunit = i['tenant']['name']
owner = i['tenant']
if not owner:
print 'Placeholder Owner'
else:
owner = i['tenant']['name']
filewriter.writerow([bunit, owner])
...
To open() function, you need to provide a filename as parameter.
Here you are providing an already open file as a parameter. Hence, first error.
The csvfile is already open and you do not need to open it again to write to it.
To write to the file, you can store the data in a list and then write to file at end of function.
Suppose I want to write r1c1,r1c2,r2c1,r2c2,r3c1,r3c2 in cells-A1,B1,A2,B2,A3,B3 respectively, then I would do something like the following:-
listtowrite = [['r1c1','r1c2'],['r2c1','r2c2'],['r3c1','r3c2']]
csvfile = open('my_file.csv', 'w')
for arow in listtowrite:
for acolumn in arow:
csvfile.write(acolumn+',')
csvfile.write('\n')
csvfile.close()
Figured it out. Just needed to write each to a list
json_data = prefixes_response.json()['results']
tenant = []
for i in json_data:
bunit = i['tenant']
if not bunit:
tenant.append('Placeholder tenant')
else:
bunit = i['tenant']['name']
tenant.append(bunit)
and zip both lists and write
rows = zip(Header1, Header2)
for row in rows:
filewriter.writerow(row)
Related
I'm new to Python so excuse me if my question is kind of dumb.
I send some data into a csv file (I'm making a password manager). So I send this to this file (in this order), the name of the site, the e-mail corresponding and finally the password.
But I would like to print all the names already written in the csv file but here is my problem, for the first row it does print the whole row but for the following rows it works just well.
Here is my code, I hope u can help me with this.
csv_file = csv.reader(open('mycsvfile.csv', 'r'), delimiter=';')
try :
print("Here are all the sites you saved :")
for row in csv_file :
print(row[0])
except :
print("Nothing already saved")
Maybe it can help, but here is how I wrote my data into the csv file:
#I encrypt the email and the password thanks to fernet and an already written key
#I also make sure that the email is valid
file = open('key.key', 'rb')
key = file.read()
file.close()
f = Fernet(key)
website = input("web site name : \n")
restart = True
while restart :
mail = input("Mail:\n")
a = isvalidEmail(mail)
if a == True :
print("e-mail validated")
restart = False
else :
print("Wrong e-mail")
pws = input("password :\n")
psw_bytes = psw.encode()
mail_bytes = mail.encode()
psw_encrypted_in_bytes = f.encrypt(psw_bytes)
mail_encrypted_in_bytes = f.encrypt(mail_bytes)
mail_encrypted_str = mail_encrypted_in_bytes.decode()
psw_encrypted_str = psw_encrypted_in_bytes.decode()
f = open('a.csv', 'a', newline='')
tup1 = (website, mail_encrypted_str, psw_encrypted_str)
writer = csv.writer(f, delimiter = ';')
writer.writerow(tup1)
print("Saved ;)")
f.close()
return
And here is my output (I have already saved data)
Output (First, you see the name of the ws with the email and the psw encrypted then just the name which is what I want
I finally succeed, instead of using a csv.Reader, i used a csv.DictReader and as all the names i'm looking for are on the same column, i juste have to use the title of the columns.
So here is the code :
with open('mycsv.csv', newline='') as csvfile:
data = csv.DictReader(csvfile)
print("Websites")
print("---------------------------------")
for row in data:
print(row['The_title_of_my_column'])
make list from csv.reader()
rows = [row for row in csv_file]
and now you can get element by identifier using rows as list of lists
rows[id1][id2]
I need to post some records to the website. I feel I am done with the complex part - the code itself, now I need to tweak the code so that my account doesn't get blocked when doing the posting - yep, just happened.
#importing libraries
import csv
import json
#changing data type
field_types = [('subject', str),
('description', str),
('email', str)]
output = []
#opening the raw file
with open('file.csv','r',encoding = 'utf-8-sig') as f:
for row in csv.DictReader(f):
row.update((key, conversion(row[key]))
for key, conversion in field_types)
output.append(row) #appending rows
with open('tickets.json','w') as outfile: #saving records as json
json.dump(output,outfile,sort_keys = True, indent = 4)
with open('tickets.json','r')as infile:
indata = json.load(infile)
output =[]
for data in indata:
r= requests.post("https://"+ domain +".domain.com/api/", auth = (api_key, password), headers = headers, json=data)
output.append(json.loads(r.text))
#saving the response code
with open('response.json', 'w') as outfile:
json.dump(output, outfile, indent = 4)
I searched and found time.sleep(5) but now sure how to use it. Will it go before output.append(json.loads(r.text))?
am trying to write a loop that gets .json from an url via requests, then writes the .json to a .csv file. Then I need it to it over and over again until my list of names (.txt file) is finished(89 lines). I can't get it to go over the list, it just get the error:
AttributeError: module 'response' has no attribute 'append'
I can´t find the issue, if I change 'response' to 'responses' I get also an error
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
OSError: [Errno 22] Invalid argument: "listan-['A..
I can't seem to find a loop fitting for my purpose. Since I am a total beginner of python I hope I can get some help here and learn more.
My code so far.
#Opens the file with pricelists
pricelists = []
with open('prislistor.txt', 'r') as f:
for i, line in enumerate(f):
pricelists.append(line.strip())
# build responses
responses = []
for pricelist in pricelists:
response.append(requests.get('https://api.example.com/3/prices/sublist/{}/'.format(pricelist), headers=headers))
#Format each response
fullData = []
for response in responses:
parsed = json.loads(response.text)
listan=(json.dumps(parsed, indent=4, sort_keys=True))
#Converts and creates a .csv file.
fullData.append(parsed['Prices'])
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw.writeheader()
for data in fullData:
dw = csv.DictWriter(outf, data[0].keys())
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
Can you make the below changes in the place where you are making the api call(import json library as well) and see?
import json
responses = []
for pricelist in pricelists:
response = requests.get('https://api.example.com/3/prices/sublist/{}/'.format(pricelist), headers=headers)
response_json = json.loads(response.text)
responses.append(response_json)
and the below code also should be in a loop which loops through items in pricelists
for pricelist in pricelists:
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw.writeheader()
for data in fullData:
dw = csv.DictWriter(outf, data[0].keys())
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
Finally got it working. Got a help from another questions I created here at the forum. #waynelpu
The misstake I did was to not put the code into a loop.
Here is the code that worked like a charm.
pricelists = []
with open('prislistor.txt', 'r') as f:
for i, line in enumerate(f): # from here on, a looping code block start with 8 spaces
pricelists = (line.strip())
# Keeps the indents
response = requests.get('https://api.example.se/3/prices/sublist/{}/'.format(pricelists), headers=headers)
#Formats it
parsed = json.loads(response.text)
listan=(json.dumps(parsed, indent=4, sort_keys=True))
#Converts and creates a .csv file.
data = parsed['Prices']
with open('listan-{}.csv'.format(pricelists), 'w') as outf:
dw = csv.DictWriter(outf, data[0].keys())
dw.writeheader()
for row in data:
dw.writerow(row)
print ("The file list-{}.csv is created!".format(pricelists))
# codes here is outside the loop but still INSIDE the 'with' block, so you can still access f here
# codes here leaves all blocks
I have an app that works to write a csv as a response to a specific URL call. But, I don't know how to escape characters.
Right now, my code looks like this
import csv
class ReturnCSV(BaseHandler):
def get(self, group_id):
self.response.headers['Content-Type'] = 'text/csv'
self.response.headers['Content-Disposition'] = "attachment; filename=scenarios.csv"
self.write(','.join(header)) #header is a list defined elsewhere
for scenario in list_of_scenarios:
#do stuff
self.write('\r\n' + ','.join(output)))
This gets me the expected output, except that it doesn't escape special characters. I figured I need to implement csv.writer with QUOTE_ALL, but when trying that, I get an IOError(errno.EROFS, 'Read-only file system', filename)
self.response.headers['Content-Type'] = 'text/csv'
self.response.headers['Content-Disposition'] = "attachment; filename=scenarios.csv"
with open('scenarios.csv', 'w') as output:
row = ','.join(header)
wr = csv.writer(output, quoting=csv.QUOTE_ALL)
wr.writerow(row)
for scenario in list_of_scenarios:
#do stuff
wr.writerow(output)
So, got it to work if I don't try to open the file and then I take out all the ','.join that I was doing to manually create CSV.
output = StringIO.StringIO()
row = header #header created elsewhere
wr = csv.writer(output, quoting=csv.QUOTE_MINIMAL)
wr.writerow(row)
for scenario in list_of_scenarios:
# do stuff
wr.writerow(datarow)
self.response.headers['Content-Type'] = 'text/csv'
self.response.headers['Content-Disposition'] = "attachment; filename=scenarios.csv"
self.write(output.getvalue())
Sorry if this has been asked, but is it possible to skip a column when writing to a csv file?
Here is the code I have:
with open("list.csv","r") as f:
reader2 = csv.reader(f)
for row in reader2:
url = 'http://peopleus.intelius.com/results.php?ReportType=33&qi=0&qk=10&qp='+row
req = urllib.request.Request(url)
response = urllib.request.urlopen(req)
html = response.read()
retrieved_name = b'class="singleName">(.*?)<\/h1'
retrieved_number = b'<div\sclass="phone">(.*?)<\/div'
retrieved_nothing = b"(Sorry\swe\scouldn\\'t\sfind\sany\sresults)"
if re.search(retrieved_nothing,html):
noth = re.search(retrieved_nothing.decode('utf-8'),html.decode('utf-8')).group(1)
add_list(phone_data, noth)
else:
if re.search(retrieved_name,html):
name_found = re.search(retrieved_name.decode('utf-8'),html.decode('utf-8')).group(1)
else:
name_found = "No name found on peopleus.intelius.com"
if re.search(retrieved_number,html):
number_found = re.search(retrieved_number.decode('utf-8'),html.decode('utf-8')).group(1)
else:
number_found = "No number found on peopleus.intelius.com"
add_list(phone_data, name_found, number_found)
with open('column_skip.csv','a+', newline='') as mess:
writ = csv.writer(mess, dialect='excel')
writ.writerow(phone_data[-1])
time.sleep(10)
Assuming that there is data in the first three rows of column_skip.csv, can I have my program start writing its info in column 4?
Yeah, don't use csv.writer method and write it as an simple file write operation:
`file_path ='your_csv_file.csv'
with open(file_path, 'w') as fp:
#following are the data you want to write to csv
fp.write("%s, %s, %s" % ('Name of col1', 'col2', 'col4'))
fp.write("\n")`
I hope this helps...