I am new in python. I would like to ask what is the optimal way of getting length on the first column of csv file in python ? I tried this, but got this error:
File "C:/Projekty/valispace_api/CreatingCvs.py", line 6, in <module>
first_col_len = len(next(zip(*reader)))
_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)
Thanks
import csv
with open('export_U43V_WIRKNETZ.csv', 'rb') as f:
reader = csv.reader(f)
first_col_len = len(next(zip(*reader)))
print(first_col_len)
I am trying to convert a json data set file into csv. I am really new to python, and have been looking on the forums and cannot seem to resolve my issues. I have attached the json data url link in below along with my code. Thanks in advance!
https://data.ny.gov/api/views/nqur-w4p7/rows.json?accessType=DOWNLOAD
import json
import csv
inputFile = ("rows.json?accessType=DOWNLOAD", "r")
data = json.load(inputFile)
with open("Data.csv","wb") as csvfile:
csv_writer = csv.DictWriter(csvfile,delimiter=",", fieldnames=["data", "new_york_state_average_gal", "albany_average_gal", "binghamton_average_gal", "bu\
ffalo_average_gal", "nassau_average_gal", "new_york_city_average_gal", "rochester_average_gal", "utica_average_gal"])
csv_writer.writerheader()
csv_writer.writerows(data)
Here is the error I am getting:
File "ChangeDataType.py", line 5, in <module>
data = json.load(inputFile)
File "/usr/lib64/python3.4/json/__init__.py", line 265, in load
return loads(fp.read(),
AttributeError: 'tuple' object has no attribute 'read'
Your error happens because you made a tuple:
inputFile = ("rows.json?accessType=DOWNLOAD", "r")
And you're trying to use json.load in that tuple. Since json.load works only on files, you need to call the open function:
inputFile = open("rows.json?accessType=DOWNLOAD", "r")
The "r" part indicates you're opening the file for reading.
So I'm trying to use the csv module in python 3.3.2 but I am getting this error.
Traceback (most recent call last):
File "C:\Users\massi_000\Desktop\csv.py", line 1, in <module>
import csv
File "C:\Users\massi_000\Desktop\csv.py", line 4, in <module>
csv.reader(f)
AttributeError: 'module' object has no attribute 'reader'
Obviously I'm going something stupendously wrong but all the code I am using is below and it looks fine. Has something changed in this version that has rendered this code unusable or..?
import csv
f = open("test.csv")
csv.reader(f)
for row in csv_fi:
print(row)
f.close()
You have named your file csv.py and this clashes with the csv module from the Python standard library.
You should rename your own file to something else so that import csv will import the standard library module and not your own. This can be confusing but this is a good rule-of-thumb going forward: avoid giving your own Python files names that are the same as modules in the standard library.
As #Simeon Visser said, you have to rename your file but you have some other issues with your code as well. Try this:
import csv
with open('test.csv', newline='') as f:
reader = csv.reader(f, delimiter=' ')
for row in reader:
print (', '.join(row))
I'm a beginner in programming and have decided to teach myself Python. After a few days, i've decided to code a little piece. I's pretty simple:
date of today
page i am at (i'm reading a book)
how i feel
then i add the data in a file. every time i launch the program, it adds a new line of data in the file
then i extract the data to make a list of lists.
truth is, i wanted to re-write my program in order to pickle a list and then unpickle the file. However, as i'm coping with an error i can't handle, i really really want to understand how to solve this. Therefore i hope you will be able to help me out :)
I've been struggling for the past hours on this apparently a simple and stupid problem. Though i don't find the solution. Here is the error and the code:
ERROR:
Traceback (most recent call last):
File "dailyshot.py", line 25, in <module>
SaveData(todaysline)
File "dailyshot.py", line 11, in SaveData
mon_pickler.dump(datatosave)
TypeError: must be str, not bytes
CODE:
import pickle
import datetime
def SaveData(datatosave):
with open('journey.txt', 'wb') as thefile:
my_pickler = pickle.Pickler(thefile)
my_pickler.dump(datatosave)
thefile.close()
todaylist = []
today = datetime.date.today()
todaylist.append(today)
page = input('Page Number?\n')
feel = input('How do you feel?\n')
todaysline = today.strftime('%d, %b %Y') + "; " + page + "; " + feel + "\n"
print('Thanks and Good Bye!')
SaveData(todaysline)
print('let\'s make a list now...')
thefile = open('journey.txt','rb')
thelist = [line.split(';') for line in thefile.readlines()]
thefile.close()
print(thelist)
Thanks a looot!
Ok so there are a few things to comment on here:
When you use a with statement, you don't have to explicitly close the file. Python will do that for you at the end of the with block (line 8).
You don't use todayList for anything. You create it, add an element and then just discard it. So it's probably useless :)
Why are you pickling string object? If you have strings just write them to the file as is.
If you pickle data on write you have to unpickle it on read. You shouldn't write pickled data and then just read the file as a plain text file.
Use a for append when you are just adding items to the file, w will overwrite your whole file.
What I would suggest is just writing a plain text file, where every line is one entry.
import datetime
def save(data):
with open('journey.txt', 'a') as f:
f.write(data + '\n')
today = datetime.date.today()
page = input('Page Number: ')
feel = input('How do you feel: ')
todaysline = ';'.join([today.strftime('%d, %b %Y'), page, feel])
print('Thanks and Good Bye!')
save(todaysline)
print('let\'s make a list now...')
with open('journey.txt','r') as f:
for line in f:
print(line.strip().split(';'))
Are you sure you posted the right code? That error can occur if you miss out the "b" when you open the file
eg.
with open('journey.txt', 'w') as thefile:
>>> with open('journey.txt', 'w') as thefile:
... pickler = pickle.Pickler(thefile)
... pickler.dump("some string")
...
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
TypeError: must be str, not bytes
The file should be opened in binary mode
>>> with open('journey.txt', 'wb') as thefile:
... pickler = pickle.Pickler(thefile)
... pickler.dump("some string")
...
>>>
I need to read a CSV file in python.
Since for last row I receive a 'NULL byte' error I would like to avoid using for keyword but the while.
Do you know how to do that?
reader = csv.reader( file )
for row in reader # I have an error at this line
# do whatever with row
I want to substitute the for-loop with a while-loop so that I can check if the row is NULL or not.
What is the function for reading a single row in the CSV module?
Thanks
Thanks
p.S. below the traceback
Traceback (most recent call last):
File "FetchNeuro_TodayTrades.py", line 189, in
for row in reader:
_csv.Error: line contains NULL byte
Maybe you could catch the exception raised by the CSV reader. Something like this:
filename = "my.csv"
reader = csv.reader(open(filename))
try:
for row in reader:
print 'Row read with success!', row
except csv.Error, e:
sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e))
Or you could use next():
while True:
try:
print reader.next()
except csv.Error:
print "Error"
except StopIteration:
print "Iteration End"
break
You need (always) to say EXACTLY what is the error message that you got. Please edit your question.
Probably this:
>>> import csv; csv.reader("\x00").next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
_csv.Error: line contains NULL byte
>>>
The csv module is not 8-bit clean; see the docs: """Also, there are currently some issues regarding ASCII NUL characters."""
The error message is itself in error: it should be "NUL", not "NULL" :-(
If the last line in the file is empty, you won't get an exception, you'll merely get row == [].
Assuming the problem is one or more NULs in your file(s), you'll need to (1) speak earnestly to the creator(s) of your file(s) (2) failing that, read the whole file in (mode="rb"), strip out the NUL(s), and feed fixed_text.splitlines() to the csv reader.
The Django community has addressed Python CSV import issues, so it might be worth searching for CSV import there, or posting a question. Also, you could edit the offending line directly in the CSV file before trying the import.
You could try cleaning the file as you read it:
def nonull(stream):
for line in stream:
yield line.replace('\x00', '')
f = open(filename)
reader = csv.reader(nonull(f))
Assuming, of course, that simply ignoring NULL characters will work for you!
If your problem is specific to the last line being empty, you can use numpy.genfromtxt (or the old matplotlib.mlab.csv2rec)
$: cat >csv_file.txt
foo,bar,baz
yes,no,0
x,y,z
$:
$: ipython
>>> from numpy import genfromtxt
>>> genfromtxt("csv_file.txt", dtype=None, delimiter=',')
array([['foo', 'bar', 'baz'],
['yes', 'no', '0'],
['x', 'y', 'z']],
dtype='|S3')
not really sure what you mean, but you can always check for existence with if
>>> reader = csv.reader("file")
>>> for r in reader:
... if r: print r
...
if this is not what you want, you should describe your problem more clearly by showing examples of things that doesn't work for you, including sample file format and desired output you want.
I don't have an answer, but I can confirm the problem, and that most answers posted don't work. You cannot catch this exception. You cannot test for if line. Maybe you could check for the NULL byte directly, but I'm not swift enough to do that... If it is always on the last line, you could of course skip that.
import csv
FH = open('data.csv','wb')
line1 = [97,44,98,44,99,10]
line2 = [100,44,101,44,102,10]
for n in line1 + line2:
FH.write(chr(n))
FH.write(chr(0))
FH.close()
FH = open('data.csv')
reader = csv.reader(FH)
for line in reader:
if '\0' in line: continue
if not line: continue
print line
$ python script.py
['a', 'b', 'c']
['d', 'e', 'f']
Traceback (most recent call last):
File "script.py", line 11, in <module>
for line in reader:
_csv.Error: line contains NULL byte
Process the initial csv file and replace the Nul '\0' with blank, and then you can read it.
The actual code looks like this:
data_initial = open(csv_file, "rU")
reader = csv.reader((line.replace('\0','') for line in data_initial))
It works for me.
And the original answer is here:csv-contain null byte