I created a python script which works with a test CSV dataset of 10 records. When I scaled this up to the actual datasets (a few thousand rows), I am getting the following error:
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
The code is as follows:
with open('./Origins.csv', 'r') as csvfile:
reader = csv.DictReader(csvfile)
origincoords = ['{Y},{X}'.format(**row) for row in reader]
The full error code is:
Traceback (most recent call last):
File "./Driving.py", line 14, in <module>
origincoords = ['{Y},{X}'.format(**row) for row in reader]
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 103, in next
self.fieldnames
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 90, in fieldnames
self._fieldnames = self.reader.next()
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
Perhaps there is a scale problem with the CSV reading method I am using?
From PEP-0278:
In a Python with universal newline support open() the mode parameter
can also be "U", meaning "open for input as a text file with universal
newline interpretation". Mode "rU" is also allowed, for symmetry with
"rb"
So try to change
with open('./Destinations.csv', 'r') as csvfile:
to
with open('./Destinations.csv', 'rb') as csvfile:
If the error persists, change to
with open('./Destinations.csv', 'rU') as csvfile:
Edited accorded to Martijn Pieters's comment.
Related
Working on a project, but I wanted read and write mode when I opened the csv file. Sadly, it is giving me an error. Here is the code:
import csv
with open("historicalData/ACAD.csv", "rw") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',')
for lines in csv_reader:
print(lines[0])
And the error I am getting:
Traceback (most recent call last):
File "c:/Users/sande/Desktop/Vihaan/ThirdPartySoftware/Python/VisualStudiosCode/DadWork/test.py", line 3, in <module>
with open("historicalData/ACAD.csv", "ra") as csv_file:
ValueError: must have exactly one of create/read/write/append mode
Can anyone help in how to achieve this?
Use r+ or a+ or w+ not rw as python doesn't recognise it
I have some code which i coded in python 2.7, however I need it to work for 3.6 and when i run it i get this error and i am not sure why.
import csv
def ReadFromFile():
with open('File.csv', 'r') as File:
cr = csv.reader(File)
for row in cr:
Name = row[0]
Gender = row[1]
print(Name + Gender)
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
ReadFromFile()
File "F:/Test.py", line 6, in ReadFromFile
Name = row[0]
IndexError: list index out of range
I am using the same code saved on a memory stick with the file in 2.7 i get my desired out come of it being read but in 3.6 i am stuck with the error. Thanks for any help
Edit: Added Print
After adding print i got
ELIZABETHFemale
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
ReadFromFile()
File "F:/Test.py", line 6, in ReadFromFile
Name = row[0]
IndexError: list index out of range
So it gave me the first line but nothing more
Python's CSV module has changed how it wants the files you pass to it to be opened. You want to avoid the file object doing any newline transformation because some CSV formats allow embedded newlines within quoted fields. The csv module will do its own newline normalization, so the usual universal newline handling the file object does is redundant.
This is mentioned in the csv.reader documentation, where it is talking about the file argument:
If csvfile is a file object, it should be opened with newline=''.
So for your code, try changing open('File.csv', 'r') to open('File.csv', 'r', newline='').
Have you tried pandas?
I think you may want to use something like
import pandas as pd
def ReadFromFile():
df = pd.read_csv('File.csv')
for row in df:
Name = row[0]
Gender = row[1]
print(Name + Gender)
import csv
with open('database.csv') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
print(row['NAME'])
Guys, I have a csv file with the first row as a index, and in linux the code read from row['NAME'] and print only the names form colum NAME, when I run it in windows, it says:
C:\Users\Desktop>python py.py
Traceback (most recent call last):
File "py.py", line 5, in <module>
print(row['NAME'])
KeyError: 'NAME'
WHY?
If you are using the python 2- versions, you need to open the csv with rb, i.e.:
with open('database.csv', 'rb') as csvfile:....
for reference, checkout https://docs.python.org/2/library/csv.html as it includes a part in the reader doc about this.
I'm having an issue using Python's csv module. I created a simple file in Excel containing two columns (names and ages of people) and saved it as a csv file. I then ran the following lines in Python:
import csv
csv_rader = csv.DictReader(open('people.csv'))
people = list(csv_reader)
and I got the following error:
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 107, in next
self.fieldnames
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 90, in fieldnames
self._fieldnames = self.reader.next()
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
Does anyone know what could be causing this, and how to go about fixing it?
I have a strange problem. A friend of mine sent me a text file. If I copy/paste the text and then paste it into my text editor and save it, the following code works. If I choose the option to save the file directly from the browser, the following code breaks. What's going on? Is it the browser's fault for saving invalid characters?
This is an example line.
When I save it, the line says
What�s going on?
When I copy/paste it, the line says
What’s going on?
This is the code:
import codecs
def do_stuff(filename):
with codecs.open(filename, encoding='utf-8') as f:
def process_line(line):
return line.strip()
lines = f.readlines()
for line in lines:
line = process_line(line)
print line
do_stuff('stuff.txt')
This is the traceback I get:
Traceback (most recent call last):
File "test-encoding.py", line 13, in <module>
do_stuff('stuff.txt')
File "test-encoding.py", line 8, in do_stuff
lines = f.readlines()
File "/home/somebody/.python/lib64/python2.7/codecs.py", line 679, in readlines
return self.reader.readlines(sizehint)
File "/home/somebody/.python/lib64/python2.7/codecs.py", line 588, in readlines
data = self.read()
File "/home/somebody/.python/lib64/python2.7/codecs.py", line 477, in read
newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 4: invalid start byte
What can I do in such cases?
How can I distribute the script if I don't know what encoding the user who runs it will use?
Fixed:
codecs.open(filename, encoding='utf-8', errors='ignore') as f:
The "file-oriented" part of the browser works with raw bytes, not characters. The specific encoding used by the page should be specified either in the HTTP headers or in the HTML itself. You must use this encoding instead of assuming that you have UTF-8 data.