I'm testing a Python 2 script that takes a CSV file's text as a command line argument.
The CSV is a standard Excel CSV file with ',' delimiters for items and presumably '\r\n' for line endings.
The problem is that I need to pass the CSV text as a single line to the script in order for it to recognise the string as a single argument. In order to do this, I open the CSV in notepad and replace all of the new lines with '\r\n' which enables me to read the test into my script successfully. However, when I try to create a csv.reader object from the string, the csv.reader only sees a single line where I want iot to see multiple lines.
Given the following CSV string example:
The,quick,brown,fox\r\njumps,over,the,lazy,dog
I would expect the following lines:
The,quick,brown,fox
jumps,over,the,lazy,dog
but instead I just end up with a single line:
The,quick,brown,fox\r\njumps,over,the,lazy,dog
Once I capture the string from the command line, I use the following to load it into a csv.reader:
input_str = self.GetCsvStringFromInput()
input_reader = csv.reader(StringIO.StringIO(input_str))
I'm using windows so I presumed that \r\n would be correct but I don't seem to be using the correct method.
Any ideas?
Thanks, in adv.!
Why don't you just read the lines directly from csv file? Such as:
with open('D:/file-1.csv','r') as csvfile:
creader=csv.reader(csvfile, delimiter=',')
for line in creader:
print line
just replace 'print line' with what your own function. If you do need manually copy each line from the csv file,you can try to split each line by '\r\n' before pass to the reader.
for line in 'The,quick,brown,fox\r\njumps,over,the,lazy,dog'.split('\r\n'):
print line
Related
I have a folder of CSV files (~100) and every file has an unknows character that looks like this �. This unknown character is supposed to be a double quote ("). Because of this unknown char, I am not able to run my CSV to xlsx converter to convert my files to XLSX format.
I tried using the csv.read() function but it does not with the replace function as csv.read() return a reader object and replace does not work with this. How can I replace that character and write the replaced contents back to csv so that I can run my csv to xlsx converter?
example :
current file contetnts:
"hello�
Output after convertion:
"hello"
Try this:
import fileinput
with fileinput.FileInput("file.csv", inplace=True) as file:
for line in file:
print(line.replace('�', '"'), end='')
The sed command is designed for this kind of work. It finds and replaces characters from a file.
Use this in a terminal.
sed -i 's/old-word/new-word/g' filename.csv
Your old-word should be the unknown character and new-word the double quote
I use this little function to deal with such problems.
The code is quite self-explanatory. It opens a file, read it all (may not work for files larger than your RAM) then rewrites it with a patched version.
def patch_file(file, original, patch):
with open(file, 'r') as f:
lines = f.readlines()
with open(file, 'w') as f:
for line in lines:
f.write(line.replace(original, patch))
patch_file(file='yourCSVfile.txt', original='�', patch'"')
Edited because it seems as though I was too vague or didn't show enough research. My apologies (newbie here).
I am trying to read a csv file and assign each new line as a value to iterate through a script that writes to an API. There's no header data in my csv. I'll be adding a regex search and then using the data that follows the regex expression and assign it as a variable to iterate through my script if that makes sense.
CSV Contents:
Type1, test.com
Type2, name.exe
Type3, sample.com
Basic premise of what I want to do in Python:
Read from CSV
Script runs with each line from the CSV as a variable (say Variable1).
The script iterates until it is out of values in the csv list, then terminates.
An example for the script syntax could be anything simple...
#!/usr/bin/python
import requests
import csv
reader = csv.reader(open('test.csv'))
for row in reader:
echo line-item
until the script runs out of Variables to print, then terminates. Where I'm struggling is the syntax on how to take a line then assign it to a variable for the for loop.
I hope that makes sense!
You should take a look at the csv module.
Here's how you would use it:
import csv
file = csv.reader(open('file.csv'), delimiter=',')
for line in file:
print(line)
This produces the following output:
['Type1', ' test.com']
['Type2', ' name.exe']
['Type3', ' sample.com']
It separates your lines into lists of strings at the occurrences of the delimiter you specify (a comma in this case).
If you want to read the file line by line (not as a CSV), you can just use:
with open('file.csv') as file:
for line in file:
print(line)
Using the with statement makes sure that the file is closed after we are done reading its contents.
wrote a python script in windows 8.1 using Sublime Text editor and I just tried to run it from terminal in OSX Yosemite but I get an error.
My error occurs when parsing the first line of a .CSV file. This is the slice of the code
lines is an array where each element is the line in the file it is read from as a string
we split the string by the desired delimiter
we skip the first line because that is the header information (else condition)
For the last index in the for loop i = numlines -1 = the number of lines in the file - 2
We only add one to the value of i because the last line is blank in the file
for i in range(numlines):
if i == numlines-1:
dataF = lines[i+1].split(',')
else:
dataF = lines[i+1].split(',')
dataF1 = list(dataF[3])
del(dataF1[len(dataF1)-1])
del(dataF1[len(dataF1)-1])
del(dataF1[0])
f[i] = ''.join(dataF1)
return f
All the lines in the csv file looks like this (with the exception of the header line):
"08/06/2015","19:00:00","1","410"
So it saves the single line into an array where each element corresponds to one of the 4 values separated by commas in a line of the CSV file. Then we take the 3 element in the array, "410" ,and create a list that should look like
['"','4','1','0','"','\n']
(and it does when run from windows)
but it instead looks like
['"','4','1','0','"','\r','\n']
and so when I concatenate this string based off the above code I get 410 instead of 410.
My question is: Where did the '\r' term come from? It is non-existent in the original files when ran by a windows machine. At first I thought it was the text format so I saved the CSV file to a UTF-8, that didn’t work. I tried changing the tab size from 4 to 8 spaces, that didn’t work. Running out of ideas now. Any help would be greatly appreciated.
Thanks
The "\r" is the line separator. The "\r\n" is also a line separator. Different platforms have different line separators.
A simple fix: if you read a line from a file yourself, then line.rstrip() will remove the whitespace from the line end.
A proper fix: use Python's standard CSV reader. It will skip the blank lines and comments, will properly handle quoted strings, etc.
Also, when working with long lists, it helps to stop thinking about them as index-addressed 'arrays' and use the 'stream' or 'sequential reading' metaphor.
So the typical way of handling a CSV file is something like:
import csv
with open('myfile.csv') as f:
reader = csv.reader(f)
# We assume that the file has 3 columns; adjust to taste
for (first_field, second_field, third_field) in reader:
# do something with field values of the current lines here
I have an assignment for class that has me transfer txt data from excel and execute in python. But every time I run it, only hex is displayed. I was wondering how to have the data displayed in ascii in the shell. This is the code I have so far. Is it possible to print it out in ascii in the shell?
infile = open("data.txt", 'r')
listName = [line.rstrip() for line in infile]
print (listName)
infile.close()
The reason its not working is because you are opening an Excel file - which is in a special format and is not a plain text file.
You can test this by yourself by opening the file in a text editor like Notepad; and you'll see the contents aren't in text.
To open the file and read its contents in Python you will need to do one of these two things:
Open the file in Excel, then save it as a text file (or a comma separated file CSV). Keep in mind if you do this, then you can only save one sheet at a time.
Use a module like pyexcel which will allow you to read the Excel file correctly in Python.
Just opening the file as plain text (or changing its extension) doesn't convert it.
Im fairly new to python, I want to read a string from a file input that text using adb shell input text then go to the next line read the next string and input the next string by using the same command and this goes on for 200 lines
I think this will solve your problem, note the use of the readlines method:
your_file = open('path to your file', 'r') #this will open your file
lines = your_file.readlines() #this is an array of lines
for line in lines: #loop over the lines
<block of operations>