I have a simple python program reading a 'coma' delimited csv file with nearly 2000 rows. The program simply reads each line and displays it. I have manually verified each line through vi editor. When reading the csv file the program in the middle of the file somehow ignores the '\n' and reads multiple lines simultaneously into a single row. Why is that happening? I am working on unix platform and have converted the file to unix format as well.
import csv
with open("fileName.csv") as f:
reader=csv.reader(f)
for row in reader:
print row
Any clues?
thanks
Related
So I'm currently trying to use Python to create a neat and tidy .csv file from a .txt file. The first stage is to get some 8-digit numbers into one column called 'Number'. I've created the header and just need to put each number from each line into the column. What I want to know is, how do I tell Python to read the first eight characters of each line in the .txt file (which correspond to the number I'm looking for) and then write them to the .csv file? This is probably very simple but I'm only new to Python!
So far, I have something which looks like this:
with open(r'C:/Users/test1.txt') as rf:
with open(r'C:/Users/test2.csv','w',newline='') as wf:
outputDictWriter = csv.DictWriter(wf,['Number'])
outputDictWriter.writeheader()
writeLine = rf.read(8)
for line in rf:
wf.write(writeLine)
You can use pandas:
import pandas as pd
df = pd.read_csv(r'C:/Users/test2.txt')
df.to_csv(r'C:/Users/test2.csv')
Here is how to read the first 8 characters of each line in a file and store them in a list:
with open('file.txt','r') as f:
lines = [line[:8] for line in f.readlines()]
You can use regex to select digits with the charecter. Search for it
pattern = re.searh(w*\d{8})
Just go one step back and read again what you need:
read the first eight characters of each line in the .txt file (which correspond to the number I'm looking for) and then write them to the .csv file
Now forget Python and explain what is to be done in pseudo code:
open txt file for reading
open csv file for writing (beware end of line is expected to be \r\n for a CSV file)
write the header to the csv file
loop reading the txt file 1 line at a time until end of file
extract 8 first characters from the line
write them to the csv file, ended with a \r\n
close both files
Ok, time to convert above pseudo code to Python language:
with open('C:/Users/test1.txt') as rf, open('C:/Users/test2.csv', 'w', newline='\r\n') as wf:
print('Number', file=wf)
for line in rf:
print(line.rstrip()[:8], file=wf)
Edited because it seems as though I was too vague or didn't show enough research. My apologies (newbie here).
I am trying to read a csv file and assign each new line as a value to iterate through a script that writes to an API. There's no header data in my csv. I'll be adding a regex search and then using the data that follows the regex expression and assign it as a variable to iterate through my script if that makes sense.
CSV Contents:
Type1, test.com
Type2, name.exe
Type3, sample.com
Basic premise of what I want to do in Python:
Read from CSV
Script runs with each line from the CSV as a variable (say Variable1).
The script iterates until it is out of values in the csv list, then terminates.
An example for the script syntax could be anything simple...
#!/usr/bin/python
import requests
import csv
reader = csv.reader(open('test.csv'))
for row in reader:
echo line-item
until the script runs out of Variables to print, then terminates. Where I'm struggling is the syntax on how to take a line then assign it to a variable for the for loop.
I hope that makes sense!
You should take a look at the csv module.
Here's how you would use it:
import csv
file = csv.reader(open('file.csv'), delimiter=',')
for line in file:
print(line)
This produces the following output:
['Type1', ' test.com']
['Type2', ' name.exe']
['Type3', ' sample.com']
It separates your lines into lists of strings at the occurrences of the delimiter you specify (a comma in this case).
If you want to read the file line by line (not as a CSV), you can just use:
with open('file.csv') as file:
for line in file:
print(line)
Using the with statement makes sure that the file is closed after we are done reading its contents.
I've noticed a really weird bug and didn't know if anyone else had seen this / knows how to stop it.
I'm writing to a CSV file using this:
def write_to_csv_file(self, object, string):
with open('data_model_1.csv', 'a') as f:
writer = csv.writer(f)
writer.writerow([object, string])
and then write to the file:
self.write_to_csv_file(self.result['outputLabel'], string)
If I open the CSV file to look at the results, the next time I write to the file, it will start in column 3 of the last line (column 1 is object, column 2 is string).
If I run self.write_to_csv_file(self.result['outputLabel'], string) multiple times without manually opening the file (obviously I open the file in the Python script), everything is fine.
It's only when I open the file so I get the issue of starting on Column 3.
Any thoughts on how to fix this?
You're opening the file in append mode, so the data is appended to the end of the file. If the file doesn't end in a newline, rows may get concatenated. Try writing a newline to the file before appending new rows:
with open("data_model_1.csv", "a") as f:
f.write("\n")
I'm trying to shave the top 7 lines off a csv file.
There is probably a more concise way to do this, but right now I am reading one file and writing each line other than the first 7 to another file. When I write to the file though, all the contents for the line show up in the first cell instead of spread out in organized columns.
Here is my code:
with open('file1.csv', 'r') as file_org:
with open("file2.csv","w") as file_stripped:
writer = csv.writer(file_stripped)
for i, line in enumerate(file_org, -7):
if i>=0:
writer.writerow([line])
Thank you!
reading in csv do need you to specify the seperator, which is usually ";", you can find the constructor usage in manual, and you should open the file and see the content, not by some other tool like excel
if you are not meant to change the content, you could just treat them as normal files / line, or just by
line.split(";")
or
";".join(splited_line)
manully
I have a csv file which contains rows from a sqlite3 database. I wrote the rows to the csv file using python.
When I open the csv file with Ms Excel, a blank row appears below every row, but the file on notepad is fine(without any blanks).
Does anyone know why this is happenning and how I can fix it?
Edit: I used the strip() function for all the attributes before writing a row.
Thanks.
You're using open('file.csv', 'w')--try open('file.csv', 'wb').
The Python csv module requires output files be opened in binary mode.
the first that comes into my mind (just an idea) is that you might have used "\r\n" as row delimiter (which is shown as one linebrak in notepad) but excel expects to get only "\n" or only "\r" and so it interprets this as two line-breaks.