We Keep Coding

CSV writer adds the first line to the last cell

I am trying to write to a csv file that has been saved with another program (excel and others).
However when I open the file to write to it, the first line written is added to last cell of the last line.
file.csv
['1','2','3']
['1','2','3']
import csv
fields=['A','B','C']
with open('file.csv', 'a', newline='') as f:
writer = csv.writer(f)
writer.writerow(fields)
Expected results:
['1','2','3']
['1','2','3']
['A','B','C']
Actual results:
['1','2','3']
['1','2','3A','B','C']
If I just write to file then write to it again without viewing it, there's no issue, but if I open the file, and save it inside the program the next line written is added to the previous line instead of becoming it's own line.
I assume there is a formatting issue with how the program saves the file, but I am at a loss as to how to fix it.

Lines in CSV files should always be terminated with a "\r\n" sequence, even if its the last line in the file. In the grand tradition of CSV programming, this is often ignored. The fix is to write a program that peeks at the file and fixes it as needed before use. And write a bug against the "other" program that wrote the nonconforming CSV in the first place.
import csv
def csv_delimiter_fixer(filename):
with open(filename, 'a+b') as fileobj:
fileobj.seek(-1, 2)
if fileobj.read(1) != b"\n":
fileobj.write(b"\r\n")
fields=['A','B','C']
filename = 'file.csv'
csv_delimiter_fixer(filename)
with open('file.csv', 'a', newline='') as f:
writer = csv.writer(f)
writer.writerow(fields)

change csv file variable from bytes to string in python

I have a csv file which I get from the frontend so I have it stored in a variable. Now I want to use the csv reader from the csv python library. This is my code:
def parseCSV(file):
csv_reader = csv.reader(file)
for line in csv_reader:
print(line)
But this isn't working because every line of the file is byte format and not string. How can I change the entire file variable from byte to string and make the code work?
I've already seen some solutions for this problem if I read a csv file from a folder and open it with with open ("file.csv", "r"). But I can't read this csv file from a folder so I somehow need to change byte to string.
I've already tried something like this but this is not working:
def parseCSV(file):
csv_reader = csv.reader(file)
for line.decode("utf-8") in csv_reader:
print(line)

So I've found a solution for this problem.
I just had to change the following line:
csv_reader = csv.reader(file)
to this line:
csv_reader = csv.reader(codecs.iterdecode(file, 'utf-8'))
and import codecs as import codecs
So if anyone else should have the same problem this should make the code work.

Reading and Writing into CSV file at the same time

I wanted to read some input from the csv file and then modify the input and replace it with the new value. For this purpose, I first read the value but then I'm stuck at this point as I want to modify all the values present in the file.
So is it possible to open the file in r mode in one for loop and then immediately in w mode in another loop to enter the modified data?
If there is a simpler way to do this please help me out
Thank you.

Yes, you can open the same file in different modes in the same program. Just be sure not to do it at the same time. For example, this is perfectly valid:
with open("data.csv") as f:
# read data into a data structure (list, dictionary, etc.)
# process lines here if you can do it line by line
# process data here as needed (replacing your values etc.)
# now open the same filename again for writing
# the main thing is that the file has been previously closed
# (after the previous `with` block finishes, python will auto close the file)
with open("data.csv", "w") as f:
# write to f here
As others have pointed out in the comments, reading and writing on the same file handle at the same time is generally a bad idea and won't work as you expect (unless for some very specific use case).

You can do open("data.csv", "rw"), this allows you to read and write at the same time.

Just like others have mentioned, modifying the same file as both input and output without any backup method is such a terrible idea, especially in a condensed file like most .csv files, which is normally more complicated than a single .Txt based file, but if you insisted you can do with the following:
import csv
file path = 'some.csv'
with open('some.csv', 'rw', newline='') as csvfile:
read_file = csv.reader(csvfile)
write_file = csv.writer(csvfile)
Note that code above will trigger an error with a message ValueError: must have exactly one of create/read/write/append mode.
For safety, I preferred to split it into two different files
import csv
in_path = 'some.csv'
out_path = 'Out.csv'
with open(in_path, 'r', newline='') as inputFile, open(out_path, 'w', newline='') as writerFile:
read_file = csv.reader(inputFile)
write_file = csv.writer(writerFile, delimiter=' ', quotechar='|', quoting=csv.QUOTE_MINIMAL)
for row in read_file:
# your modifying input data code here
........

How do I add to a variable in separate file using python?

My situation is, I have csv file and here is its code.
user_file = Path(str(message.author.id) + '.cvs')
if user_file.exists():
with open('test.csv', 'a') as fp:
writer = csv.writer(fp, delimiter=',')
writer.writerows(data)
else:
with open(user_file, 'w') as fp:
data = [('xp', 0)]
writer = csv.writer(fp, delimiter=',')
writer.writerows(data)
I'm wanting a csv file that keeps track of how many times they type a message so i need a way of editing the csv file and adding 1 to what it already has. But i have no idea how to do that! please help!<3

test.csv:
4
Python:
# Replace test.csv with the file you wish to open. Keep "w+"
with open("test.csv", "w+") as dat:
# Assumes the text in the file is an int
n = int(dat.read())
dat.write(str(n+1))
Result in test.csv:
5
This way it opens the file as write and read, reads the number, then writes it back as a string. Note that write() will override any text in the current file, so you don't need to remove the text
P.S if that if else statement is to check the file actually exists, it's unnecessary. If you try to open() a file which doesn't exist, python will create it for you.

Merging several csv files and storing the file names as a variable - Python

I am trying to append several csv files into a single csv file using python while adding the file name (or, even better, a sub-string of the file name) as a new variable. All files have headers. The following script does the trick of merging the files, but does not cover the file name as variable issue:
import glob
filenames=glob.glob("/filepath/*.csv")
outputfile=open("out.csv","a")
for line in open(str(filenames[1])):
outputfile.write(line)
for i in range(1,len(filenames)):
f = open(str(filenames[i]))
f.next()
for line in f:
outputfile.write(line)
outputfile.close()
I was wondering if there are any good suggestions. I have about 25k small size csv files (less than 100KB each).

You can use Python's csv module to parse the CSV files for you, and to format the output. Example code (untested):
import csv
with open(output_filename, "wb") as outfile:
writer = None
for input_filename in filenames:
with open(input_filename, "rb") as infile:
reader = csv.DictReader(infile)
if writer is None:
field_names = ["Filename"] + reader.fieldnames
writer = csv.DictWriter(outfile, field_names)
writer.writeheader()
for row in reader:
row["Filename"] = input_filename
writer.writerow(row)
A few notes:
Always use with to open files. This makes sure they will get closed again when you are done with them. Your code doesn't correctly close the input files.
CSV files should be opened in binary mode.
Indices start at 0 in Python. Your code skips the first file, and includes the lines from the second file twice. If you just want to iterate over a list, you don't need to bother with indices in Python. Simply use for x in my_list instead.

Simple changes will achieve what you want:
For the first line
outputfile.write(line) -> outputfile.write(line+',file')
and later
outputfile.write(line+','+filenames[i])