i've run into a problem where: My code creates a file with headers, writes data to it. And then when i run it for a second time it overwrites the data, which it should but start a new line. Also, what does delimiter mean?
#Intro
import csv
headers=["Name","Age","Year Group"]
with open("Details.csv","a" and "w") as i:
w=csv.writer(i,delimiter=",")
w.writerow(headers)
print("Welcome User, to my Topics Quiz!\n------------------------------
---------\nYou can choose from 3 different topics:\n • History\n • Music\n •
Computer Science\n---------------------------------------")
#Storing: User's name, age and year group
print("Before we start, we need to register an account.")
User=input("Enter your name:\n")
Age=input("Enter your age:\n")
Year=input("Enter your year group:\n")
details=[User,Age,Year]
w.writerow(details)
with open("UserPass.csv","a" and "w") as Userpass:
w=csv.writer(Userpass,delimiter=",")
headers=["Username","Password"]
w.writerow(headers)
NewUser=(User[:3]+Age)
print("Great! Your username is set to: {}".format(NewUser))
Pass=input("Enter a password for your account:\n")
userpass=[NewUser,Pass]
w.writerow(userpass)
So the code will write out the data when i want it to add.
Thanks in advance.
You're mode is only "w":
>>> "a" and "w" == "w"
True
Instead use only "a".
a is enough for the open mode.
Append: "a" has Write: "w" in its definition. When you say you want to append it means you want to write at the end... So don't use them together.
delimiter means separator which most of the times refers to some characters like comma, space, dot, etc.
If you wish to append to an existing .csv file you need to either:
skip writing the header if the file already exists, (check with os.path.exists() before opening it) or
open the file in read mode & read the file into memory first, (as a list of lists), close it then add the new row to the data and then overwrite the whole thing, including headers, over the original, (this does let you do things like sorting the rows).
As others have said your file open mode should be one of:
"r" = Read Text
"rb" = Read Binary
"w" = *Over***W**rite Text
"wb" = *Over***Write **Binary
"a" = Append Text
"ab" = Append Binary
The delimiter specifies what separates the fields in the .csv file.
As an aside you should never store passwords - instead you should store a hash of the password and when the user next enters a password calculate the same hash and compare it with the stored hash. Something like:
import hashlib
import getpass
pswd = getpass.getpass()
userpass = hashlib.sha256(pswd.encode('ascii')).hexdigest()
As for the term delimiter, remember that CSV stands for Comma Seperated Values. Delimiter means seperator; CSV uses a comma (",") as its seperator or delimiter, but the python module csv gives you the option to specify a different delimiter, such as a tab character ("\t"), etc.
As for the file operations, you could check if the file is blank. If it is, write the headers. Then (always) append the data. Something like this:
# open the file for appending (the "a"). Create it if it doesn't exist (the "+")
with open(filename, "a+") as f:
w = csv.writer(f, delimiter=",")
# only write the headers if the file is blank (i.e. the first time the program runs)
if f.read() == "":
w.writerows[headers]
# get `details` from the user.
# ...
# the data always gets appended
w.writerows(details)
Hope this helps!
Related
This question already has answers here:
Why should text files end with a newline?
(19 answers)
Closed 11 months ago.
I have created a program in python that uses a .txt file as a database, so I save the user data there.
The data is stored in the file as follows:
Each line in the file represents a user
Users are stored as: User_ID,First Name,Last Name,Phone Number,Country
The problem arises when I try to delete a user that is in the last position of the file, because it leaves a blank line that breaks the function of listing the users in the program.
Example:
Suppose I have two users in the text file: image1
And now I delete the last user (00002): image2
So, there is a blank line, and when I add a new user, it looks like this: image3
Here is the code I use to delete users:
def delete_user(file, user_id):
with open(file, 'r+') as f:
users = f.readlines()
f.seek(0)
for user in users:
if user.split(',')[0] != user_id:
f.write(user)
f.truncate()
Code I use to add users:
def add_user(file, first_name, last_name, phone_number, country):
filesize = os.path.getsize(file)
with open(file, 'r+') as f:
if filesize == 0:
f.write(f'{"1".zfill(5)},{first_name},{last_name},{phone_number},'
f'{country}')
else:
last_line = f.readlines()[-1]
user_id = int(last_line.split(',')[0])
f.write(f'\n{str(user_id+1).zfill(5)},{first_name},{last_name},'
f'{phone_number},{country}')
Upon investigation, I realized that the problem occurs because there is a newline character (\n) left at the end of the last line, which should not be there.
So, how could I solve this problem?
A trailing newline shouldn't break your program. You seem to be using a newline as a line separator, but on Unix-like OSs, a newline is actually considered a line terminator, meaning there should be one at the end of every line.
However, you'll have a much easier time by using a standard data format. What you're using resembles CSV, so I recommend csv.DictReader and csv.DictWriter, which would also involve putting a column header in the file to label the fields. You might also consider using Pandas (e.g. pandas.read_csv() and df.to_csv()).
I have a database.txt file the first column is for usernames the second passwords and the rest 5 recovery question and answers alternating. I want to allow the user to be able to change the password of their details, without affecting another users username as they may be the same. I have found a way to delete the previous one and append the new line of modified details to the file. However, the is always a string or unknown characters at the start of the appended line. AND other characters are being changed not the second value in the list. Please help me find a way to avoid this.
https://repl.it/repls/NecessaryBoldButtonsYou can find the code here changing it will affect everyone, so please copy it elsewhere.
https://onlinegdb.com/BJbsn9-cL
I just need the password to be changed on a user input not other strings, the reason for all this code is that when changing a person's password another username could be changed.This is the original file
This is what happens afterwards, the second string in the list of the line which where data[0] = "bye" should only be changed to newpass, not all of the others
'''
import linecache
f = open("database.txt" , "r+")
for loop in range(3):
line = f.readline()
data = line.split(",")
if data[1] == "bye":
print(data[1]) #These are to help me understand what is happening
print(data[0])
b = data[0]
newpass = "Hi"
a = data[1]
fn = 'database.txt'
e = open(fn)
output = []
str="happy"
for line in e:
if not line.startswith(str):
output.append(line)
e.close()
print(output)
e = open(fn, 'w')
e.writelines(output)
e.close()
line1 = linecache.getline("database.txt" ,loop+1)
print(line)
password = True
print("Password Valid\n")
write = (line1.replace(a, newpass))
write = f.write(line1.replace(a, newpass))
f.close()
'''
This is the file in text:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3,Recovery4,Answer4,
Recovery5,Answer5,o,o,o,o,o,o,o,o,o,o,
happy,bye,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
Support is very much appreciated
Feel free to change the code as much as you need to, as it is already a mess
Thanks in Advance
This should be pretty easy. The basic idea is:
open input file for reading
open output file for writing
for each line in input file
if password = "happy"
change user name in line
write line to output file
It should be pretty easy to convert that to python.
From comments, and by examining your code, I get the feeling that you're trying to update a line in-place. That is, it looks like your expectation is that given the file "database.txt" that contains this:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3, Recovery4,Answer4,Recovery5,Answer5,
o,o,o,o,o,o,o,o,o,o,
happy,bye,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
When you make the change, your new "database.txt" will contain this:
username,password,Recovery1,Answer1,Recovery2,Answer2,Recovery3,Answer3, Recovery4,Answer4,Recovery5,Answer5,
o,o,o,o,o,o,o,o,o,o,
happy,Hi,o,o,o,o,o,o,o,o,o,o,
bye,happy,o,o,o,o,o,o,o,o,o,o,
You can do that, but you can't do it in-place. You have to write all the lines of the file, including the changed line, to a new temporary file. Then you can delete the old "database.txt" and rename the temporary file.
You can't update a line in a text file, because if you change the length of the line then you'll either end up with extra space at the end of the line you changed (because the new line has fewer characters than the old line), or you'll overwrite the beginning of the next line (the new line is longer than the old line).
The only other option is to load all of the lines into memory and close the file. Then change the line or lines you want to change, in memory. Finally, open the "database.txt" file for writing and output all of the lines from memory to the file.
I am trying to create a basic mathematical quiz and need to be able to store the name of the user next to their score. To ensure that I could edit the data dynamically regardless of the length of the user's name or the number of digits in their score, I decided to split up the name and score with a comma and use the split function. I'm new to file handling in python so don't know if I am using the wrong mode ("r+") but when I complete the quiz, my score is not recorded at all, nothing is added to the file. Here is my code:
for line in class_results.read():
if student_full_name in line:
student = line.split(",")
student[1] = correct
line.replace(line, "{},{}".format(student_full_name, student[1]))
else:
class_results.write("{},{}".format(student_full_name, correct))
Please let me know how I can get this system to work. Thank you in advance.
Yes r+ opens the file for both reading and writing and to summarize:
r when the file will only be read
w for only writing (an existing file with the same name will be erased)
a opens the file for appending; any data written to the file is automatically added to the end.
I will recommend instead of comma separation to benifit from json or yaml syntax, it fits better in this case.
scores.json:
{
"student1": 12,
"student2": 798
}
The solution:
import json
with open(filename, "r+") as data:
scores_dict = json.loads(data.read())
scores_dict[student_full_name] = correct # if already exist it will be updated otherwise it will be added
data.seek(0)
data.write(json.dumps(scores_dict))
data.truncate()
scores.yml will looks as follow:
student1: 45
student2: 7986
Solution:
import yaml
with open(filename, "r+") as data:
scores_dict = yaml.loads(data.read())
scores_dict[student_full_name] = correct # if already exist it will be updated otherwise it will be added
data.seek(0)
data.write(yaml.dump(scores_dict, default_flow_style=False))
data.truncate()
to instal yaml python package: pip install pyyaml
Modifying a file in place is generally a poor way to do this. It risks errors causing the resulting file to be half new data, half old, with the split point being corrupted. The usual pattern is to write to a new file, then atomically replace the old file with the new file, so either you have the entire original old file and a partial new file, or the new file, not a mish-mash of both.
Given your example code, here is how you would fix it up to do that:
import csv
import os
from tempfile import NamedTemporaryFile
origfile = '...'
origdir = os.path.dirname(origfile)
# Open original file for read, and tempfile in same directory for write
with open(origfile, newline='') as inf, NamedTemporaryFile('w', dir=origdir, newline='') as outf:
old_results = csv.reader(inf)
new_results = csv.writer(outf)
for name, oldscore in old_results:
if name == student_full_name:
# Found our student, replace their score
new_results.writerow((name, correct))
# The write out the rest of the lines unchanged
new_results.writerows(old_results)
# and we're done
break
else:
new_results.writerow((name, oldscore))
else:
# else block on for loop executes if loop ran without break-ing
new_results.writerow((student_full_name, correct))
# If we got here, no exceptions, so let's keep the new data to replace the old
outf.delete = False
# Atomically replaces the original file with the temp file with updated data
os.replace(outf.name, origfile)
I am trying to add a line to the end of a txt file. I have been reading some posts here and trying differents options, but, for some reason, the new line is neved added after the last one, it is just appended next to the last one.
So I was wondering what I am doing wrong....here I am showing my tests:
TEST 1:
#newProt is a new data entered by the user in this case 12345
exists = False
f = open('protocols.txt', 'a+')
for line in f:
if newProt == line:
exists = True
if not exists:
f.write(newProt)
f.close()
txt file after this code:
2sde45
21145
we34z12345
TEST 2:
exists = False
with open('protocols.txt', 'r+') as f:
for line in f:
if newProt == line:
exists = True
if not exists:
f.write(newProt)
txt file after this code: exactly the same as above...
And, like this, I have tested some combinations of letters to open the file, rb+, w, etc but for some reason I never get the desired output txt file:
2sde45
21145
we34z
12345
So I do not know what I am doing wrong, I am following some examples I gor from some other posts here.
Try this:
exists = False
f = open('protocols.txt', 'a+')
for line in f:
if newProt == line:
exists = True
if not exists:
f.write('\n' + newProt)
f.close()
This adds the new line character to the end of the file then adds 'newProt'.
EDIT:
The reason why your code did not produce the desired result is because you were simply writing a string to the file. New lines in text are not really 'in' the text file. The text file is literally a series of bytes known as chars. The reason why various applications such as text editors show you new lines is because it interprets certain characters as formatting elements rather than letters or numbers.
'\n' is one such formatting character (in the ASCII standard), and it tells your favorite text editor to start a new line. There are others such as '\t' which makes a tab.
Have a look at the wiki article on Newline character for more info
You can use f.seek(-x,x), reach the last line and then f.write().
Otherwise my understanding is if you open a file in "a" (append) mode, it'll anyways be written in the end
Refer to this link: Appending line to a existing file having extra new line in Python
I have a file where each line starts with a number. The user can delete a row by typing in the number of the row the user would like to delete.
The issue I'm having is setting the mode for opening it. When I use a+, the original content is still there. However, tacked onto the end of the file are the lines that I want to keep. On the other hand, when I use w+, the entire file is deleted. I'm sure there is a better way than opening it with w+ mode, deleting everything, and then re-opening it and appending the lines.
def DeleteToDo(self):
print "Which Item Do You Want To Delete?"
DeleteItem = raw_input(">") #select a line number to delete
print "Are You Sure You Want To Delete Number" + DeleteItem + "(y/n)"
VerifyDelete = str.lower(raw_input(">"))
if VerifyDelete == "y":
FILE = open(ToDo.filename,"a+") #open the file (tried w+ as well, entire file is deleted)
FileLines = FILE.readlines() #read and display the lines
for line in FileLines:
FILE.truncate()
if line[0:1] != DeleteItem: #if the number (first character) of the current line doesn't equal the number to be deleted, re-write that line
FILE.write(line)
else:
print "Nothing Deleted"
This is what a typical file may look like
1. info here
2. more stuff here
3. even more stuff here
When you open a file for writing, you clobber the file (delete its current contents and start a new file). You can find this out by reading documentation for the open() command.
When you open a file for appending, you do not clobber the file. But how can you delete just one line? A file is a sequence of bytes stored on a storage device; there is no way for you to delete one line and have all the other lines automatically "slide down" into new positions on the storage device.
(If your data was stored in a database, you could actually delete just one "row" from the database; but a file is not a database.)
So, the traditional way to solve this: you read from the original file, and you copy it to a new output file. As you copy, you perform any desired edits; for example, you can delete a line simply by not copying that one line; or you can insert a line by writing it in the new file.
Then, once you have successfully written the new file, and successfully closed it, if there is no error, you go ahead and rename the new file back to the same name as the old file (which clobbers the old file).
In Python, your code should be something like this:
import os
# "num_to_delete" was specified by the user earlier.
# I'm assuming that the number to delete is set off from
# the rest of the line with a space.
s_to_delete = str(num_to_delete) + ' '
def want_input_line(line):
return not line.startswith(s_to_delete)
in_fname = "original_input_filename.txt"
out_fname = "temporary_filename.txt"
with open(in_fname) as in_f, open(out_fname, "w") as out_f:
for line in in_f:
if want_input_line(line):
out_f.write(line)
os.rename(out_fname, in_fname)
Note that if you happen to have a file called temporary_filename.txt it will be clobbered by this code. Really we don't care what the filename is, and we can ask Python to make up some unique filename for us, using the tempfile module.
Any recent version of Python will let you use multiple statements in a single with statement, but if you happen to be using Python 2.6 or something you can nest two with statements to get the same effect:
with open(in_fname) as in_f:
with open(out_fname, "w") as out_f:
for line in in_f:
... # do the rest of the code
Also, note that I did not use the .readlines() method to get the input lines, because .readlines() reads the entire contents of the file into memory, all at once, and if the file is very large this will be slow or might not even work. You can simply write a for loop using the "file object" you get back from open(); this will give you one line at a time, and your program will work with even really large files.
EDIT: Note that my answer is assuming that you just want to do one editing step. As #jdi noted in comments for another answer, if you want to allow for "interactive" editing where the user can delete multiple lines, or insert lines, or whatever, then the easiest way is in fact to read all the lines into memory using .readlines(), insert/delete/update/whatever on the resulting list, and then only write out the list to a file a single time when editing is all done.
def DeleteToDo():
print ("Which Item Do You Want To Delete?")
DeleteItem = raw_input(">") #select a line number to delete
print ("Are You Sure You Want To Delete Number" + DeleteItem + "(y/n)")
DeleteItem=int(DeleteItem)
VerifyDelete = str.lower(raw_input(">"))
if VerifyDelete == "y":
FILE = open('data.txt',"r") #open the file (tried w+ as well, entire file is deleted)
lines=[x.strip() for x in FILE if int(x[:x.index('.')])!=DeleteItem] #read all the lines first except the line which matches the line number to be deleted
FILE.close()
FILE = open('data.txt',"w")#open the file again
for x in lines:FILE.write(x+'\n') #write the data to the file
else:
print ("Nothing Deleted")
DeleteToDo()
Instead of writing out all lines one by one to the file, delete the line from memory (to which you read the file using readlines()) and then write the memory back to disk in one shot. That way you will get the result you want, and you won't have to clog the I/O.
You could mmap the file... after haven read the suitable documentation...
You don't need to check for the lines numbers in your file, you can do something like this:
def DeleteToDo(self):
print "Which Item Do You Want To Delete?"
DeleteItem = int(raw_input(">")) - 1
print "Are You Sure You Want To Delete Number" + str(DeleteItem) + "(y/n)"
VerifyDelete = str.lower(raw_input(">"))
if VerifyDelete == "y":
with open(ToDo.filename,"r") as f:
lines = ''.join([a for i,a in enumerate(f) if i != DeleteItem])
with open(ToDo.filename, "w") as f:
f.write(lines)
else:
print "Nothing Deleted"