I have been reviewing the tutorial for file management in Python 3 but it doesn't mention how to create a file if one doesn't exist. How can I do that?
Just open the file in w mode, and it will be created it.
If you want to open an existing file if possible, but create a new file otherwise (and don't want to truncate an existing file), read the paragraph in your link that lists the modes. Or, for complete details, see the open reference docs. For example, if you want to append to the end instead of overwriting from the start, use a.
Just open the file in write mode:
f = open('fileToWrite.txt', 'w')
Note that this will clobber an existing file. The safest approach is to use append mode:
f = open('fileToWrite.txt', 'a')
As mentioned in this answer, it's generally better to use a with statement to ensure that the file is closed when you have finished with it.
A new file is only created in write or append modes.
open('file', 'w')
In shell:
$ ls
$ python -c 'open("file", "w")'
$ ls
file
$
Of course.
with open('newfile.txt', 'w') as f:
f.write('Text in a new file!')
There are two types of files you can make. a text and a binary.
to make a text file just use file = open('(file name and location goes here).txt', 'w').
to make a binary file you first import pickle, then to put data (like lists numbers ect..) in them you will need to use 'wb' and pickle.dump(data, file_variable) to take out you will need to use 'rb' and pickle.load(file_variable) and give that a variable becuase that is how you refrence the data.
Here is a exaple:
import pickle #bring in pickle
shoplistfile = 'shoplist.data'
shoplist = ['apple', 'peach', 'carrot', 'spice'] #create data
f = open(shoplistfile, 'wb') # the 'wb'
pickle.dump(shoplist, f) #put data in
f.close
del shoplist #delete data
f = open(shoplistfile, 'rb') #open data remember 'rb'
storedlist = pickle.load(f)
print (storedlist) #output
note that if such a file exists it will be writen over.
Related
I need to create a text file in Python to store certain data from a game. I do not want to use numpy, or any external libraries if at all possible.
I need to put some numerical data. Do text files require string data? Also does the data come out of the file as a string?
I know how to create and open a text file, and how to convert string to integer and vice versa, as well as handle CSV file data. I do not know how to handle a text file.
Any ideas on what to do?
To create a file:
file = open("textfile.txt","w+")
This will create a file if it doesn't exist in the directory.
To write inside it:
file.write("This is the content of the file.")
And then you'll have to close the instance with
file.close()
by using the with open command you can create and use it
here is an example
Here w is for writing mode
with open('test.txt','w') as d:
d.write('your text goes here')
You can write to file like this if the file not exists then it will be created
Any ideas on what to do?
Put your data into dict and use built-in json module, example:
import json
data = {'gold': 500, 'name': 'xyzzy'}
# writing
with open('save.json', 'w') as f:
json.dump(data, f)
# reading
with open('save.json', 'r') as f:
data2 = json.load(f)
This create human-readable text file.
In python, there are a few flags you can supply when opening a file for operation. I am a bit baffled at finding a combination that allow me to do random write without truncating. The behavior I am looking for is equivalent to C: create it if it doesn't exist, otherwise, open for write (not truncating)
open(filename, O_WRONLY|O_CREAT)
Python's document is confusing (to me): "w" will truncate the file first, "+" is supposed to mean updating, but "w+" will truncate it anyway. Is there anyway to achieve this without resorting to the low-level os.open() interface?
Note: the "a" or "a+" doesn't work either (please correct if I am doing something wrong here)
cat test.txt
eee
with open("test.txt", "a+") as f:
f.seek(0)
f.write("a")
cat test.txt
eeea
Is that so the append mode insist on writing to the end?
You can do it with os.open:
import os
f = os.fdopen(os.open(filename, os.O_RDWR | os.O_CREAT), 'rb+')
Now you can read, write in the middle of the file, seek, and so on. And it creates the file. Tested on Python 2 and 3.
You should try reading the file then open writing mode, as seen here:
with open("file.txt") as reading:
r = reading.read()
with open("file.txt", "w") as writing:
writing.write(r)
According to the discussion Difference between modes a, a+, w, w+, and r+ in built-in open function, the open with a mode will always write to the end of file irrespective of any intervening fseek(3) or similar.
If you only want to use python built-in function. I guess the solution is to first check if the file exist, and then open with r+ mode.
For Example:
import os
filepath = "test.txt"
if not os.path.isfile(filepath):
f = open(filepath, "x") # open for exclusive creation, failing if the file already exists
f.close()
with open(filepath, "r+") as f: # random read and write
f.seek(1)
f.write("a")
You need to use "a" to append, it will create the file if it does not exist or append to it if it does.
You cannot do what you want with append as the pointer automatically moves to the end of the file when you call the write method.
You could check if the file exists then use fileinput.input with inplace=True inserting a line on whichever line number you want.
import fileinput
import os
def random_write(f, rnd_n, line):
if not os.path.isfile(f):
with open(f, "w") as f:
f.write(line)
else:
for ind, line in enumerate(fileinput.input(f, inplace=True)):
if ind == rnd_n:
print("{}\n".format(line) + line, end="")
else:
print(line, end="")
http://linux.die.net/man/3/fopen
a+
Open for reading and appending (writing at end of file). The file is created if it does not exist. The initial file position for reading is at the beginning of the file, but output is always appended to the end of the file.
fileinput makes a f.bak copy of the file you pass in and it is deleted when the output is closed. If you specify a backup extension backup=."foo" the backup file will be kept.
When I use pickle, it works fine and I can dump any load.
The problem is if I close the program and try to dump again, it replaces the old file data with the new dumping. Here is my code:
import pickle
import os
import time
dictionary = dict()
def read():
with open('test.txt', 'rb') as f:
a = pickle.load(f)
print(a)
time.sleep(2)
def dump():
chs = raw_input('name and number')
n = chs.split()
dictionary[n[0]] = n[1]
with open('test.txt', 'wb') as f:
pickle.dump(dictionary, f)
Inpt = raw_input('Option : ')
if Inpt == 'read':
read()
else:
dump()
When you open a file in w mode (or wb), that tells it to write a brand-new file, erasing whatever was already there.
As the docs say:
The most commonly-used values of mode are 'r' for reading, 'w' for writing (truncating the file if it already exists), and 'a' for appending…
In other words, you want to use 'ab', not 'wb'.
However, when you append new dumps to the same file, you end up with a file made up of multiple separate values. If you only call load once, it's just going to load the first one. If you want to load all of them, you need to write code that does that. For example, you can load in a loop until EOFError.
Really, it looks like what you're trying to do is not to append to the pickle file, but to modify the existing pickled dictionary.
You could do that with a function that loads and merges all of the dumps together, like this:
def Load():
d = {}
with open('test.txt', 'rb') as f:
while True:
try:
a = pickle.load(f)
except EOFError:
break
else:
d.update(a)
# do stuff with d
But that's going to get slower and slower the more times you run your program, as you pile on more and more copies of the same values. To do that right you need to load the old dictionary, modify that, and then dump the modified version. And for that, you want w mode.
However, a much better way to persist a dictionary, at least if the keys are strings, is to use dbm (if the values are also strings) or shelve (otherwise) instead of a dictionary in the first place.
Opening a file in "wb" mode truncates the file -- that is, it deletes the contents of the file, and then allows you to work on it.
Usually, you'd open the file in append ("ab") mode to add data at the end. However, Pickle doesn't support appending, so you'll have to save your data to a new file (come up with a different file name -- ask the user or use a command-line parameter such as -o test.txt?) each time the program is run.
On a related topic, don't use Pickle. It's unsafe. Consider using JSON instead (it's in the standard lib -- import json).
I have text file which I want to erase in Python. How do I do that?
In python:
open('file.txt', 'w').close()
Or alternatively, if you have already an opened file:
f = open('file.txt', 'r+')
f.truncate(0) # need '0' when using r+
Opening a file in "write" mode clears it, you don't specifically have to write to it:
open("filename", "w").close()
(you should close it as the timing of when the file gets closed automatically may be implementation specific)
Not a complete answer more of an extension to ondra's answer
When using truncate() ( my preferred method ) make sure your cursor is at the required position.
When a new file is opened for reading - open('FILE_NAME','r') it's cursor is at 0 by default.
But if you have parsed the file within your code, make sure to point at the beginning of the file again i.e truncate(0)
By default truncate() truncates the contents of a file starting from the current cusror position.
A simple example
As #jamylak suggested, a good alternative that includes the benefits of context managers is:
with open('filename.txt', 'w'):
pass
When using with open("myfile.txt", "r+") as my_file:, I get strange zeros in myfile.txt, especially since I am reading the file first. For it to work, I had to first change the pointer of my_file to the beginning of the file with my_file.seek(0). Then I could do my_file.truncate() to clear the file.
Writing and Reading file content
def writeTempFile(text = None):
filePath = "/temp/file1.txt"
if not text: # If not provided return file content
f = open(filePath, "r")
slug = f.read()
return slug
else:
f = open(filePath, "a") # Create a blank file
f.seek(0) # sets point at the beginning of the file
f.truncate() # Clear previous content
f.write(text) # Write file
f.close() # Close file
return text
It Worked for me
If security is important to you then opening the file for writing and closing it again will not be enough. At least some of the information will still be on the storage device and could be found, for example, by using a disc recovery utility.
Suppose, for example, the file you're erasing contains production passwords and needs to be deleted immediately after the present operation is complete.
Zero-filling the file once you've finished using it helps ensure the sensitive information is destroyed.
On a recent project we used the following code, which works well for small text files. It overwrites the existing contents with lines of zeros.
import os
def destroy_password_file(password_filename):
with open(password_filename) as password_file:
text = password_file.read()
lentext = len(text)
zero_fill_line_length = 40
zero_fill = ['0' * zero_fill_line_length
for _
in range(lentext // zero_fill_line_length + 1)]
zero_fill = os.linesep.join(zero_fill)
with open(password_filename, 'w') as password_file:
password_file.write(zero_fill)
Note that zero-filling will not guarantee your security. If you're really concerned, you'd be best to zero-fill and use a specialist utility like File Shredder or CCleaner to wipe clean the 'empty' space on your drive.
You have to overwrite the file. In C++:
#include <fstream>
std::ofstream("test.txt", std::ios::out).close();
You can also use this (based on a few of the above answers):
file = open('filename.txt', 'w')
file.close()
of course this is a really bad way to clear a file because it requires so many lines of code, but I just wrote this to show you that it can be done in this method too.
happy coding!
You cannot "erase" from a file in-place unless you need to erase the end. Either be content with an overwrite of an "empty" value, or read the parts of the file you care about and write it to another file.
Assigning the file pointer to null inside your program will just get rid of that reference to the file. The file's still there. I think the remove() function in the c stdio.h is what you're looking for there. Not sure about Python.
Since text files are sequential, you can't directly erase data on them. Your options are:
The most common way is to create a new file. Read from the original file and write everything on the new file, except the part you want to erase. When all the file has been written, delete the old file and rename the new file so it has the original name.
You can also truncate and rewrite the entire file from the point you want to change onwards. Seek to point you want to change, and read the rest of file to memory. Seek back to the same point, truncate the file, and write back the contents without the part you want to erase.
Another simple option is to overwrite the data with another data of same length. For that, seek to the exact position and write the new data. The limitation is that it must have exact same length.
Look at the seek/truncate function/method to implement any of the ideas above. Both Python and C have those functions.
This is my method:
open the file using r+ mode
read current data from the file using file.read()
move the pointer to the first line using file.seek(0)
remove old data from the file using file.truncate(0)
write new content and then content that we saved using file.read()
So full code will look like this:
with open(file_name, 'r+') as file:
old_data = file.read()
file.seek(0)
file.truncate(0)
file.write('my new content\n')
file.write(old_data)
Because we are using with open, file will automatically close.
I am trying to add a new row to my old CSV file. Basically, it gets updated each time I run the Python script.
Right now I am storing the old CSV rows values in a list and then deleting the CSV file and creating it again with the new list value.
I wanted to know are there any better ways of doing this.
with open('document.csv','a') as fd:
fd.write(myCsvRow)
Opening a file with the 'a' parameter allows you to append to the end of the file instead of simply overwriting the existing content. Try that.
I prefer this solution using the csv module from the standard library and the with statement to avoid leaving the file open.
The key point is using 'a' for appending when you open the file.
import csv
fields=['first','second','third']
with open(r'name', 'a') as f:
writer = csv.writer(f)
writer.writerow(fields)
If you are using Python 2.7 you may experience superfluous new lines in Windows. You can try to avoid them using 'ab' instead of 'a' this will, however, cause you TypeError: a bytes-like object is required, not 'str' in python and CSV in Python 3.6. Adding the newline='', as Natacha suggests, will cause you a backward incompatibility between Python 2 and 3.
Based in the answer of #G M and paying attention to the #John La Rooy's warning, I was able to append a new row opening the file in 'a'mode.
Even in windows, in order to avoid the newline problem, you must declare it as newline=''.
Now you can open the file in 'a'mode (without the b).
import csv
with open(r'names.csv', 'a', newline='') as csvfile:
fieldnames = ['This','aNew']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writerow({'This':'is', 'aNew':'Row'})
I didn't try with the regular writer (without the Dict), but I think that it'll be ok too.
If you use pandas, you can append your dataframes to an existing CSV file this way:
df.to_csv('log.csv', mode='a', index=False, header=False)
With mode='a' we ensure that we append, rather than overwrite, and with header=False we ensure that we append only the values of df rows, rather than header + values.
Are you opening the file with mode of 'a' instead of 'w'?
See Reading and Writing Files in the python docs
7.2. Reading and Writing Files
open() returns a file object, and is most commonly used with two arguments: open(filename, mode).
>>> f = open('workfile', 'w')
>>> print f <open file 'workfile', mode 'w' at 80a0960>
The first argument is a string containing the filename. The second argument is
another string containing a few characters describing the way in which
the file will be used. mode can be 'r' when the file will only be
read, 'w' for only writing (an existing file with the same name will
be erased), and 'a' opens the file for appending; any data written to
the file is automatically added to the end. 'r+' opens the file for
both reading and writing. The mode argument is optional; 'r' will be
assumed if it’s omitted.
On Windows, 'b' appended to the mode opens the file in binary mode, so
there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows
makes a distinction between text and binary files; the end-of-line
characters in text files are automatically altered slightly when data
is read or written. This behind-the-scenes modification to file data
is fine for ASCII text files, but it’ll corrupt binary data like that
in JPEG or EXE files. Be very careful to use binary mode when reading
and writing such files. On Unix, it doesn’t hurt to append a 'b' to
the mode, so you can use it platform-independently for all binary
files.
If the file exists and contains data, then it is possible to generate the fieldname parameter for csv.DictWriter automatically:
# read header automatically
with open(myFile, "r") as f:
reader = csv.reader(f)
for header in reader:
break
# add row to CSV file
with open(myFile, "a", newline='') as f:
writer = csv.DictWriter(f, fieldnames=header)
writer.writerow(myDict)
I use the following approach to append a new line in a .csv file:
pose_x = 1
pose_y = 2
with open('path-to-your-csv-file.csv', mode='a') as file_:
file_.write("{},{}".format(pose_x, pose_y))
file_.write("\n") # Next line.
[NOTE]:
mode='a' is append mode.
# I like using the codecs opening in a with
field_names = ['latitude', 'longitude', 'date', 'user', 'text']
with codecs.open(filename,"ab", encoding='utf-8') as logfile:
logger = csv.DictWriter(logfile, fieldnames=field_names)
logger.writeheader()
# some more code stuff
for video in aList:
video_result = {}
video_result['date'] = video['snippet']['publishedAt']
video_result['user'] = video['id']
video_result['text'] = video['snippet']['description'].encode('utf8')
logger.writerow(video_result)