This is some example data from the csv I'm reading/writing to:
ALTIMA_SD,2014,12,Promoter,Promoter,3/5/2015
ALTIMA_SD,2015,3,Promoter,Promoter,3/5/2015
PATHFINDER,2014,12,Promoter,Promoter,3/5/2015
I'm looking to add a column between the first two, so that the end product looks like this:
ALTIMA_SD,Comparo,2014,12,Promoter,Promoter,3/5/2015
SENTRA,Comparo,2015,3,Promoter,Promoter,3/5/2015
PATHFINDER,Pathfinder,2014,12,Promoter,Promoter,3/5/2015
This is my code:
with open('pathfinder query2.csv','r+') as myfile:
for i in myfile:
i=i.rstrip('\n')
i=i.split(',')
if i[0]=='PATHFINDER' or i[0]=='PATHFINDER_H':
myfile.write(str(i[0])+','+'Pathfinder'+','+str(i[1])+','+str(i[2])+','+str(i[3])+','+str(i[4])+','+str(i[5])+'\n')
else:
myfile.write(str(i[0])+','+'Comparo'+','+str(i[1])+','+str(i[2])+','+str(i[3])+','+str(i[4])+','+str(i[5])+'\n')
I'm getting a IOError: [Errno 0] Error in reference to line 8. I've seen in other posts that the myfile.seek() function should be used, but I'm not sure how to apply it. Any help would be greatly appreciated.
Opening a file in Python is similar to fopen in C, see here for an overview of the modes. The problem is you are opening the file with r+ (read + update), while you need to open your file with w+ (write + update). Hence you must change your code to:
with open('pathfinder query2.csv','w+') as myfile:
Also as commented (Thx. Jon.), it's not wise to read and write to the same file. I would recommend you make a copy of the file or read the entire file to a buffer.
Related
If I run
file = open("BAL.txt","w")
I = '200'
file.write(I)
file.close
from a script, it outputs nothing in the file. (It literally overwrites the file with nothing)
Furthermore, running cat BAL.txt just goes to the next line like nothing is in the file.
But if I run it line by line in a python console it works perfectly fine.
Why does this happen. ( I am a begginner learning python the mistake may be super obvious. I have thrown about 2 hours into trying to figure this out)
Thanks in advance
You aren't closing your file properly. To close it you are missing the () at the end of file.close so it should look like this:
file = open("BAL.txt", "w")
file.write("This has been written to a file")
file.close()
This site has the same example and may be of some use to you.
Another way, especially useful when you are appending multiple values into a single file is to use something like with open("BAL.txt","w") as file:. Here is your script rewritten to include this example:
I = '200'
with open("BAL.txt","w") as file:
file.write(I)
This opens our file with the value file and allows us to write values to it. Also note that file.close() is not needed here and when appending text w+ needs to be used.
to write to a file you do this:
file = open("file.txt","w")
file.write("something")
file.close()
when you use file.write() it deletes all of the contents of the file, if you want to write to the end of the file do this:
file = open("file.text","w+")
file.write(file.read()+"something")
file.close()
There are other ways to do this but this one is the most intuitive (not the most efficient), also the other way tends to be buggy so there is no reason to post it because this is reliable.
Firstly, you're missing the parentheses when you're closing the file. Secondly, writing to a file should be done like this:
file = open("BAL.txt", "w")
file.write("This has been written to a file")
file.close()
Let me know if you have any questions.
I am bassicly trying to read a number from a file, convert it to an int, add one to it, then rewrite the new number back to the file. However every time I run this code when i open the .txt file it is blank. Any help would be appreciated thanks! I am a python newb.
f=open('commentcount.txt','r')
counts = f.readline()
f.close
counts1 = int(counts)
counts1 = counts1 + 1
print(counts1)
f2 = open('commentcount.txt','w') <---(the file overwriting seems to happen here?)
f2.write(str(counts1))
Having empty files
This issue is caused by you failing to close the file descriptor. You have f.close but it should be f.close() (a function call). And you also need an f2.close() in the end.
Without the close it takes a while until the contents of the buffer arrive in the file. And it is a good practice to close file descriptors as soon as they are not used.
As a side note, you can use the following syntactic sugar to ensure that the file descriptor is closed as soon as possible:
with open(file, mode) as f:
do_something_with(f)
Now, regarding the overwriting part:
Writing to file without overwriting the previous content.
Short answer: You don't open the file in the proper mode. Use the append mode ("a").
Long answer:
It is the intended behavior. Read the following:
>>> help(open)
Help on built-in function open in module __builtin__:
open(...)
open(name[, mode[, buffering]]) -> file object
Open a file using the file() type, returns a file object. This is the
preferred way to open a file. See file.__doc__ for further information.
>>> print file.__doc__
file(name[, mode[, buffering]]) -> file object
Open a file. The mode can be 'r', 'w' or 'a' for reading (default),
writing or appending. The file will be created if it doesn't exist
when opened for writing or appending; it will be truncated when
opened for writing. Add a 'b' to the mode for binary files.
Add a '+' to the mode to allow simultaneous reading and writing.
If the buffering argument is given, 0 means unbuffered, 1 means line
buffered, and larger numbers specify the buffer size. The preferred way
to open a file is with the builtin open() function.
Add a 'U' to mode to open the file for input with universal newline
support. Any line ending in the input file will be seen as a '\n'
in Python. Also, a file so opened gains the attribute 'newlines';
the value for this attribute is one of None (no newline read yet),
'\r', '\n', '\r\n' or a tuple containing all the newline types seen.
So, reading the manuals shows that if you want the content to be kept you should open in append mode:
open(file, "a")
you should use the with statement. this assume that the file descriptor is closed no matter what:
with open('file', 'r') as fd:
value = int(fd.read())
with open('file', 'w') as fd:
fd.write(value + 1)
You never close the file. If you don't properly close the file the OS might not commit any changes. To avoid this problem it is recommended that you use Python's with statement to open files as it it will close them for you once you are done with the file.
with open('my_file.txt', a) as f:
do_stuff()
python open file paramters:
w:
Opens a file for writing only. Overwrites the file if the file exists.
If the file does not exist, creates a new file for writing.
You can use a (append):
Opens a file for appending. The file pointer is at the end of the file
if the file exists. That is, the file is in the append mode. If the
file does not exist, it creates a new file for writing.
for more information you can read here
One more advice is to use with:
with open("x.txt","a") as f:
data = f.read()
............
For example:
with open('c:\commentcount.txt','r') as fp:
counts = fp.readline()
counts = str(int(counts) + 1)
with open('c:\commentcount.txt','w') as fp:
fp.write(counts)
Note this will work only if you have a file name commentcount and it has a int at the first line since r does not create new file, also it will be only one counter...it won't append a new number.
I am attempting to store a csv file on an ftp server using python's ftplib module.
Right now, I have about 30 lines of code which generates probabilities of weather values in a 2-d array. I then write this 2-d array to a csv file.
When I write the csv file onto my local drive, the file displays as expected within excel. However, when I view the file after I uploaded it to an ftp server, I see that a new line character has been added after every row.
I've done some minor testing to see what the problem may be, and I have been able to upload the csv file with coreftp. The csv file displays correctly after I do that. So I am pretty sure the file is fine, its something that is happening when python uploads it onto an ftp server.
I was originally creating a text file with a .csv extension file then reopening it as a binary file and uploading it. I thought that may be the issue so I tried using the csv module, but same issue.
Here is my code at the moment...
TEMPSHEADER = [i-50 for i in range(181)]#upper bounds exclusive
WINDSHEADER = [i for i in range(101)]#upper bounds exclusive
HEADER = TEMPSHEADER + WINDSHEADER
for site in ensmosdic:
ensmos = ensmosdic.get(site)
with open(utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.csv","w",newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=",")
writer.writerow(["CODE ","F","ForecastDate","HOUR"]+HEADER)
siteTable =[[0 for x in range(286)] for y in range(24,169)]#upper bounds exclusive
###########
#other code here, but not important with regards to post
###########
for i in siteTable:
writer.writerow(i)
csvfile.close()#not sure if you have to close csv file, not in csv module docs
f = open(utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.csv","rb")
ftpInno.storbinary("STOR " + utcnow.strftime("%Y-%m-%d-") + site +"-prob.csv",f)
f.close()
ftpInno.close()
Thanks in advance
After an hour or so of trouble shooting, the answer is fairly simple, although I am not entirely sure why it works.
what i did was create a text file instead of a csv file which I was doing in my original question
with open(FILELOCATION + utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.txt","w") as f:
#write to the file below
f.close()
#open file again as a txt file
f = open(FILELOCATION + utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.txt","rb")
ftp.storlines("STOR " + utcnow.strftime("%Y-%m-%d-") + site +"-prob.csv",f)
f.close()
reading the file as a binary file and then storing it with the storlines method removed the extra lines I was seeing within the file after I uploaded it to an ftp server.
This might shed some light on your issue. I had a project where I was using windows command line and also windows powershell to transfer .csv files with the ftp get and mget commands. And like you said I was getting a extra between each row. It seems like switching to binary transfer mode fixed my issue. For example once you are in the ftp dialog just type "binary" and hit enter and it switches the mode.
def ConvertFile():
FileNameIn = 'Hexdata.dat'
HexFile = open(FileNameIn, 'r')
for Line in HexFile:
print (Line)
print (Binary(Line))
HexFile.close()
So far I have that, which, when the program is run, converts the Hexidecimal number in the file to binary. This is in a file called Hexdata.dat
What I want to do is then save the binary output into a file called Binarydata.dat
How would I approach this in code? Be aware I'm new with Python and haven't covered this properly. I've tried different bits of code but they've all been unsuccessful, as really, they're all guesses.
I'm not asking you to solve the problem for me, but more asking how I would save the output of a program into a new text file.
You're already most of the way there. You already know how to open a file for reading:
HexFile = open(FileNameIn, 'r')
The 'r' there means "open for reading". If you look at the documentation for the open function, you will see that replacing the r with a w will open a file for writing:
OutputFile = open(FileNameOut, 'w')
And then you can send output to it like this:
print >>OutputFile, "Something to print"
Or use the write method on the file object:
OutputFile.write("Something to print\n")
Read the documentation of the open function (to open the file in write mode) and File Objects (to write information to the opened file).
You have to have 2 files in this script. The one you're reading from and the one you're writing to. Use the option wb (write binary) when opening the file you are going to write into. These two links should help a beginner with little or no Python knowledge complete your exercise: Intro to File Objects and Tutorial on File I/O.
You are currently opening the file in reading mode, so in order to write to the file, you would want to open the file with the buffering mode as ('w'). Quote from: http://docs.python.org. You can do so easily by replacing your 'r' with 'w'.
'w' for writing (truncating the file if it already exists
For more reference see open(name[, mode[, buffering]])
# the file name
FileNameIn = 'Hexdata.dat'
# create a file object: open it with "write" mode
HexFile = open(FileNameIn,"w")
for line in HexFile:
HexFile.write(Binary(line))
HexFile.close()
Have you tried using open('Binarydata.dat', 'w') for writing to the file? There are plenty of ways to write to a file, most of which can be found here: http://docs.python.org/tutorial/inputoutput.html
I have text file which I want to erase in Python. How do I do that?
In python:
open('file.txt', 'w').close()
Or alternatively, if you have already an opened file:
f = open('file.txt', 'r+')
f.truncate(0) # need '0' when using r+
Opening a file in "write" mode clears it, you don't specifically have to write to it:
open("filename", "w").close()
(you should close it as the timing of when the file gets closed automatically may be implementation specific)
Not a complete answer more of an extension to ondra's answer
When using truncate() ( my preferred method ) make sure your cursor is at the required position.
When a new file is opened for reading - open('FILE_NAME','r') it's cursor is at 0 by default.
But if you have parsed the file within your code, make sure to point at the beginning of the file again i.e truncate(0)
By default truncate() truncates the contents of a file starting from the current cusror position.
A simple example
As #jamylak suggested, a good alternative that includes the benefits of context managers is:
with open('filename.txt', 'w'):
pass
When using with open("myfile.txt", "r+") as my_file:, I get strange zeros in myfile.txt, especially since I am reading the file first. For it to work, I had to first change the pointer of my_file to the beginning of the file with my_file.seek(0). Then I could do my_file.truncate() to clear the file.
Writing and Reading file content
def writeTempFile(text = None):
filePath = "/temp/file1.txt"
if not text: # If not provided return file content
f = open(filePath, "r")
slug = f.read()
return slug
else:
f = open(filePath, "a") # Create a blank file
f.seek(0) # sets point at the beginning of the file
f.truncate() # Clear previous content
f.write(text) # Write file
f.close() # Close file
return text
It Worked for me
If security is important to you then opening the file for writing and closing it again will not be enough. At least some of the information will still be on the storage device and could be found, for example, by using a disc recovery utility.
Suppose, for example, the file you're erasing contains production passwords and needs to be deleted immediately after the present operation is complete.
Zero-filling the file once you've finished using it helps ensure the sensitive information is destroyed.
On a recent project we used the following code, which works well for small text files. It overwrites the existing contents with lines of zeros.
import os
def destroy_password_file(password_filename):
with open(password_filename) as password_file:
text = password_file.read()
lentext = len(text)
zero_fill_line_length = 40
zero_fill = ['0' * zero_fill_line_length
for _
in range(lentext // zero_fill_line_length + 1)]
zero_fill = os.linesep.join(zero_fill)
with open(password_filename, 'w') as password_file:
password_file.write(zero_fill)
Note that zero-filling will not guarantee your security. If you're really concerned, you'd be best to zero-fill and use a specialist utility like File Shredder or CCleaner to wipe clean the 'empty' space on your drive.
You have to overwrite the file. In C++:
#include <fstream>
std::ofstream("test.txt", std::ios::out).close();
You can also use this (based on a few of the above answers):
file = open('filename.txt', 'w')
file.close()
of course this is a really bad way to clear a file because it requires so many lines of code, but I just wrote this to show you that it can be done in this method too.
happy coding!
You cannot "erase" from a file in-place unless you need to erase the end. Either be content with an overwrite of an "empty" value, or read the parts of the file you care about and write it to another file.
Assigning the file pointer to null inside your program will just get rid of that reference to the file. The file's still there. I think the remove() function in the c stdio.h is what you're looking for there. Not sure about Python.
Since text files are sequential, you can't directly erase data on them. Your options are:
The most common way is to create a new file. Read from the original file and write everything on the new file, except the part you want to erase. When all the file has been written, delete the old file and rename the new file so it has the original name.
You can also truncate and rewrite the entire file from the point you want to change onwards. Seek to point you want to change, and read the rest of file to memory. Seek back to the same point, truncate the file, and write back the contents without the part you want to erase.
Another simple option is to overwrite the data with another data of same length. For that, seek to the exact position and write the new data. The limitation is that it must have exact same length.
Look at the seek/truncate function/method to implement any of the ideas above. Both Python and C have those functions.
This is my method:
open the file using r+ mode
read current data from the file using file.read()
move the pointer to the first line using file.seek(0)
remove old data from the file using file.truncate(0)
write new content and then content that we saved using file.read()
So full code will look like this:
with open(file_name, 'r+') as file:
old_data = file.read()
file.seek(0)
file.truncate(0)
file.write('my new content\n')
file.write(old_data)
Because we are using with open, file will automatically close.