I'm creating a xlsx output with xlsxwriter into a temporary file using tempfile module, I store the path to this temporary file inside a variable that I later use in another script to open it.
The problem is that sometimes opening the file fails with the error :
"[Errno 2] No such file or directory: '/tmp/xls5TnVsx'"
Sorry I don't have an exact idea about the frequency of this problem occurring but it seems like it happens from time to time, so I don't understand why...
This is how I save into a temporary file :
f = tempfile.NamedTemporaryFile(prefix="xls",delete=False)
xlsfilename = f.name
Then to create the xlsx output :
wb = xlsxwriter.Workbook(filename)
ws = wb.add_worksheet(sheetName)
# Write header
....
# Write data
for row, row_data in enumerate(data, start=1):
for column, key in enumerate(headers):
....
wb.close()
f.close()
Then in a Python CGI script I use the variable xlsxfilename which is the path to the script to open it :
print "Content-type: application/msexcel"
print "Content-Disposition: attachment; filename="+xlsfilename
print
try :
print open(xlsfilename,"rb").read()
finally:
try:
xlsfilename.close()
except:
pass
os.unlink(xlsfilename)
What am I doing wrong here and any ideas on how to solve this by maybe using another method to storing into a temporary file?
I believe the issue here is that your program is overwriting the created file with its own output, as the
wb = xlsxwriter.Workbook(filename)
statement creates a new file. The conditions under which this might be deleted will depend on when the named temporary file is deleted (technically this happens on close()).
You should think about using mkstemp instead, since you already explicity delete the file you are creating. Overwriting that file, whose name is guaranteed unique and which is not deleted automatically, should be more controllable.
Related
I am currently using two different types of python scripts. One extracts data and saves it as a CSV file, and the other characterizes data. Both work perfectly separately, but I am trying to find a way to characterize the data from the outputted CSV file without having to run them separately. Importing script1 into script2 is easy, but reading the CSV file from script1 is what I can't figure out. I am going to provide the output of script1 and where I am trying to insert it in script2:
# create file or append to file
filename = '%s.csv' % gps
if os.path.exists(filename):
append_write = 'a' # append if already exists
else:
append_write = 'w' # make a new file if not
# save file
with open('!/usr/bin/env python/%s.csv' % gps, mode=append_write) as features_file:
features_writer = csv.writer(features_file, delimiter=' ', quotechar='"', quoting=csv.QUOTE_MINIMAL)
(!/usr/bin/env python has replaced the directory I am actually saving this CSV file in due to privacy reasons.)
I am trying to then place the file from this output into the following command:
x_new = pd.read_csv('filename %s.csv gps' , names = attributes)
I have tried a variety of ways to input the script1 output into this command, but can't find the correct way to do this. Please help me out. If any further information is needed please let me know.
This line is definitely wrong:
x_new = pd.read_csv('filename %s.csv gps' , names = attributes)
probably you mean:
x_new = pd.read_csv(filename , names = attributes)
Or
x_new = pd.read_csv('%s.csv' % gps , names = attributes)
but I don't think the filename part should be there either.
But don't write it to a file. You can just keep manipulating the data. And if you want to save the file anyway, you don't have to read the file, you can still just keep manipulating the array/frame you have.
also open(filename,"w") is all you need, if the file doesn't exist, it will be created.
You should try using formatted strings and putting variable directly into your file filepath (this assumes you're using Python 3+) :
x_new = pd.read_csv(f'{gps}.csv')
I need to do the following:
create a csv file if it does not exist, append data if it exists
when create a new csv file, created with heading from dict1.
My code:
def main():
list1 = [ 'DATE','DATASET','name1','name2','name3']
dict1 =dict.fromkeys(list1,0)
with open('masterResult.csv','w+b')as csvFile:
header = next(csv.reader(csvFile))
dict_writer = csv.DictWriter(csvFile,header,0)
dict_writer.writerow(dict1)
if __name__ =='__main__':
main()
I've written the below sample code which you can refer and use for your requirement. First of all, if you use, append mode for opening file, you can append if the file exists and newly write if it does not exist. Now, coming to your header writing, you can check the size of the file in prior. If the size is zero, then it is a new file obviously and you can write your header first. If the size is not zero, then you can append only data records without writing header. Below is my sample code. For the first time when you run it, it will create file with header. The next time you run the code, it will append only the data records and not the header.
import os
header='Name,Age'
filename='sample.csv'
filesize=0
if(os.path.exists(filename) and os.path.isfile(filename)):
filesize=os.stat(filename).st_size
f=open(filename,'a')
if(filesize == 0):
f.write('%s\n' % header)
f.write('%s\n' % 'name1,25')
f.close()
The w mode will overwrite an existing file. Instead, you need to use the a (append) mode:
with open('masterResult.csv','a+b') as csvFile:
# here -------------------^
I want to overwrite specific cells in an already existing excel file. I've searched and found this answer, writing to existing workbook using xlwt. I've applied it as the following,
def wrtite_to_excel (self):
#first I must open the specified excel file, sice open_file is in the same class, hence we can get it using self.sheet.
bookwt = copy(self.workbook)
sheetwt= bookwt.get_sheet(0)
#now, I must know the column that was estimated so I overwrite it,
colindex= self.columnBox.current() #returns the index of the estimated column
for i in range (1, self.grid.shape[0]):
if (str (self.sheet.cell_value(i,colindex)).lower () == self.missingBox.get().lower()):
#write the estimated value:
sheetwt.write (i, colindex, self.grid[i])
bookwt.save(self.filename + '.out' + os.path.splitext(self.filename)[-1])
Notice that, self.workbook already exists in another method in the same class this way,
def open_file (self, file_name):
try:
self.workbook = xlrd.open_workbook(file_name)
I really don't know what this means, '.out' + os.path.splitext(self.filename)[-1], but it seems that it causes the modified file to be saved in the same path of the original one with a different name.
After running the program, a new Excel file gets saved in the same path of the original one, however it is saved with a weird name as data.xlsx.out.xlsx and it doesn't open. I think it's caused by this line '.out' + os.path.splitext(self.filename)[-1]. I removed that line in order to overwrite the original file and not saving a copy, but when running the program I become unable to open the original file and I get an error message saying that the file can't be opened because the file format or extension is not valid.
What I really want is to modify the original file not to create a modified copy.
EDIT: SiHa's answer could modify the existing file without creating a copy if only the file name is specified like this,
bookwt.save(self.filename)
And, it could save a new copy this way,
filepath, fileext = os.path.splitext(self.filename)
bookwt.save(filepath + '_out' + fileext)
Or as the line provided in my code in the question. However, in all of these methods the same problem exists, where after modifying the file it can't be opened. After searching I found that the problem could be solved by changing the extension of the original file from .xlsx to .xls. After making this change, the problem was solve. This is the link where I found the solution http://www.computing.net/answers/office/the-file-formatfile-extension-is-not-valid/19454.html
Thank You.
To explain the line in question:
(self.filename + '.out' Means concatenate `.out' to the end of the original filename.
+ os.path.splitext(self.filename)[-1]) Means split the filename into a list of ['path', 'extension'] then concatenate the last element (the extension) back onto the end again.
So you end up with data.xlsx.out.xlsx
You should just be able to use bookwt.save(self.filename), although you may run in to errors if you still have the file open for reading. It may be safer to create a copy in a similar manner to the above:
filepath, fileext = os.path.splitext(self.filename)
bookwt.save(filepath + '_out' + fileext)
Which should give you data_out.xlsx
You can save excel file as CSV files this means that when they are open by python they show the values in plain text seperated by commas for example the spreadsheet with the address in the columns a to b and the rows 1 to 2 would look like this
A1,B1
A2,B2
this means that you can edit them like normal files and excel can still open them
If you are simply planning to write to a file using a python script as show below:
#!/usr/bin/python
count = 1
fo = open('DbCount.txt', 'w')
fo.write(str(count))
#fo.flush()
fo.close()
The Dbcount.txt file which was placed in the same folder as the script(attempting to modify the Dbcount.txt). i dont see any change in the txt file and no error is shown by the interpreter, its very strange, any help ?
first of all, always use the with statement variant, that will always close the file, even on errors:
#!/usr/bin/python
count = 1
with open('DbCount.txt', 'w') as fo:
fo.write(str(count))
then the 'w' overwrites your file each time you write to it. If you want to append, use 'a'.
About your specific problem, did you look only in the directory of your script, or in the current directory you're calling the script from? As you wrote your code, the file's path you write to is relative to where you execute your code from.
try:
import os
count = 1
with open(os.path.join(os.path.dirname(__file__), 'DbCount.txt'), 'w') as fo:
fo.write(str(count))
then it should output DbCount.txt in the same path as your script.
I have a list with .dbf files which I want to change to .csv files. By hand I open them in excel and re-save them as .csv, but this takes too much time.
Now I made a script which changes the file name, but when I open it, it is still a .dbf file type (although it is called .csv). How can I rename the files in such a way that the file type also changes?
My script uses (the dbf and csv file name are listed in a seperate csv file):
IN = dbffile name
OUT = csvfile name
for output_line in lstRename:
shutil.copyfile(IN,OUT)
Changing the name of a file (and the extension is just part of the complete name) has absolutely no effect on the contents of the file. You need to somehow convert the contents from one format to the other.
Using my dbf module and python it is quite simple:
import dbf
IN = 'some_file.dbf'
OUT = 'new_name.csv'
dbf.Table(IN).export(filename=OUT)
This will create a .csv file that is actually in csv format.
If you have ever used VB or looked into VBA, you can write a simple excel script to open each file, save it as csv and then save it with a new name.
Use the macro recorder to record you once doing it yourself and then edit the resulting script.
I have now created a application that automates this. Its called xlsto (look for the xls.exe release file). It allows you to pick a folder and convert all xls files to csv (or any other type).
You need a converter
Search for dbf2csv in google.
It depends what you want to do. It seems like you want to convert files to other types. There are many converters out there, but a computer alone doesn't know every file type. For that you will need to download some software. If all you want to do is change the file extension,
(ex. .png, .doc, .wav) then you can set your computer to be able to change both the name and the extension. I hoped I helped in some way :)
descargar libreria dbfpy desde http://sourceforge.net/projects/dbfpy/?source=dlp
import csv,glob
from dbfpy import dbf
entrada = raw_input(" entresucarpetadbf ::")
lisDbf = glob.glob(entrada + "\\*dbf")
for db in lisDbf:
print db
try:
dbfFile = dbf.Dbf(open(db,'r'))
csvFile = csv.writer(open(db[:-3] + "csv", 'wb'))
headers = range(len(dbfFile.fieldNames))
allRows = []
for row in dbfFile:
rows = []
for num in headers:
rows.append(row[num])
allRows.append(rows)
csvFile.writerow(dbfFile.fieldNames)
for row in allRows:
print row
csvFile.writerow(row)
except Exception,e:
print e
It might be that the new file name is "xzy.csv.dbf". Usually in C# I put quotes in the filename. This forces the OS to change the filename. Try something like "Xzy.csv" in quotes.