I am using Pandas on Mac, to read and write a CSV file, and the weird thing is when using full path, it has error and when using just a file name, it works. I post my code which works and which not works in my comments below, and also detailed error messages. Anyone have any good ideas?
sourceDf = pd.read_csv(path_to_csv)
sourceDf['nameFull'] = sourceDf['nameFirst'] + ' ' + sourceDf['nameLast']
sourceDf.to_csv('newMaster.csv') # working
sourceDf.to_csv('~/Downloads/newMaster.csv') # not working
Traceback (most recent call last):
File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 36, in <module>
add_full_name(path_to_csv, path_to_new_csv)
File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 28, in add_full_name
sourceDf.to_csv('~/Downloads/newMaster.csv')
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
formatter.save()
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
encoding=self.encoding)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/newMaster.csv'
Tried to use prefix r, but not working,
path_to_csv = r'~/Downloads/Master.csv'
path_to_new_csv = r'~/Downloads/Master_new.csv'
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
formatter.save()
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
encoding=self.encoding)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/Master_new.csv'
thanks in advance,
Lin
Try using os.path.join().
import os
(...)
output_filename = 'newMaster.csv'
output_path = os.path.join('Downloads', output_filename)
(...)
sourceDf.to_csv(output_path)
Use the same methodology to point pandas.read_csv() in the right direction.
You didn't specify python version.
On 3.4 you can use pathlib, otherwise use os.path.join() or quoting:
sourceDf.to_csv(r'~/Downloads/newMaster.csv')
Notice the r.
The problem is that /n is newline, which is not allowed in a path.
Related
I am encountering a problem when running with openpyxl the code below
import openpyxl
import os
wb = openpyxl.load_workbook('example.xlsx')
sheet = wb.get_sheet_by_name('Sheet1')
sheet["A1"].value
sheet["A1"].value == None
sheet["A1"].value = 42
sheet["A3"].value = 'Hello'
os.chdir("/Users/mac/Desktop")
wb.save('exceeeel.xlsx')
The error is
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/openpyxl/reader/excel.py", line 312, in load_wo
rkbook
reader = ExcelReader(filename, read_only, keep_vba,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/openpyxl/reader/excel.py", line 124, in __init_
_
self.archive = _validate_archive(fn)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/openpyxl/reader/excel.py", line 96, in _validat
e_archive
What am I doing wrong? I am using the current version of the openpyxl library.
I can't provide a confident answer because the question only includes a partial traceback. That being said, it looks like the traceback one would get for a FileNotFoundError:
C:\Python37\python.exe C:/Users/user/PycharmProjects/scratch/scratch2.py
Traceback (most recent call last):
File "C:/Users/user/PycharmProjects/scratch/scratch2.py", line 3, in <module>
wb = openpyxl.load_workbook('example.xlsx')
File "C:\Python37\lib\site-packages\openpyxl\reader\excel.py", line 313, in load_workbook
data_only, keep_links)
File "C:\Python37\lib\site-packages\openpyxl\reader\excel.py", line 124, in __init__
self.archive = _validate_archive(fn)
File "C:\Python37\lib\site-packages\openpyxl\reader\excel.py", line 96, in _validate_archive
archive = ZipFile(filename, 'r')
File "C:\Python37\lib\zipfile.py", line 1207, in __init__
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'example.xlsx'
Process finished with exit code 1
This error will be raised when the path your provided to the file you want to load with openpyxl.load_workbook does not contain the specified file. Since the only argument you provided in your call of that function is 'example.xlsx' that probably means there is no file in the folder you are running this script from.
If this 'example.xlsx' file is in a different folder then you'll want to either specify the relative path to that file as your argument or move the file into the same folder as your script.
If this isn't what's going on then you'll need to provide the full traceback that you are seeing on your end in order to get a better answer.
I m using mac and have added rsa key on desktop
The path I 'm using is
host_key = paramiko.RSAKey(filename='~/Desktop/test_rsa.key')
Error:
Traceback (most recent call last):
File "/Users/vidit/PycharmProjects/untitled6/server.py", line 7, in <module>
host_key = paramiko.RSAKey(filename='~/Desktop/test_rsa.key')
File "/Library/Python/2.7/site-packages/paramiko/rsakey.py", line 45, in __init__
self._from_private_key_file(filename, password)
File "/Library/Python/2.7/site-packages/paramiko/rsakey.py", line 163, in _from_private_key_file
data = self._read_private_key_file('RSA', filename, password)
File "/Library/Python/2.7/site-packages/paramiko/pkey.py", line 267, in _read_private_key_file
with open(filename, 'r') as f:
IOError: [Errno 2] No such file or directory: '~/Desktop/test_rsa.key'
You can not use ~ in the path of a file directly. It is a shell feature and expanded by the shell.
Use os.path.expanduser(path) to expand the ~ in the file path before using it.
You can't use ~ in Python paths. Try hardcoding your home directory and it will work.
You can use expanduser() if you want to use tilde (~)
from os.path import expanduser
keypath = expanduser("~/Desktop/test_rsa.key")
So I'm fairly new to python and i'm writing a script that needs to untar a file. I use this simple function.
def untar(source_filename, dest_dir):
for f in os.listdir():
print(f)
if(source_filename.endswith("tar.gz") or source_filename.endswith(".tar")):
tar = tarfile.open(source_filename)
tar.extractall(dest_dir)
tar.close()
else:
raise Exception("Could not retrieve .depends for that file.")
I added the initial for loop for debugging purposes. When I invoke it, it prints out the name of the file i need in the current working directory meaning that it does exist. Here is the whole output.
dep.tar.gz
Traceback (most recent call last):
File "init.py", line 70, in <module>
untar('dep.tar.gz', ".")
File "init.py", line 17, in untar
tar = tarfile.open(source_filename)
File "/usr/lib/python3.4/tarfile.py", line 1548, in open
return func(name, "r", fileobj, **kwargs)
File "/usr/lib/python3.4/tarfile.py", line 1646, in bz2open
compresslevel=compresslevel)
File "/usr/lib/python3.4/bz2.py", line 102, in __init__
self._fp = _builtin_open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'dep.tar.gz'
Can someone tell me how it can see the file in the working directory, and then suddenly not be able to see the file in the working directory?
The program I was using to create the tar placed a space at the beginning of the filename. So python was looking for 'dep.tar.gz' and the actual filename was ' dep.tar.gz'. ty #Ben
TIL - filenames can start with spaces.
I have a python script that analyses a file tree and records its findings in an xlsx.
Analysis is going fine, but when I try to record my results, I get an error:
Traceback (most recent call last):
File ".\call_validation.py", line 103, in <module>
wb.save(wb_name)
File "C:\Python\lib\site-packages\openpyxl\workbook\workbook.py", line 298, i
save_workbook(self, filename)
File "C:\Python\lib\site-packages\openpyxl\writer\excel.py", line 196, in sav
writer.save(filename, as_template=as_template)
File "C:\Python\lib\site-packages\openpyxl\writer\excel.py", line 178, in sav
archive = ZipFile(filename, 'w', ZIP_DEFLATED)
File "C:\Python\lib\zipfile.py", line 923, in __init__
self.fp = io.open(file, modeDict[mode])
OSError: [Errno 22] Invalid argument: 'move_generated-2015-05-07 10:08:26.xlsx'
I am generating my filename using datetime.datetime.now() like so:
save_time = str(datetime.datetime.now()).split(".")[0]
wb_name = "move_generated-" + save_time + ".xlsx"
wb.save(wb_name)
I don't believe the filename is too long, its only in C:\code\call_flow and I've tried stripping all the non-alphanumeric characters out of the name. Any ideas?
EDIT: Solution ended up being that I had failed to strip the colons from the time. As #nivix zixer said I fixed it by replacing
save_time = str(datetime.datetime.now()).split(".")[0]
with
save_time = str(datetime.datetime.now()).split(".")[0].replace(':', '_')
Perhaps the problem is you have a space in the filename?
Replace str(datetime.datetime.now()).split(".")[0] with this: str(datetime.datetime.now()).split(".")[0].replace(' ', '_').
Glad I could help Will!
Im trying to read a dataset and collect meta features from it.
I get the following error after executing the python file.
Traceback (most recent call last):
File "runmeta.py", line 79, in <module>
np.savetxt('datasets/'+str(i)+'/metafeatures',meta[i],delimiter=',')
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 940, in savetxt
fh = open(fname, 'w')
IOError: [Errno 2] No such file or directory: 'datasets/2/metafeatures'
the error you're getting is simply telling you it didn't find the file. i would suggest looking into absolute and relative file paths.
advice in error handling:
the error is triggered on this line
fh = open(fname, 'w')
so as you debug your program, look at the line python shows you. maybe change the variable fname. that is where i would start.
currently
fname = 'datasets/2/metafeatures'