i have a code like this
import rarfile
pwd = None
rar = rarfile.RarFile(source_filename)
rar.extractall(dest_dir,None,pwd) # error from here
this code in working in ubuntu.
when i run this on windows i get error like this
Traceback (most recent call last):
File "1_bete_rar.pyw", line 132, in extract
File "1_bete_rar.pyw", line 176, in unrar_file
File "rarfile.pyc", line 586, in extractall
File "rarfile.pyc", line 1112, in _extract
File "rarfile.pyc", line 1704, in custom_popen
File "subprocess.pyc", line 711, in __init__
File "subprocess.pyc", line 948, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified
what is the problem with my code? how can i extract rar file with python in windows?
As the rarfile FAQ states (and as is made evident by traces of subprocess in the stack trace),
[rarfile] depends on unrar command-line utility to do the actual decompression.
Note that by default it expect it to be in PATH. If unrar launching fails, you need to fix this.
So get UnRAR from http://www.rarlab.com/rar_add.htm and put it somewhere in your PATH (such as the directory you're running your script from).
Looks like source_filename isn't pointing to a valid RAR file, do this little check before, just to be sure:
import os.path
os.path.isfile(source_filename) # what's the value returned?
If the file exists, then check if the path is in the correct format. For example, this won't work:
source_filename = 'c:\documents\file.rar'
Try this instead:
source_filename = 'c:\\documents\\file.rar'
Or even better, use raw strings:
source_filename = r'c:\documents\file.rar'
One common problem using python with Windows is that the path separator is \, but this is a special character that needs to be escaped in Python. If you print the source_filename you should be able to see if this is set correctly.
e.g.
source_filename = 'c:\users\prosserc\documents\myfile.txt'
will not work correctly. Here are a couple of alternatives:
Use a raw string:
source_filename = r'c:\users\prosserc\documents\myfile.txt'
or use os.path.join to join to an eviornment variable such as user_profile
source_filename = os.path.join(os.getenv('userprofile'), 'documents', 'myfile.txt')
Related
I'm trying to write a script that needs to rename (in the script itself, not in the folder) some .txt files to be able to use them in a loop, enumerating them.
I decided to use a dictionary, something like this:
import os
import fnmatch
dsc = {}
for filename in os.listdir('./texto'):
if fnmatch.fnmatch(filename, 'dsc_hydra*.txt'):
dsc[filename[:6]] = filename
print(dsc)
print(dsc['dsc_hydra1'])
The 'print(something)' are just to check if everything is going well.
I need to rename them because I'm using them in future functions and I don't want to address them using all that path stuff, something like:
IFOV = gi.IFOV_generic(gmatOUTsat1, matrixINPUTsat1, dsc['dsc_hydra1'], 'ifovfileMST.json', k_lim, height, width)
Using dsc['dsc_hydra1'], I get this error:
Traceback (most recent call last):
File "mainSMART_MST.py", line 429, in <module>
IFOV1= gi.IFOV_generic(gmatOUTsat1,matrixINPUTsat1,dsc['dsc_hydra1'],'ifovfileMST.jso',k_lim, height, width)
File "/home/alumno/Escritorio/HDD_Nuevo/HO(PY)/src/generateIFOV.py", line 49, in IFOV_generic
DCM11,DCM12,DCM13,DCM21,DCM22,DCM23,DCM31,DCM32,DCM33 = np.loadtxt(gmatDCM,unpack=True,skiprows = 2,dtype = float)
File "/home/alumno/.local/lib/python3.5/site-packages/numpy/lib/npyio.py", line 962, in loadtxt
fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
File "/home/alumno/.local/lib/python3.5/site-packages/numpy/lib/_datasource.py", line 266, in open
return ds.open(path, mode, encoding=encoding, newline=newline)
File "/home/alumno/.local/lib/python3.5/site-packages/numpy/lib/_datasource.py", line 624, in open
raise IOError("%s not found." % path)
OSError: dsc_hydra1.txt not found.
I've already checked the folder and the file is there, why do I keep getting this error?
I had this same issue. It cannot locate the .txt file because you're in the wrong directory. Make sure that where you're trying to execute the code is within the directories of which the code needs. Hope this helps.
I had the same problem. In my case, inside the file.txt, I had a space at the end of the string. You should control the spaces! For example, inside the file.txt (space = -):
-365-
string1-
string2
-string3
if you remove all the spaces (-) it should work!
I have a question to the zipfile library in python 2.7.12.
It seems that if I try to extract a .zip that is password protected, an exception is thrown.
I am able to extract a zipdirectory without password protection with python, and I have confirmed that i am able to extract it on my linux system without python and the right passphrase.
I will now post the unzipping process on my local console:
>>> import zipfile
>>> z = zipfile.ZipFile("folder.zip","r")
>>> z.extractall(pwd="taddel")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/zipfile.py", line 1040, in extractall
self.extract(zipinfo, path, pwd)
File "/usr/lib/python2.7/zipfile.py", line 1028, in extract
return self._extract_member(member, path, pwd)
File "/usr/lib/python2.7/zipfile.py", line 1082, in _extract_member
with self.open(member, pwd=pwd) as source, \
File "/usr/lib/python2.7/zipfile.py", line 1007, in open
raise RuntimeError("Bad password for file", name)
RuntimeError: ('Bad password for file', <zipfile.ZipInfo object at 0x7f31cd1c3370>)
>>>
I could give you the script I was developing at, but broken down it does nothing more than trying to dictonary-force the password and split everything into nice threads.
I know this IS the right password for this file, cause i made it myself.
I also already tried to .strip("\n") and to convert to ("utf-8").
I also know I do not have to specify a folder in which everything is extracted into, cause with a not password protected zip-archieve this code works perfectly fine.
How can I fix this? Or even, are there updated python libs available if the standart ones are not working?
I am trying to open a file that exists using python, and opens perfectly if I open it using gedit in command line.
However, I get the following error message:
andreas#ubuntu:~/Desktop/Thesis/Codes/ModifiedFiles$ python vis.py -f myoutputcsv.csv
Matplotlib version 1.3.1
Traceback (most recent call last):
File "vis.py", line 1082, in <module>
reliability_table = ReliabilityTable(reliability_table_file)
File "vis.py", line 112, in __init__
self.read(filename)
File "vis.py", line 139, in read
self.data = genfromtxt(filename, delimiter=',',comments='#', dtype=float)
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 1344, in genfromtxt
fhd = iter(np.lib._datasource.open(fname, 'rbU'))
File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 147, in open
return ds.open(path, mode)
File "/usr/lib/python2.7/dist-packages/numpy/lib/_datasource.py", line 496, in open
raise IOError("%s not found." % path)
IOError: ~/Desktop/Thesis/Codes/ModifiedFiles/reliability_table_2.csv not found.
Do you know what I may be doing wrong? I have very little experience with python and I cannot find the reason the file opens at command line but not using python.
The ~ (tilde) is a shell expansion, not a special "filesystem expansion".
So ~ expands to the current user directly only when found in a shell command:
$echo ~
/home/username
But not if used in the filename passed to python's file objects.
The python code:
open('some/file/name')
is equivalent to opening the file 'some/file/name' in the shell, and I mean literally with the single quotes that prevent expansions included.
So:
open('~/file.txt')
Tries to open:
$echo '~/file.txt'
~/file.txt
And not:
$echo ~/file.txt
/home/username/file.txt
This is stated at the top of the documentation of the os.path module too:
Unlike a unix shell, Python does not do any automatic path expansions.
Functions such as expanduser() and expandvars() can be invoked
explicitly when an application desires shell-like path expansion. (See
also the glob module.)
In fact you can create a file called ~:
$touch '~'
$ls | grep '~'
~
The single quotes are necessary, because touch ~ would simply execute touch on /home/username and no file would be created.
Now if you try to delete it you must escape its name, otherwise the shell will expand it into /home/username:
$echo ~
/home/username
$rm ~ # translation: "rm: cannot remove "/home/username": It's a directory"
rm: impossibile rimuovere "/home/username": È una directory
$rm '~' # proper way to delete it
If you want to expand the ~ in filenames use the os.path.expanduser function:
>>> import os.path
>>> os.path.expanduser('~/file.txt')
'/home/username/file.txt'
Note that realpath and abspath do not expand the ~:
>>> os.path.realpath('~/file.txt')
'/home/username/~/file.txt'
>>> os.path.abspath('~/file.txt')
'/home/username/~/file.txt'
So, if you want to be sure to convert a pathname given by the user in "shell language"1 into an absolute path usable with python's file objects you should do:
os.path.abspath(os.path.expanduser(path))
1 Not saying sh/bash because they are cross platform.
In a python script, I issue the command:
def copy_file(csv_file): #csv_file = "wpa-01.csv"
subprocess.call(["cp",csv_file,"tempfile.csv"])
I get the error:
cp: cannot stat 'wpa-01.csv' : No such file or directory
-tempfile.csv is a valid file, it is open
-I have tried adding quotes around wpa-01.csv, ie
subprocess.call(["cp","\"wpa-01.csv\"","tempfile.csv"])
-I have tried adding escape character in front of the '-'
-I have tried including the directory in front og the file name
-I am using gedit on a local Linux machine (so its not a dos2unix kind of solution), but the script is being ran on a remote Raspberry Pi
in every case I get the same error. I am at a loss for solutions. any suggestions?
***Here is the problem: "wpa-01.csv" is a 'live'/'dynamic' file. There is an active process that is updating that file in real time. I think that the file will have to be 'dead'/'static' in order to issue cp command? This is not ideal for my purposes. Is there a way to work around this like changing the mod or something? If not I suppose I can try to find an alternative solution.
print "wpa-01.csv" in os.listdir(".") #make sure file really does exist
subprocess.call(["cp","\"wpa-01.csv\"","tempfile.csv"],shell=True)
My guess is you need to set shell=True so that it uses your path to find cp executes in your shell ... if you don't use shell=True it wont use your path ...
Unfortunately all it is is a guess ...
Anyway, here is some supporting evidence:
>>> subprocess.call("copy tmp5.py tmp55.py")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python26\lib\subprocess.py", line 470, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python26\lib\subprocess.py", line 623, in __init__
errread, errwrite)
File "C:\Python26\lib\subprocess.py", line 833, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
>>> subprocess.call("copy tmp5.py tmp55.py",shell=True)
1 file(s) copied.
0
I have a python cgi script that runs an application via subprocess over and over again (several thousand times). I keep getting the same error...
Traceback (most recent call last):
File "/home/linuser/Webpages/cgi/SnpEdit.py", line 413, in <module>
webpage()
File "/home/linuser/Webpages/cgi/SnpEdit.py", line 406, in main
displayOmpResult(form['odfFile'].value)
File "/home/linuser/Webpages/cgi/SnpEdit.py", line 342, in displayContainerDiv
makeSection(position,sAoiInput)
File "/home/linuser/Webpages/cgi/SnpEdit.py", line 360, in displayData
displayTable(i,j,lAmpAndVars,dOligoSet[key],position)
File "/home/linuser/Webpages/cgi/SnpEdit.py", line 247, in displayTable
p = subprocess.Popen(['/usr/bin/pDat',sInputFileLoc,sOutputFileLoc],stdout=fh, stderr=fh)
File "/usr/lib/python2.6/subprocess.py", line 633, in __init__
errread, errwrite)
File "/usr/lib/python2.6/subprocess.py", line 1039, in _execute_child
errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files
The function causing it is below.
def displayTable(sData):
# convert the data to the proper format
sFormattedData = convertToFormat(sData)
# write the formatted data to file
sInputFile = tempfile.mkstemp(prefix='In_')[1]
fOpen = open(sInputFile,'w')
fOpen.write(sFormattedData)
fOpen.close()
sOutputFileLoc = sInputFile.replace('In_','Out_')
# run app, requires two files; an input and an output
# temp file to holds stdout stderr of subprocess
fh = tempfile.TemporaryFile(mode='w',dir=tempfile.gettempdir())
p = subprocess.Popen(['/usr/bin/pDat',sInputFileLoc,sOutputFileLoc],stdout=fh, stderr=fh)
p.communicate()
fh.close()
# open output file and print parsed data into a list of dictionaries
sOutput = open(sOutputFileLoc).read()
lOutputData = parseOutput(sOutput)
displayTableHeader(lOutputData)
displaySimpleTable(lOutputData)
As far as I can tell, I'm closing the files properly. When I run...
import resource
print resource.getrlimit(resource.RLIMIT_NOFILE)
I get...
(1024, 1024)
Do I have to increase this value? I read that subprocess opens several file descriptors. I tried adding "close_fds = True" and I tried using the with statement when creating my file but the result was the same. I suspect the problem may be with the application that I'm subprocessing, pDat, but this program was made by someone else. It requires two inputs; an input file and the location of where you want the output file written to. I suspect it may not be closing the output file that it creates. Aside from this, I can't see what I might be doing wrong. Any suggestions? Thanks.
EDIT:
I'm on ubuntu 10.04 running python 2.6.5 and apache 2.2.14
Instead of this...
sInputFile = tempfile.mkstemp(prefix='In_')[1]
fOpen = open(sInputFile,'w')
fOpen.write(sFormattedData)
fOpen.close()
I should have done this...
iFileHandle,sInputFile = tempfile.mkstemp(prefix='In_')
fOpen = open(sInputFile,'w')
fOpen.write(sFormattedData)
fOpen.close()
os.close(iFileHandle)
The mkstemp function makes OS level handles to a file and I wasn't closing them. The solution is described in more detail here...
http://www.logilab.org/blogentry/17873
You want to add close_fds=True to the popen call (just in case).
Then, here:
# open output file and print parsed data into a list of dictionaries
sOutput = open(sOutputFileLoc).read()
lOutputData = parseOutput(sOutput)
...I might remember wrong, but unless you use the with syntax, I do not think that the output file descriptor has been closed.
UPDATE: the main problem is that you need to know which files are open. On Windows this would require something like Process Explorer. In Linux it's a bit simpler; you just have to invoke the CGI from command line, or be sure that there is only one instance of the CGI running, and fetch its pid with ps command.
Once you have the pid, run a ls -la on the content of the /proc/<PID>/fd directory. All open file descriptors will be there, with the name of the files they point to. Knowing that file so-and-so is opened 377 times, that goes a long way towards finding out where exactly that file is opened (but not closed).