I get a strange error in python. When I try to extract a password protected file using the zip module, I get an exception when trying to set "oy" as password. Everything else seems to work. A bug in ZipFile module?
import zipfile
zip = zipfile.ZipFile("file.zip", "r")
zip.setpassword("oy".encode('utf-8'))
zip.extractall() #Above password "oy" generates the error here
zip.close()
This is the exception I get:
Traceback (most recent call last):
File "unzip.py", line 4, in <module>
zip.extractall()
File "C:\Program Files\Python32\lib\zipfile.py", line 1002, in extrac
l
self.extract(zipinfo, path, pwd)
File "C:\Program Files\Python32\lib\zipfile.py", line 990, in extract
return self._extract_member(member, path, pwd)
File "C:\Program Files\Python32\lib\zipfile.py", line 1035, in _extra
member
shutil.copyfileobj(source, target)
File "C:\Program Files\Python32\lib\shutil.py", line 65, in copyfileo
buf = fsrc.read(length)
File "C:\Program Files\Python32\lib\zipfile.py", line 581, in read
data = self.read1(n - len(buf))
File "C:\Program Files\Python32\lib\zipfile.py", line 633, in read1
max(n - len_readbuffer, self.MIN_READ_SIZE)
zlib.error: Error -3 while decompressing: invalid block type
If I use UTF-16 as encoding I get this error:
zlib.error: Error -3 while decompressing: invalid distance too far back
EDIT
I have now tested on a virtual Linux machine with following stuff:
Python version: 2.6.5
I created a password protected zip file with zip -e file.zip
hello.txt
Now it seems the problem is something else. Now I can extract the zip file even if the password is wrong!
try:
zip.setpassword("ks") # "ks" is wrong password but it still extracts the zip
zip.extractall()
except RuntimeException:
print "wrong!"
Sometimes I can extract the zip file with an incorrect password. The file (inside the zip file) is then extracted but when I try to open it the information seems to be corrupted/decrypted.
If there's a problem with the password, usually you get the following exception:
RuntimeError: ('Bad password for file', <zipfile.ZipInfo object at 0xb76dec2c>)
Since your exception complains about block type, most probably your .zip archive is corrupted, have you tried to unpack it with standalone unzip utility?
Or maybe you have used something funny, like 7zip to create it, which makes incompatible .zip archives.
You don't provide enough information (OS version? Python version? ZIP archive creator and contents? are there many files in those archives or single file in single archive? do all those files give same errors, or you can unpack some of them?), so here's quick Q&A section, which should help you to find and remedy the problem.
Q1. Is this a bug in Python?
A1. Unlikely.
Q2. What might cause this behaviour?
A2. Broken zip files, incompatible zip compressors -- since you don't tell anything, it's hard to point the the exact cause.
Q3. How to find the cause?
A3. Try to isolate the problem, find the file which gives you an error, try to use zip.testzip() and/or decompress that particular file with different unzip utility, share the results. Only you have access to the problematic files, so nobody can help you unless you try to do something yourself.
Q4. How to fix this?
A4. You cannot. Use different zip extractor, ZipFile won't work.
Try using the testzip() method to check the file's integrity before extracting files.
It could be possibly a bug in zipfile, or a bug in your zip implementation. I noted that your line numbers do not match mine so I guess this is python 3.2 earlier than the current 3.2.3 release I have.
Now, as to your code, it does work for me on Python 3.2.3 on Linux. I suggest you update to the latest 3.2.x as there seem to be a number of bug fixes related to zipfile and zlib, including fixes for crashes.
Related
Over the last few days, I have been working on automating the generation of some pivot tables for a number of reports.
Boiled down to the minimum, the following code was working without issue:
import win32com.client
objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
objExcelApp.Visible = 1
This would pop-up an instance of excel and I could continue working in Python. But suddenly, today my scripts are failing with the following:
>>>import win32com.client
>>> objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 534, in EnsureDispatch
mod = EnsureModule(tla[0], tla[1], tla[3], tla[4], bForDemand=bForDemand)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 391, in EnsureModule
module = GetModuleForTypelib(typelibCLSID, lcid, major, minor)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 266, in GetModuleForTypelib
AddModuleToCache(typelibCLSID, lcid, major, minor)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 552, in AddModuleToCache
dict = mod.CLSIDToClassMap
AttributeError: module 'win32com.gen_py.00020813-0000-0000-C000-000000000046x0x1x9' has no attribute 'CLSIDToClassMap'
The code has not changed from yesterday to today. I have no idea what is happening!!!.
Another interesting kicker. if I do the same code in the same session again I get a different error:
>>> objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 534, in EnsureDispatch
mod = EnsureModule(tla[0], tla[1], tla[3], tla[4], bForDemand=bForDemand)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 447, in EnsureModule
if module.MinorVersion != tlbAttributes[4] or genpy.makepy_version != module.makepy_version:
AttributeError: module 'win32com.gen_py.00020813-0000-0000-C000-000000000046x0x1x9' has no attribute 'MinorVersion'
>>>
So I jump over to a windows machine with a fresh windows install, install python37 and pip install pypiwin32. Run the very same lines and excel opens just like it did yesterday on my original machine.
I tried un-installing and re-installing with no luck. Any idea what is going on here?
NOTE:
Dynamic dispatch still works:
import win32com.client
objExcelApp = win32com.client.Dispatch("Excel.Application")
objExcelApp.Visible = 1
But I specifically need static dispatch as Pivot Tables won't work with a dynamically dispatched object (much later in my code):
objExcelPivotCache = objExcelWorkbook.PivotCaches().Create(SourceType=win32c.xlDatabase, SourceData=objExcelPivotSourceRange)
I had the same issue and I resolved it by following the instructions here: https://mail.python.org/pipermail/python-win32/2007-August/006147.html
Deleting the gen_py output directory and re-running makepy SUCCEEDS
and subsequently the test application runs OK again.
So the symptom is resolved, but any clues as to how this could have
happened. This is a VERY long running application (think 24x7 for
years) and I'm concerned that whatever caused this might occur again.
To find the output directory, run this in your python console / python session:
import win32com
print(win32com.__gen_path__)
Based on the exception message in your post, the directory you need to remove will be titled '00020813-0000-0000-C000-000000000046x0x1x9'. So delete this directory and re-run the code. And if you're nervous about deleting it (like I was) just cut the directory and paste it somewhere else.
💡Note that this directory is usually in your "TEMP" directory (copy-paste %TEMP%/gen_py in Windows File Explorer and you will arrive there directly).
I have no idea why this happens nor do I know how to prevent it from happening again, but the directions in the link I provided seemed to work for me.
A more straightforward solution was posted in a related question Issue in using win32com to access Excel file.
Basically, you just need to delete the folder C:\Users\<your username>\AppData\Local\Temp\gen_py and rerun your code.
💡TIP: You can also put in your Windows file explorer %TEMP%\gen_py to access it directly, and then delete its content.
Execute this command line in a powershell or cmd (NOT in Administrator mode => wouldn't work for me)
python -m win32com.client.makepy "Excel.Application"
It fixes all errors and you don't have to change your python code.
And keep using
win32com.client.gencache.EnsureDispatch("Excel.Application")
With gencache.EnsureDispatch you have access to the constants of the application loaded dynamically by makepy which must have the registered application (in our case Excel.Application).
If you have the same problem with Outlook, use "Outlook.Application" in above.
If still not working, reinstall pywin32 of your python distribution
<path to python root or venv>\pip.exe uninstall pywin32
<path to python root or venv>\pip.exe install pywin32
What has worked for me is:
excel = win32.gencache.EnsureDispatch('Excel.Application')
#change to =>
excel = win32.Dispatch('Excel.Application')
For me, it seems, the issue was that I have multiple processes that interact with Windows apps through win32com.
Since win32com creates the "gen_py" directory in win32api.GetTempPath() this can cause conflicts and the cache getting corrupted.
My solution is to set a custom location for "gen_py" for each process. A simple example:
from pathlib import Path
import win32com
gen_py_path = '/some/custom/location/gen_py'
Path(gen_py_path).mkdir(parents=True, exist_ok=True)
win32com.__gen_path__ = gen_py_path
# Any other imports/code that uses win32com
This way you don't have to delete the default "gen_py" folder and wonder what issues might arise. But if you still find you need to delete, you can just delete the custom folder and know you're deleting the cache just for that process.
To add to this discussion, for those receiving this error as part of an unsupervised automated process, you can fully automating the recovery process and allow any processes to continue unsupervised.
As answers by Ian and Qin mention, we need to delete the gen_py output directory and restart the process. As far as automation goes, there are two problems with this: the gen_py output directory is a temporary directory that can change, and currently running processes that rely on gen_py continue to fail even after it is regenerated.
To address these problems, we can dynamically look up the location, then we can completely nuke the whole process and restart it. I've tried deleting and re-importing win32com, but it seems a reference to the corrupted cache is still maintained, so the whole process needs to be restarted.
temp_data_dir = os.environ.get('LOCALAPPDATA'))
gen_py_dir = ''
for curr_path, dirs, files, in walk(temp_data_dir)
if 'gen_py' in dirs:
gen_py_dir = Path(curr_path).joinpath('gen_py')
shutil.rmtree(gen_py_dir)
execv(restart_args)
The restarted process should call win32com.client.gencache.EnsureDispatch('Excel.Application') again and a fresh copy of gen_py will now be utilized, then we can retry the failed Excel automation code.
The issue happens sporadically in a completely unreliable way that I can't replicate. My best guess is that the gen_py cache is sometimes corrupted somehow and just needs to be refreshed.
I want to learn the skill to log in websites with Python, so I copy other's code then try to understand them.
I've write several web crawls which don't need to post data to the websites by myself.
But this time, when I try to learn crawls that need post data, Python doesn't work.
It's the wrong message from Windows Command Line:
C:\Python26\lib\cookielib.py:1579: UserWarning: cookielib bug!
Traceback (most recent call last):
File "C:\Python26\lib\cookielib.py", line 1577, in make_cookies
parse_ns_headers(ns_hdrs), request)
File "C:\Python26\lib\cookielib.py", line 1534, in _cookies_from_attrs_set
cookie = self._cookie_from_cookie_tuple(tup, request)
File "C:\Python26\lib\cookielib.py", line 1527, in _cookie_from_cookie_tuple
rest)
File "C:\Python26\lib\cookielib.py", line 764, in __init__
self._rest = copy.copy(rest)
AttributeError: 'module' object has no attribute 'copy'
_warn_unhandled_exception()
Actually, I have tried to use Python2.7.6 and Python2.6 and refresh 2.7.6 for times.
The code that I copied is from http://www.oschina.net/code/snippet_1253232_26503.
This code is right because lots of people run this code .
I provide an account:
Email='13018015260'(Also can be telephone number in this site),password='qwerty'
The code is not the problem, but the Python itself. Once I use cookielib in code, Python on my computer doesn't work!
Sorry for my poor English, I want to know the answer, because I even can't find similar problems in Google!
Make sure that you do not have your own copy.py or copy.pyc in python path.
You can check that using following command in python interactive shell.
python -c "import copy; print(copy.__file__)"
It should print path something like:
c:\python27\lib\copy.pyc
If the above command print something that does not reference the file in standard library path, renamed the file (copy.py), and make sure you delete copy.pyc file.
I'm having this problem with ptyz:
>>> import pytz
>>> pytz.timezone('US/Mountain')
Traceback (most recent call last):
File "stdin", line 1, in module
File "/home/user/.local/lib/python2.6/site-packages/pytz-2013b-py2.6.egg/pytz/__init__.py", line 181, in timezone
_tzinfo_cache[zone] = build_tzinfo(zone, fp)
File "/home/user/.local/lib/python2.6/site-packages/pytz-2013b-py2.6.egg/pytz/tzfile.py", line 78, in build_tzinfo
while ttinfo[i][1]:
IndexError: list index out of range
I have two accounts in a server, and two different installations of pytz, though they are a copy of each other. In one of them, pytz works as a charm, but in the other one I have this error. It looks like an error inside ptyz, but it cannot be, because pytz works fine in the other account.
Has anyone handled this befor or have any idea of what may be causing this?
Update:
This error just happen when the script is called by a page, and that's the exact last message:
build_tzinfo(zone='US/Michigan', fp=<closed file '/home/user/.local/lib/python2...e-packages/pytz/zoneinfo/US/Michigan', mode 'rb'>)
and fp is a variable inside init.py that just receives it:
open(filename, 'rb')
It seems that the file is found, opened, but not really opened. I found at http://docs.python.org/2/tutorial/inputoutput.html that there are differences between linux and windows on the way they see binary files. My server runs Linux and, again, on the other account, with the same instalation, pytz works as a charm.
That's what happened:
After I updated my question, I noticed that the problem was with python opening a binary file. A friend told me to check the software I was using to send files to the server (Filezila), and there was three types of transfer: ASCII, Binary or Automatic. The software was set to Automatic. I send again the binary files in the binary transfer mode and it worked.
I'm trying to write a small Python script that will get query results from a database, write them to a file, and then sftp the file to a different server. The pieces work just fine but I'm getting a weird error when trying to sftp the file immediately after it's written.
The error I'm getting is
File "/usr/lib/python2.4/site-packages/paramiko/sftp_client.py", line 558, in put
file_size = os.stat(localpath).st_size
TypeError: coercing to Unicode: need string or buffer, file found
The offending line of code is just
sftp.put(outputfile, sftpoutputfile)
I tried using a copy of the output file instead of the one that's being written in the script and that worked exactly as it's supposed to. I'm calling file.close() after the file is written (and before setting up the sftp) so it seems like the file should be, well, closed and usable after that. Can someone tell me what I'm doing wrong? I can post more of the code if that would be helpful. Thank you very much.
The error message is telling you that it (in this case, os.stat) wants a stringlike object, and you're giving it the file instead.
Looking at the source of sftp_client.py in my copy of paramiko, we see
def put(self, localpath, remotepath, callback=None, confirm=True):
[...]
file_size = os.stat(localpath).st_size
fl = file(localpath, 'rb')
try:
fr = self.file(remotepath, 'wb')
fr.set_pipelined(True)
so I'm pretty sure that it wants the filename, not the file itself.
I'm working on a program to do some image wrangling in Python for work. I'm using FreeImagePy because PIL doesn't support multi-page TIFFs. Whenever I try to save a file with it from my program I get this error message (or something similar depending on which way I try to save):
Error returned. TIFF FreeImage_Save: failed to open file C:/OCRtmp/ocr page0
Traceback (most recent call last):
File "C:\Python25\Projects\OCRPageUnzipper\PageUnzipper.py", line 102, in <mod
ule> OCRBox.convertToPages("C:/OCRtmp/ocr page",FIPY.FIF_TIFF)
File "C:\Python25\lib\site-packages\FreeImagePy\FreeImagePy\FreeImagePy.py", l
ine 2080, in convertToPages self.Save(FIF, dib, fileNameOut, flags)
File "C:\Python25\lib\site-packages\FreeImagePy\FreeImagePy\FreeImagePy.py", l
ine 187, in Save return self.__lib.Save(typ, bitmap, fileName, flags)
WindowsError: exception: priviledged instruction
When I try and do the same things from IDLE, it works fine.
Looks like a permission issues, make sure you don't have the file open in another application, and that you have write permissions to the file location your trying to write to.
That's what I thought too, but I figured it out a couple hours ago. Apparently if the directory/file I'm trying to write to doesn't exist, FreeImagePy isn't smart enough to create it (most of the time. Creating a new multipage image seems to work fine) but i guess running it within IDLE, IDLE figures it out and takes care of it or something. I managed to work around it by using os.mkdir to explicitly make sure things that I need exist.