I'm trying to inflate a zlib compressed file using Python with this code:
import zlib
data = open("3B42.110531.21.6A.HDF.Z", 'rb').read()
inflated = zlib.decompress(data)
f = open('3B42.110531.21.6A.HDF', 'wb')
f.write(inflated)
f.close()
I've already done several attempts with different options:
Adding a second parameter to zlib.decompress (zlib.decompress(data,-15))
Skipping the first two bytes zlib.decompress(data[2:-4]) / zlib.decompress(data[2:] /.. )
Basecoding to 64 bits.
Anyway, I keep failing with this message:
Traceback (most recent call last):
File "C:\opt\Python25\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 325, in RunScript
exec codeObject in __main__.__dict__
File "E:\Tesis\data\uncompress.py", line 6, in <module>
inflated = zlib.decompress(data)
error: Error -3 while decompressing data: incorrect header check
The only difference is using a negative parameter in zlib.decompress: invalid block type.
import zlib
data = open("3B42.110531.21.6A.HDF.Z", 'rb').read()
inflated = zlib.decompress(data,-15)
f = open('3B42.110531.21.6A.HDF', 'wb')
f.write(inflated)
f.close()
Traceback (most recent call last):
File "C:\opt\Python25\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 325, in RunScript
exec codeObject in __main__.__dict__
File "E:\Tesis\data\uncompress.py", line 6, in <module>
inflated = zlib.decompress(data,-15)
error: Error -3 while decompressing data: invalid block type
I'm sure that the file is not corrupted, I can open it from WinRAR.
(environment: Windows x64, Python 2.5, I guess that the file is in a Unix machine..binary downloaded)
I've already read the following links
zlib-decompression-in-python
python-inflate and deflate-implementation
.Z indicates a LZC/compress file. Despite the name similarity, this compression format differs from gzip, which is what zlib implements.
Try using the command-line compress utility to uncompress the file (Your gzip program may also be able to decompress it).
The file extension '.Z' and the attempts you tried so far sound like you either use zLib wrong (but it seems correct according to your posted links) or the zLib stream isn't right at the beginning of the file.
You can use my tool Precomp with the file to detect the position of zLib stream(s) inside the file:
precomp -v -slow 3B42.110531.21.6A.HDF.Z
It should output something like this:
Possible zLib-Stream (slow mode) found at position 85, windowbits = 15
Can be decompressed to 9264 bytes
This will tell you both the position of the stream and the windowbits parameter to use (negated).
It will also tell you if there are zLib streams inside the file at all because as phihag said, it's possible that the file is compressed with something different than deflate/zLib. Note that in this case, there'll probably be some misdetections as the zLib header is only 2 bytes in size, but those can be identified by decompressing to <100 bytes.
Related
path of file is:
"C:\Users\deana\OneDrive\Marlon's files\Programming\Python\PITT\PITT_LIbrary\Lists\test.txt"
lines of code are:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt'))
the error is this:
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
exec(open('test.txt'))
TypeError: exec() arg 1 must be a string, bytes or code object
also if I try on one line as such:
exec(open(r"C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists/test.txt"))
i'ts the same error. (with and without r)
super frustrationg as it reads like i'm not inputting string... but it is string!?!
also I've done this litteraly the same way before, restarted IDLE shell, no difference.
ugh! I always get stupid errors with file paths.
I should have been using os.startfile() to open this.
It was confusing by using .open(). as I was attempting to open in default app.
before, i've used exec.open() to open .py files and guess I got them confused.
exec is just used to open other scripts... need stronger coffee next time.
Try this:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt', 'rb'))
You can convert the txt file to bytes by opening it with rb (read bytes).
I am trying to use Python 3.7.2 with PyPDF2 1.26 to select some pages of an input PDF file and write the output to stdout (the actual code is more complicated, this is just a MCVE):
import sys
from PyPDF2 import PdfFileReader, PdfFileWriter
input = PdfFileReader("example.pdf")
output = PdfFileWriter()
output.addPage(input.getPage(0))
output.write(sys.stdout)
This fails with the following error:
UserWarning: File <<stdout>> to write to is not in binary mode. It may not be written to correctly. [pdf.py:453]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.7/site-packages/PyPDF2/pdf.py", line 487, in write
stream.write(self._header + b_("\n"))
TypeError: write() argument must be str, not bytes
The problem seems to be that sys.stdout is not open in binary mode. As some of the answers suggest, I have tried the following:
output.write(sys.stdout.buffer)
This fails with the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.7/site-packages/PyPDF2/pdf.py", line 491, in write
object_positions.append(stream.tell())
OSError: [Errno 29] Illegal seek
I have also tried the answer from Changing the way stdin/stdout is opened in Python 3:
sout = open(sys.stdout.fileno(), "wb")
output.write(sout)
This fails with the same error as above.
How can I use the PyPDF2 library to output a PDF to standard output?
More generally, how do I correctly switch sys.stdout to binary mode (akin to Perl's binmode STDOUT)?
Note: There is no need to tell me that I can open a file in binary mode and write the PDF to that file. That works; however, I specifically want to write the PDF to stdout.
From the documentation:
write(stream)
Writes the collection of pages added to this object out as a PDF file.
Parameters: stream – An object to write the file to. The object must support the write method and the tell method, similar to a file object.
It turns out that sys.stdout.buffer is not tellable if not redirected to a file, hence you can't use it as a stream for PdfFileWriter.write.
Say your script is called myscript. If you call just myscript, then you'll get this error, but if you use it with a redirection, as in:
myscript > myfile.pdf
then Python understands it's a seekable stream, and you won't get the error.
I am trying to figure out the basics of importing DICOM files in Python using pydicom. While trying really simple code, I get following errors:
For code:
import dicom
filePath="C:\Python34\Lib\site-packages\dicom\testfiles"
ds=dicom.read_file(filePath[0])
I get error:
C:\Python34\python.exe C:/Users/041213/PycharmProjects/D/Deki.py
Traceback (most recent call last):
File "C:/Users/041213/PycharmProjects/D/Deki.py", line 4, in
ds=dicom.read_file(filePath[0])
File "C:\Python34\lib\site-packages\dicom\filereader.py", line 589, in read_file
fp = open(fp, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'C'
I am using Python 3.4, pydicom 0.9.9 and JetBrains PyCharm Community Edition 2016.3.2
If anyone can help me with this, or even just help me how to load a DICOM file in general, I would appreciate it a lot.
you're passing the first char of the string (C) instead of the full string. Just do:
ds=dicom.read_file(filePath)
next error you'll stumble into: use raw prefix or \t gets interpreted as a tabulation character:
filePath=r"C:\Python34\Lib\site-packages\dicom\testfiles"
^
Use and \ escape character to avoid issues with tab and other special characters. Also remember when you do filePath[0] on a string it returns the first character
filePath="C:\\Python34\\Lib\\site-packages\\dicom\\testfiles"
ds=dicom.read_file(filePath)
I'm in the process of writing a python module to POST files to a server , I can upload files of size of upto 500MB but when I tried to upload a 1gb file the upload failed, If I were to use something like cURL it won't fail. I got the code after googling how to upload multipart formdata using python , the code can be found here. I just compiled and ran that code , the error I'm getting is this
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
opener.open("http://127.0.0.1/test_server/upload",params)
File "C:\Python27\lib\urllib2.py", line 392, in open
req = meth(req)
File "C:\Python27\MultipartPostHandler.py", line 35, in http_request
boundary, data = self.multipart_encode(v_vars, v_files)
File "C:\Python27\MultipartPostHandler.py", line 63, in multipart_encode
buffer += '\r\n' + fd.read() + '\r\n'
MemoryError
I'm new to python and having a hard time grasping it. I also came across another program here , I'll be honest I don't know how to run it. I tried running it by guessing based on the function name , but that didn't work.
The script in question isn't very smart and builds the POST body in memory.
Thus, to POST a 1GB file, you'll need 1GB of memory just to hold that data, plus the HTTP headers, boundaries, and python and the code itself.
You'd have to rework the script to use mmap instead, where you first construct the whole body in a temp file before handing that file wrapped in a mmap.mmap value to passing it to request.add_data.
See Python: HTTP Post a large file with streaming for hints on how to achieve that.
I get a strange error in python. When I try to extract a password protected file using the zip module, I get an exception when trying to set "oy" as password. Everything else seems to work. A bug in ZipFile module?
import zipfile
zip = zipfile.ZipFile("file.zip", "r")
zip.setpassword("oy".encode('utf-8'))
zip.extractall() #Above password "oy" generates the error here
zip.close()
This is the exception I get:
Traceback (most recent call last):
File "unzip.py", line 4, in <module>
zip.extractall()
File "C:\Program Files\Python32\lib\zipfile.py", line 1002, in extrac
l
self.extract(zipinfo, path, pwd)
File "C:\Program Files\Python32\lib\zipfile.py", line 990, in extract
return self._extract_member(member, path, pwd)
File "C:\Program Files\Python32\lib\zipfile.py", line 1035, in _extra
member
shutil.copyfileobj(source, target)
File "C:\Program Files\Python32\lib\shutil.py", line 65, in copyfileo
buf = fsrc.read(length)
File "C:\Program Files\Python32\lib\zipfile.py", line 581, in read
data = self.read1(n - len(buf))
File "C:\Program Files\Python32\lib\zipfile.py", line 633, in read1
max(n - len_readbuffer, self.MIN_READ_SIZE)
zlib.error: Error -3 while decompressing: invalid block type
If I use UTF-16 as encoding I get this error:
zlib.error: Error -3 while decompressing: invalid distance too far back
EDIT
I have now tested on a virtual Linux machine with following stuff:
Python version: 2.6.5
I created a password protected zip file with zip -e file.zip
hello.txt
Now it seems the problem is something else. Now I can extract the zip file even if the password is wrong!
try:
zip.setpassword("ks") # "ks" is wrong password but it still extracts the zip
zip.extractall()
except RuntimeException:
print "wrong!"
Sometimes I can extract the zip file with an incorrect password. The file (inside the zip file) is then extracted but when I try to open it the information seems to be corrupted/decrypted.
If there's a problem with the password, usually you get the following exception:
RuntimeError: ('Bad password for file', <zipfile.ZipInfo object at 0xb76dec2c>)
Since your exception complains about block type, most probably your .zip archive is corrupted, have you tried to unpack it with standalone unzip utility?
Or maybe you have used something funny, like 7zip to create it, which makes incompatible .zip archives.
You don't provide enough information (OS version? Python version? ZIP archive creator and contents? are there many files in those archives or single file in single archive? do all those files give same errors, or you can unpack some of them?), so here's quick Q&A section, which should help you to find and remedy the problem.
Q1. Is this a bug in Python?
A1. Unlikely.
Q2. What might cause this behaviour?
A2. Broken zip files, incompatible zip compressors -- since you don't tell anything, it's hard to point the the exact cause.
Q3. How to find the cause?
A3. Try to isolate the problem, find the file which gives you an error, try to use zip.testzip() and/or decompress that particular file with different unzip utility, share the results. Only you have access to the problematic files, so nobody can help you unless you try to do something yourself.
Q4. How to fix this?
A4. You cannot. Use different zip extractor, ZipFile won't work.
Try using the testzip() method to check the file's integrity before extracting files.
It could be possibly a bug in zipfile, or a bug in your zip implementation. I noted that your line numbers do not match mine so I guess this is python 3.2 earlier than the current 3.2.3 release I have.
Now, as to your code, it does work for me on Python 3.2.3 on Linux. I suggest you update to the latest 3.2.x as there seem to be a number of bug fixes related to zipfile and zlib, including fixes for crashes.