HDF4Error: SD (59): HDF Internal error - python

I am trying to download a hdf file and read it python as follows
from pyhdf import SD
file = open("temp.hdf", 'w')
ftp.retrbinary('RETR '+ filename, file.write)
file.close()
hdf=SD.SD('temp.hdf')
It works but soon after the I am getting the following Error:
Traceback (most recent call last):
File "<ipython-input-46-55805a9d569b>", line 6, in <module>
hdf=SD.SD('temp.hdf')
File "/usr/local/lib/python2.7/dist-packages/pyhdf/SD.py", line 1444, in __init__
_checkErr('SD', id, "cannot open %s" % path)
File "/usr/local/lib/python2.7/dist-packages/pyhdf/error.py", line 23, in _checkErr
raise HDF4Error(err)
HDF4Error: SD (59): HDF Internal error

You need to open the output file in binary mode:
file = open("temp.hdf", 'wb') # was 'w'
Better would be to use with to automatically close the file:
with open("temp.hdf", 'wb') as out:
ftp.retrbinary('RETR '+ filename, out.write)

Related

cannot read simple txt file in python

I am a beginner. I have a simple txt file which I need to read (using numpy). I have the program in the same directory as the .txt file.
I have checked the cwd and it's the right one. Also, I've written a text file in order to see if python wants to open that one - that file opens just fine.
import os
import numpy as np
np.loadtxt("test2.txt")
The code above gives me the error.
The code below works just fine.
import os
import numpy as np
x = np.array([1, 2, 3])
np.savetxt("test.txt", x)
y = np.loadtxt("test.txt")
print(y)
The error I get is:
Traceback (most recent call last):
File "D:\detest\admi.py", line 5, in <module>
np.loadtxt("test2.txt")
File "C:\Users\Mircea\AppData\Roaming\Python\Python37\site-packages\numpy\lib\npyio.py", line 962, in loadtxt
fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
File "C:\Users\Mircea\AppData\Roaming\Python\Python37\site-packages\numpy\lib\_datasource.py", line 266, in open
return ds.open(path, mode, encoding=encoding, newline=newline)
File "C:\Users\Mircea\AppData\Roaming\Python\Python37\site-packages\numpy\lib\_datasource.py", line 624, in open
raise IOError("%s not found." % path)
OSError: test2.txt not found.
Can you use the Python read file instead?
path = '' # location of your file
openfile = open(path, 'r') # open file
openfile.read() # return all content of file
openfile.close() # close file

"[WinError 6] The handle is invalid" with Urllib

import urllib.request
def Download(url, file_name):
urllib.request.urlretrieve(url, file_name)
f = open("links.txt", "r")
lines = f.readlines()
for line in lines:
for x in range (1, 5):
filenaame = x
cut_string = line.split('?$')
new_string = cut_string[0]
numerator = new_string.split('/1/')
separator = ''
link = (separator.join(numerator[0] + "/{}/".format(x) + numerator[1]))
file_name = link.split('/{}/'.format(x))
file_name = file_name[1]
file_name = file_name.split('.')
file_name = (separator.join(file_name[0] + "{}".format(filenaame)))
filenaame =+ 1
print("Downloading: {}".format(file_name))
Download(link, filenaame)
Error:
Traceback (most recent call last):
File "C:\python\downloader\rr.py", line 29, in <module>
Download(link, filenaame)
File "C:\python\downloader\rr.py", line 5, in Download
urllib.request.urlretrieve(url, file_name)
File "C:\python\lib\urllib\request.py", line 258, in urlretrieve
tfp = open(filename, 'wb')
OSError: [WinError 6] The handle is invalid
I have googled a lot about this and in every result I found the person was using the subprocess module, which I'm not, which makes it even more difficult.
The code is for downloading images. It does download the first one successfully, then it crashes. Anyone know what's causing the error? I'm still a beginner.
you're passing your counter as a second parameter. In the end the function uses that as a filename to copy the downloaded data to.
By passing an integer you enable the "pass a lowlevel file descriptor" to open:
file is either a string or bytes object giving the pathname (absolute or relative to the current working directory) of the file to be opened or an integer file descriptor of the file to be wrapped.
But the file descriptor doesn't exist (fortunately!)
Let's reproduce the issue here:
>>> open(4,"rb")
Traceback (most recent call last):
File "<string>", line 301, in runcode
File "<interactive input>", line 1, in <module>
OSError: [Errno 9] Bad file descriptor
the fix is obviously:
Download(link, file_name)
(I'd suggest that you rename your filenaame counter for something more meaningful, it avoids mistakes like that)
At this line,
urllib.request.urlretrieve(url, file_name)
here file_name should be a string so before calling Download(link, filenaame) make integer filenaame to string try.
Download(link, str(filenaame))
Sample working for downloading an image
def download_img(url):
name = random.randrange(100,1000)
full_name = str(name)+ ".jpg"
urllib.request.urlretrieve(url, full_name)
download_img("http://images.freeimages.com/images/small-previews/25d/eagle-1523807.jpg")

Dropbox API - trying to upload a file

I've just rebuilt my Raspberry Pi and hence installed the latest version of the Dropbox API and now my program doesn't work. I think this is due to point 1 in these breaking changes: https://github.com/dropbox/dropbox-sdk-python/releases/tag/v7.1.0. I'm sure this question from SO (Dropbox API v2 - trying to upload file with files_upload() - throws TypeError) solves my problem... but as a newbie, I can't figure out how to actually implement it - and anyway, I'm already using f.read()... can anyone help?
This is my code:
def DropboxUpload(file):
sourcefile = "/home/pi/Documents/iot_pm2/dropbox_transfer/" + filename
targetfile = "/" + filename
dbx = dropbox.Dropbox(cfg.dropboxtoken)
f = open(sourcefile, "r")
filecontents = f.read()
try:
dbx.files_upload(filecontents, targetfile, mode=dropbox.files.WriteMode.overwrite)
except dropbox.exceptions.ApiError as err:
print(err)
f.close()
And this is the error:
Traceback (most recent call last):
File "/home/pi/Documents/iot_pm2/dropbox_uploader.py", line 20, in <module>
DropboxUpload(filename)
File "/home/pi/Documents/iot_pm2/dropbox_uploader.py", line 12, in DropboxUpload
dbx.files_upload(filecontents, targetfile, mode=dropbox.files.WriteMode.overwrite)
File "/usr/local/lib/python3.5/dist-packages/dropbox/base.py", line 2125, in files_upload
f,
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 272, in request
timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 363, in request_json_string_with_retry
timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/dropbox/dropbox.py", line 407, in request_json_string
type(request_binary))
TypeError: expected request_binary as binary type, got <class 'str'>
Thanks in advance.
You need to supply bytes, but you're supplying str.
You can get bytes by changing the file mode to binary. I.e., instead of:
f = open(sourcefile, "r")
do:
f = open(sourcefile, "rb")

How to unpack mnist dataset?

I want to unpack a pkl file from a MNIST dataset. I have used this code to do that:
import cPickle, gzip, numpy
# Load the dataset
f = gzip.open(’mnist.pkl.gz’, ’rb’)
train_set, valid_set, test_set = cPickle.load(f)
f.close()
But I am getting this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/gzip.py", line 34, in open
return GzipFile(filename, mode, compresslevel)
File "/usr/lib/python2.7/gzip.py", line 94, in __init__
fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: 'mnist.pkl.gz'
The mnist.pkl.gz file is on the desktop. How to specify the path in the code?
Put /home/<username>/Desktop/ in front of mnist.pkl.gz or launch it with the terminal working directory being the Desktop.

Python, unpacking a .jar file, doesnt work

So, I'm trying to unzip a .jar file using this code:
It won't unzip, only 20 / 500 files, and no folders/pictures
The same thing happens when I enter a .zip file in filename.
Any one any suggestions?
import zipfile
zfilename = "PhotoVieuwer.jar"
if zipfile.is_zipfile(zfilename):
print "%s is a valid zip file" % zfilename
else:
print "%s is not a valid zip file" % zfilename
print '-'*40
zfile = zipfile.ZipFile( zfilename, "r" )
zfile.printdir()
print '-'*40
for info in zfile.infolist():
fname = info.filename
data = zfile.read(fname)
if fname.endswith(".txt"):
print "These are the contents of %s:" % fname
print data
filename = fname
fout = open(filename, "w")
fout.write(data)
fout.close()
print "New file created --> %s" % filename
print '-'*40
But, it doesn't work, it unzips maybe 10 out of 500 files
Can anyone help me on fixing this?
Already Thanks!
I tried adding, what Python told me, I got this:
Oops! Your edit couldn't be submitted because:
body is limited to 30000 characters; you entered 153562
and only the error is :
Traceback (most recent call last):
File "C:\Python27\uc\TeStINGGFDSqAEZ.py", line 26, in <module>
fout = open(filename, "w")
IOError: [Errno 2] No such file or directory: 'net/minecraft/client/ClientBrandRetriever.class'
The files that get unzipped:
amw.Class
amx.Class
amz.Class
ana.Class
ane.Class
anf.Class
ang.Class
ank.Class
anm.Class
ann.Class
ano.Class
anq.Class
anr.Class
anx.Class
any.Class
anz.Class
aob.Class
aoc.Class
aod.Class
aoe.Class
This traceback tells you what you need to know:
Traceback (most recent call last):
File "C:\Python27\uc\TeStINGGFDSqAEZ.py", line 26, in <module>
fout = open(filename, "w")
IOError: [Errno 2] No such file or directory: 'net/minecraft/client/ClientBrandRetriever.class'
The error message says that either the file ClientBrandRetriever.class doesn't exist or the directory net/minecraft/client does not exist. When a file is opened for writing Python creates it, so it can't be a problem that the file does not exist. It must be the case that the directory does not exist.
Consider that this works
>>> open('temp.txt', 'w')
<open file 'temp.txt', mode 'w' at 0x015FF0D0>
but this doesn't, giving nearly identical traceback to the one you are getting:
>>> open('bogus/temp.txt', 'w')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'bogus/temp.txt'
Creating the directory fixes it:
>>> os.makedirs('bogus')
>>> open('bogus/temp.txt', 'w')
<open file 'bogus/temp.txt', mode 'w' at 0x01625D30>
Just prior to opening the file you should check if the directory exists and create it if necessary.
So to solve your problem, replace this
fout = open(filename, 'w')
with this
head, tail = os.path.split(filename) # isolate directory name
if not os.path.exists(head): # see if it exists
os.makedirs(head) # if not, create it
fout = open(filename, 'w')
If python -mzipfile -e PhotoVieuwer.jar dest works then you could:
import zipfile
with zipfile.ZipFile("PhotoVieuwer.jar") as z:
z.extractall()

Categories