I am trying to read h5 file from AWS S3. I am getting the following errors using s3fs/boto3. Can you help? Thanks!
import s3fs
fs = s3fs.S3FileSystem(anon=False, key='key', secret='secret')
with fs.open('file', mode='rb') as f:
h5 = pd.read_hdf(f)
TypeError: expected str, bytes or os.PathLike object, not S3File
fs = s3fs.S3FileSystem(anon=False, key='key', secret='secret')
with fs.open('file', mode='rb') as f:
hf = h5py.File(f)
TypeError: expected str, bytes or os.PathLike object, not S3File
client = boto3.client('s3',aws_access_key_id='key',aws_secret_access_key='secret')
result = client.get_object(Bucket='bucket', Key='file')
with h5py.File(result['Body'], 'r') as f:
data = f
TypeError: expected str, bytes or os.PathLike object, not StreamingBody
Your h5py version should work, but you'll need h5py version 2.9. See "file-like objects" here: http://docs.h5py.org/en/stable/high/file.html.
Related
#bot.on_message(filters.command('song'))
def songs(_,message):
msg = message.text.replace(message.text.split(' ')[0], '')
videosSearch = VideosSearch(msg , limit = 1)
f = videosSearch.result()
nani = f['result']
for link in nani:
url = link['link']
video = pafy.new(url)
audiostreams = video.audiostreams
best = video.getbestaudio()
ad = best.download()
file = open(ad, 'wb')
bot.send_document(message.chat.id, file)
file.close()
i cant find the error please help
file = open(ad, 'wb')
TypeError: expected str, bytes or os.PathLike object, not NoneType
Since the other two replies aren't really getting the issue at hand, I'm going to chip in here.
On these Lines:
ad = best.download()
file = open(ad, 'wb')
bot.send_document(message.chat.id, file)
file.close()
You assign the result of best.download() to ad, which you want to send with Pyrogram. Since you get a TypeError on open() it is clear that open() didn't get the type it expected, namely a NoneType. You have to make sure that something is downloaded, and that the method returns a value you can use.
Maybe try the caveman approach and use print(ad) before trying to open() it.
In addition to all that: Pyrogram does not support sending a file representation (open()). You can open a file in bytes mode, read the bytes and use that with BytesIO, but when you have an actual file on your system, you can just use the path to the file: app.send_document(chat_id, "my_file.webm").
See the Documentation for app.send_document().
Replace this:
file = open(ad, 'wb')
With This:
file = open('ad', 'wb')
I am trying to write the contents read from S3 object to a file . I am getting syntax error while doing the same.
object =s3.get_object(Bucket=bucket_name, Key="toollib/{0}/{1}/stages/{0}.groovy".format(tool,platform))
print(object)
jenkinsfile = object['Body'].read()
print(jenkinsfile)
basepath = '/mnt/efs/{0}/{1}/{2}/'.format(orderid, platform, technology)
filename = basepath+fileName
print(filename)
#file1=open(filename, "a")
with open(filename, 'a') as file:
file.write(jenkinsfile)
Error : "errorMessage": "write() argument must be str, not bytes"
Opening the file in binary mode should do the trick:
with open(filename, 'ab') as file:
file.write(jenkinsfile)
def convertToBinaryData(filename):
# Convert digital data to binary format
with open(filename, 'rb') as file:
binaryData = file.read()
return binaryData
This is my function for converting an image to binary...
uploaded_file = request.files['file']
if uploaded_file.filename != '':
uploaded_file.save(uploaded_file.filename)
empPicture = convertToBinaryData(uploaded_file)
and this is the block of code where the uploaded file is received and saved, however, when it runs, I get this error...
with open(filename, 'rb') as file:
TypeError: expected str, bytes or os.PathLike object, not FileStorage
I'm pretty new to python and I've been stuck on this for a while, any help would be appreciated. Thanks in advance
while calling 'convertToBinaryData' you are passing 'uploaded_file' which is not a filename but and object.
You need to pass the filename (with correct path if saved in custom location) to your 'convertToBinaryData' funciton.
Something like this:
convertToBinaryData(uploaded_file.filename)
uploaded_file is not a filename, it's a Flask FileStorage object. You can read from this directly, you don't need to call open().
So just do:
empPicture = uploaded_file.read()
See Read file data without saving it in Flask
I would like to open and parse a JSON file, but I keep getting the following error:
TypeError: Object of type 'bytes' is not JSON serializable
Here is my code,
class FileStore(object):
def __init__(self, filePath, data = None):
self.filePath = filePath
self.data = data
def store_json(self):
with open(self.filePath, 'w') as outfile:
json.dump(self.data, outfile) # this is where the error occurs
Could please help me fix this issue?
Try this:
json.dump(self.data.decode(), outfile)
If your data is valid but only Bytes this will fix your problem.
I am receiving the
TypeError: a bytes-like object is required, not str
but my str-object is bytes and have searched for an answer.
This is where the decoding happens without error:
for line in tmpFile:
sl = codecs.decode(bytes(line, encoding = 'utf-8'), encoding = 'utf-8').strip().split('\t')
writer.writerow(sl)
If I take out the (bytes(line,encofing = 'utf-8') part then I get the TypeError.
This is how I'm opening the file:
with open(output_path, mode = 'w') as out_file:
write_final_output(out_file, delimiter, tmpFile)
I've tried opening it with mode = 'w+' and with codecs and I still have the same problem which is that my csv file comes out encoded with the beginning reading like this:
b'101361\x01