We Keep Coding

Saving an object to CSV directly to S3

I have a file-like object received through HTTP response, I want to directly save it as CSV in S3. I tried using the S3 bucket and its path but I am facing an error, as no such file was found
Can someone help me with the code here- r5 is the URL response, rr= r5.raw which has the response data object. This object has to be saved as CSV into S3 directly.

Docs with example of sending existing file: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html
If you have a file content in a string, use BytesIO to make it a file-like object:
s3 = boto3.resource("s3")
bucket = s3.Bucket(bucket_name)
stream = io.BytesIO(my_str_data.encode()) # boto3 only allows to bytes data, so we need to encode
filename = "filename_here"
bucket.upload_fileobj(stream, filename)
Since you said you already have a file-like object, make sure it's bytes file-like object and just put in in place of stream

TypeError: expected str, bytes or os.PathLike object, not FileStorage keeps popping up

def convertToBinaryData(filename):
# Convert digital data to binary format
with open(filename, 'rb') as file:
binaryData = file.read()
return binaryData
This is my function for converting an image to binary...
uploaded_file = request.files['file']
if uploaded_file.filename != '':
uploaded_file.save(uploaded_file.filename)
empPicture = convertToBinaryData(uploaded_file)
and this is the block of code where the uploaded file is received and saved, however, when it runs, I get this error...
with open(filename, 'rb') as file:
TypeError: expected str, bytes or os.PathLike object, not FileStorage
I'm pretty new to python and I've been stuck on this for a while, any help would be appreciated. Thanks in advance

while calling 'convertToBinaryData' you are passing 'uploaded_file' which is not a filename but and object.
You need to pass the filename (with correct path if saved in custom location) to your 'convertToBinaryData' funciton.
Something like this:
convertToBinaryData(uploaded_file.filename)

uploaded_file is not a filename, it's a Flask FileStorage object. You can read from this directly, you don't need to call open().
So just do:
empPicture = uploaded_file.read()
See Read file data without saving it in Flask

Read h5 file using AWS S3 s3fs/boto3

I am trying to read h5 file from AWS S3. I am getting the following errors using s3fs/boto3. Can you help? Thanks!
import s3fs
fs = s3fs.S3FileSystem(anon=False, key='key', secret='secret')
with fs.open('file', mode='rb') as f:
h5 = pd.read_hdf(f)
TypeError: expected str, bytes or os.PathLike object, not S3File
fs = s3fs.S3FileSystem(anon=False, key='key', secret='secret')
with fs.open('file', mode='rb') as f:
hf = h5py.File(f)
TypeError: expected str, bytes or os.PathLike object, not S3File
client = boto3.client('s3',aws_access_key_id='key',aws_secret_access_key='secret')
result = client.get_object(Bucket='bucket', Key='file')
with h5py.File(result['Body'], 'r') as f:
data = f
TypeError: expected str, bytes or os.PathLike object, not StreamingBody

Your h5py version should work, but you'll need h5py version 2.9. See "file-like objects" here: http://docs.h5py.org/en/stable/high/file.html.

FTP Upload--ASCII mode--a bytes-like object is required, not 'str'

I'm trying to write a FTP upload program for text files. However I'm getting this error:
builtins.TypeError: a bytes-like object is required, not 'str'.
I am using Python 3.6.
Here is my code:
def _upload_to_ftp(self, ftp_handle, name):
# upload a single file to ftp directory
with open(name,'r') as f:
print("uploading"+name)
filename = os.path.basename(name)
ftp_handle.storlines('STOR %s' %filename, f)
I could not figure out why.

Unfortunately, what FTP calls text is still bytes for Python 3. Python 3 strings use Unicode characters that need to be encoded to bytes in order to be written to files and FTP deals with files. But here it is even simpler: you have just to open the local file in binary mode to have it deliver bytes instead of strings:
def _upload_to_ftp(self, ftp_handle, name):
# upload a single file to ftp directory
with open(name,'rb') as f: # use binary mode for file
print("uploading"+name)
filename = os.path.basename(name)
ftp_handle.storlines('STOR %s' %filename, f)

TypeError: a bytes-like object is required, not str but my str-object is bytes

I am receiving the
TypeError: a bytes-like object is required, not str
but my str-object is bytes and have searched for an answer.
This is where the decoding happens without error:
for line in tmpFile:
sl = codecs.decode(bytes(line, encoding = 'utf-8'), encoding = 'utf-8').strip().split('\t')
writer.writerow(sl)
If I take out the (bytes(line,encofing = 'utf-8') part then I get the TypeError.
This is how I'm opening the file:
with open(output_path, mode = 'w') as out_file:
write_final_output(out_file, delimiter, tmpFile)
I've tried opening it with mode = 'w+' and with codecs and I still have the same problem which is that my csv file comes out encoded with the beginning reading like this:
b'101361\x01