I'm attempting to grab an image attached to an email using Jython 2.5.3. I get the email (using they Jython version of the Python imap library). I can get the attachment by looping through the parts, finding the correct part type using get_content_type():
image, img_ext = None, None
for part in self.mail.get_payload():
part_type, part_ext = part.get_content_type().split('/')
part_type = part_type.lower().strip()
part_ext = part_ext.lower().strip()
if part_type == 'image':
image = part.get_payload(decode=True)
img_ext = part_ext
return image, img_ext
'image' is returned as a big block of bytes, which in regular Python I'd write out directly to a file. However when I try the same thing in Jython I get the following error:
TypeError: write(): 1st arg can't be coerced to java.nio.ByteBuffer[], java.nio.ByteBuffer
What's the right way to make Jython recognize my big blob of data as a byte array?
PS: the writing code uses tempfile.mkstmp(), which defaults to writing binary...
For future readers, here's how I got around it. In the code tha does the writing:
from org.python.core.util import StringUtil
from java.nio import ByteBuffer
tmp, filename = tempfile.mkstemp(suffix = "." + extension, text=True)
bytes = StringUtil().toBytes(attachment)
bb = ByteBuffer.wrap(bytes)
tmp.write(bb)
tmp.close()
Related
I have tried to upload an XML File to S3 using boto3. As recommended by Amazon, I would like to send a Base64 Encoded MD5-128 Bit Digest(Content-MD5) of the data.
https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.put
My Code:
with open(file, 'rb') as tempfile:
body = tempfile.read()
tempfile.close()
hash_object = hashlib.md5(body)
base64_md5 = base64.encodebytes(hash_object.digest())
response = s3.Object(self.bucket, self.key + file).put(
Body=body.decode(self.encoding),
ACL='private',
Metadata=metadata,
ContentType=self.content_type,
ContentEncoding=self.encoding,
ContentMD5=str(base64_md5)
)
When i try this the str(base64_md5) create a string like 'b'ZpL06Osuws3qFQJ8ktdBOw==\n''
In this case, I get this Error Message:
An error occurred (InvalidDigest) when calling the PutObject operation: The Content-MD5 you specified was invalid.
For Test purposes I copied only the Value without the 'b' in front: 'ZpL06Osuws3qFQJ8ktdBOw==\n'
Then i get this Error Message:
botocore.exceptions.HTTPClientError: An HTTP Client raised and unhandled exception: Invalid header value b'hvUe19qHj7rMbwOWVPEv6Q==\n'
Can anyone help me how to save Upload a File to S3?
Thanks,
Oliver
Starting with #Isaac Fife's example, stripping it down to identify what's required vs not, and to include imports and such to make it a full reproducible example:
(the only change you need to make is to use your own bucket name)
import base64
import hashlib
import boto3
contents = "hello world!"
md = hashlib.md5(contents.encode('utf-8')).digest()
contents_md5 = base64.b64encode(md).decode('utf-8')
boto3.client('s3').put_object(
Bucket="mybucket",
Key="test",
Body=contents,
ContentMD5=contents_md5
)
Learnings: first, the MD5 you are trying to generate will NOT look like what an 'upload' returns. We actually need a base64 version, it returns a md.hexdigest() version. hex is base16, which is not base64.
(Python 3.7)
Took me hours to figure this out because the only error you get is "The Content-MD5 you specified was invalid." Super useful for debugging... Anyway, here is the code I used to actually get the file to upload correctly before refactoring.
json_results = json_converter.convert_to_json(result)
json_results_utf8 = json_results.encode('utf-8')
content_md5 = md5.get_content_md5(json_results_utf8)
content_md5_string = content_md5.decode('utf-8')
metadata = {
"md5chksum": content_md5_string
}
s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
obj = s3.Object(bucket, 'filename.json')
obj.put(
Body=json_results_utf8,
ContentMD5=content_md5_string,
ServerSideEncryption='aws:kms',
Metadata=metadata,
SSEKMSKeyId=key_id)
and the hashing
def get_content_md5(data):
digest = hashlib.md5(data).digest()
return base64.b64encode(digest)
The hard part for me was figuring out what encoding you need at each step in the process and not being very familiar with how strings are stored in python at the time.
get_content_md5 takes a utf-8 bytes-like object only, and returns the same. But to pass the md5 hash to aws, it needs to be a string. You have to decode it before you give it to ContentMD5.
Pro-tip - Body on the other hand, needs to be given bytes or a seekable object. Make sure if you pass a seekable object that you seek(0) to the beginning of the file before you pass it to AWS or the MD5 will not match. For that reason, using bytes is less error prone, imo.
I need to use win32com.client to make an email where I add a signature with the .htm extension to the mail.HtmlBody. However, each time I do this, I get UnicodeDecodeError.
In other words, how do I correct the UnicodeDecodeError problem and add my string & htm file to the HtmlBody?
self.mail = win32.Dispatch('outlook.application').CreateItem(0)
self.curText = str(self.email.currentText())
self.projectNameT = ' '.join(self.curText.split(' ')[7:])
self.mail.To = 'ABC#XYZ.com'
self.mail.Subject = "Subject: " + str(self.projectNameT)
self.someStr = 'Hello '
self.html_url = open("SomePath//Signature.htm",encoding = 'utf16')
self.data = self.html_url.read()
self.mail.HtmlBody = self.someStr + ('<p>self.data</p>')
If you want to insert a signature in using python and fully programatically, Redemption exposes the RDOSignature object which implements ApplyTo method (it deals with signature image files and merges HTML styles). Because with the outlook security patch, a lot is unable to be done inherrently, so you must work around this before you can procede as normal
I'm working for the first time with IBM Watson Visual Recognition. My Python app needs to pass images that it's managing in memory to the service. However, the rather limited documentation and sample code I've been able to find from IBM shows calls to the API as referencing saved files. The file is passed to the call as an io.BufferedReader.
with open(car_path, 'rb') as images_file:
car_results = service.classify(
images_file=images_file,
threshold='0.1',
classifier_ids=['default']
).get_result()
My application will be working with images from memory and I don't want to have to save every image to file before I can make a call. I tried replacing the BufferedReader with an io.BytesIO stream, and I got back an error saying I was missing an images_filename param. When I added a mock filename (e.g. 'xyz123.jpg') I get back the following error:
TypeError: a bytes-like object is required, not 'float'
Can I make calls to the analysis API using an image from memory? If so, how?
EDIT:
This is essentially what I'm trying to do:
def analyze_image(pillow_img: PIL.Image):
byte_stream = io.BytesIO()
pillow_img.save(byte_stream, format='JPEG')
bytes_img = byte_stream.getvalue()
watson_vr = VisualRecognitionV3(
'2019-04-30',
url='https://gateway.watsonplatform.net/visual-recognition/api',
iam_apikey='<API KEY>'
)
result_json = watson_vr.classify(
images_file=bytes_img,
threshold=0.1,
classifier_ids=['default']
).get_result()
Thanks
How about
bytes_img = byte_stream.getbuffer()
...
result_json = watson_vr.classify(
images_file=bytes_img,
threshold=0.1,
classifier_ids=['default']
).get_result()
or
with byte_stream as images_file:
result_json = watson_vr.classify(
images_file=images_file,
threshold='0.1',
classifier_ids=['default']
).get_result()
I write a snippet using ctypes and tesseract 3.0.2 referring to the example:
import ctypes
from PIL import Image
libname = '/opt/tesseract/lib/libtesseract.so.3.0.2'
tesseract = ctypes.cdll.LoadLibrary(libname)
api = tesseract.TessBaseAPICreate()
rc = tesseract.TessBaseAPIInit3(api, "", 'eng')
filename = '/opt/ddl.ddl.exp654.png'
text_out = tesseract.TessBaseAPIProcessPages(api, filename, None, 0)
result_text = ctypes.string_at(text_out)
print result_text
It passes filename as a parameter, I have no idea to call which method in API to pass the raw data like:
tesseract.TessBaseAPIWhichMethod(api, open(filename).read())
I can't say for sure but I don't think you can pass complex python objects to that specific API, it won't know how to handle them. Your best bet would to be to look at a wrapper like http://code.google.com/p/python-tesseract/ which will allow you to use file buffers
import tesseract
api = tesseract.TessBaseAPI()
api.Init(".","eng",tesseract.OEM_DEFAULT)
api.SetVariable("tessedit_char_whitelist", "0123456789abcdefghijklmnopqrstuvwxyz")
api.SetPageSegMode(tesseract.PSM_AUTO)
mImgFile = "eurotext.jpg"
mBuffer=open(mImgFile,"rb").read()
result = tesseract.ProcessPagesBuffer(mBuffer,len(mBuffer),api) #YAY for buffers.
print "result(ProcessPagesBuffer)=",result
Edit
http://code.google.com/p/python-tesseract/source/browse/python-tesseract-0.7.4/debian/python-tesseract/usr/share/pyshared/tesseract.py might provide you with the insight that you need.
...
Acutally if you don't mind what happens when you replace
text_out = tesseract.TessBaseAPIProcessPages(api, filename, None, 0)
with
text_out = tesseract.ProcessPagesBuffer(mBuffer,len(mBuffer),api)
Im trying to read in image file from a server , with the code below . It keeps going into the exception. I know the correct number of bytes are being sent as I print them out when received. Im sending the image file from python like so
#open the image file and read it into an object
imgfile = open (marked_image, 'rb')
obj = imgfile.read()
#get the no of bytes in the image and convert it to a string
bytes = str(len(obj))
#send the number of bytes
self.conn.send( bytes + '\n')
if self.conn.sendall(obj) == None:
imgfile.flush()
imgfile.close()
print 'Image Sent'
else:
print 'Error'
Here is the android part , this is where I'm having the problem. Any suggestions on the best way to go about receiving the image and writing it to a file ?
//read the number of bytes in the image
String noOfBytes = in.readLine();
Toast.makeText(this, noOfBytes, 5).show();
byte bytes [] = new byte [Integer.parseInt(noOfBytes)];
//create a file to store the retrieved image
File photo = new File(Environment.getExternalStorageDirectory(), "PostKey.jpg");
DataInputStream dis = new DataInputStream(link.getInputStream());
try{
os =new FileOutputStream(photo);
byte buf[]=new byte[1024];
int len;
while((len=dis.read(buf))>0)
os.write(buf,0,len);
Toast.makeText(this, "File recieved", 5).show();
os.close();
dis.close();
}catch(IOException e){
Toast.makeText(this, "An IO Error Occured", 5).show();
}
EDIT: I still cant seem to get it working. I have been at it since and the result of all my efforts have either resulted in a file that is not the full size or else the app crashing. I know the file is not corrupt before sending server side. As far as I can tell its definitely sending too as the send all method in python sends all or throws an exception in the event of an error and so far it has never thrown an exception. So the client side is messed up . I have to send the file from the server so I cant use the suggestion suggested by Brian .
The best way to get a bitmap from a server is to execute the following.
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet("http://yoururl");
HttpResponse response = client.execute(get);
InputStream is = response.getEntity().getContent();
Bitmap image = BitmapFactory.decodeStream(is);
That will give you your bitmap, to save it to a file do something like the following.
FileOutputStream fos = new FileOutputStream("yourfilename");
image.compress(CompressFormat.PNG, 1, fos);
fos.close();
You can also combine the two and just write straight to disk
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet("http://yoururl");
HttpResponse response = client.execute(get);
InputStream is = response.getEntity().getContent();
FileOutputStream fos = new FileOutputStream("yourfilename");
byte[] buffer = new byte[256];
int read = is.read(buffer);
while(read != -1){
fos.write(buffer, 0, read);
read = is.read(buffer);
}
fos.close();
is.close();
Hope this helps;
I'm not sure I understand your code. You are calling dis.readFully(bytes); to write the content of dis into your byte array. But then you don't do anything with the array, and then try to write the content of dis through a buffer into your FileOutputStream.
Try commenting out the line dis.readFully(bytes);.
As a side note, I would write to the log rather than popping up a toast for things like the number of bytes or when an exception occurs:
...
} catch (IOException e) {
Log.e("MyTagName","Exception caught " + e.toString());
e.printStackTrace();
}
You could look at these links for examples of writing a file to the SD card:
Android download binary file problems
Android write to sd card folder
I solved it with the help of a Ubuntu Forums member. It was the reading of the bytes that was the problem . It was cutting some of the bytes from the image. The solution was to just send the image whole and remove the sending of the bytes from the equation all together