We Keep Coding

AWS lambda put object multiple images at once

I am trying to resize a source image to multiple dimensions+extensions.
For example: when I upload a source image, say abc.jpg I need to resize it .jpg and .webp with different dimensions like abc_320.jpg, abc_320.webp, abc_640.jpg, abc_640.webp with s3 event trigger. So with my current python lambda handler I can do it with multiple put_object call to destination bucket but I want to make it more optimize as in future my dimension+extension may increase. So how can I store all the resized images to destination bucket with one call?
Current Lambda Handler:
import json
import boto3
import os
from os import path
from io import BytesIO
from PIL import Image
# boto3 S3 initialization
s3_client = boto3.client("s3")
def lambda_handler(event, context):
destination_bucket_name = 'destination-bucket'
# event contains all information about uploaded object
print("Event :", event)
# Bucket Name where file was uploaded
source_bucket_name = event['Records'][0]['s3']['bucket']['name']
# Filename of object (with path)
dest_bucket_perfix = 'resized'
file_key_name = event['Records'][0]['s3']['object']['key']
image_obj = s3_client.get_object(Bucket=source_bucket_name, Key=file_key_name)
image_obj = image_obj.get('Body').read()
img = Image.open(BytesIO(image_obj))
dimensions = [320, 640]
# Checking the extension and
img_extension = path.splitext(file_key_name)[1].lower()
extension_dict = {".jpg":"JPEG", ".png":"PNG", ".jpeg":"JPEG"}
extensions = ["WebP"]
if img_extension in extension_dict.keys():
extensions.append(extension_dict[img_extension])
print ("test-1")
for dimension in dimensions:
WIDTH = HEIGHT = dimension
for extension in extensions:
resized_img = img.resize((WIDTH, HEIGHT))
buffer = BytesIO()
resized_img.save(buffer, extension)
buffer.seek(0)
# I don't want to use this put_object in loop <<<---
s3_client.put_object(Bucket=destination_bucket_name, Key=file_key_name.replace("upload", dest_bucket_perfix, 1), Body=buffer)
return {
'statusCode': 200,
'body': json.dumps('Hello from S3 events Lambda!')
}
You can see I need to call put_object on every iteration of dimension+extension which is costly. I also thought about multi-threading and zipped solution but looking for others possible thoughts/solutions

Amazon S3 API calls only allow one object to be uploaded per call.
However, you could modify your program for multi-threading and upload the objects in parallel.

Django choose in what location images are loaded in model's save method

Here's my save method for model with field image = models.ImageField(upload_to='products/%Y/%m/%d', blank=True):
def save(self, *args, **kwargs):
if self.image:
self.image = compress(self.image)
if self.second_image:
self.second_image = compress(self.second_image)
if self.third_image:
self.third_image = compress(self.third_image)
if self.fourth_image:
self.fourth_image = compress(self.fourth_image)
super().save(*args, **kwargs)
It works fine and compresses all images but changes image's directory every time when I click save in django admin. It makes all images' paths be like:
before edited and saved: products/2020/11/05/img.jpeg
after: products/2020/11/05/products/2020/11/05/img.jpeg
click save one more time: products/2020/11/05/products/2020/11/05/products/2020/11/05/img.jpeg
And then I get this error:
SuspiciousFileOperation at /admin/shop/product/6/change/
Storage can not find an available filename for "products\2020\11\05\products\2020\11\05\products\2020\...... .jpeg".
Please make sure that the corresponding file field allows sufficient "max_length".
How can I fix this problem? I think I need to choose location where saved images would be stored. Django doesn't let me use absolute path in upload_to field so I have no idea.
compress func is:
from io import BytesIO
from PIL import Image
from django.core.files import File
def compress(image):
im = Image.open(image)
if im.mode != 'RGB':
im = im.convert('RGB')
im_io = BytesIO()
im.save(im_io, 'JPEG', quality=70)
compressed_image = File(im_io, name=image.name)
return compressed_image

The problem is that the filename is updated everytime. At each step, you should make sure that the filename is only a filename. Something like this:
import os
if self.image:
self.image = compress(self.image)
self.image.name = os.path.basename(self.image.name)
I don't know exactly what is your compress function, but maybe you could also check that it doesn't do something weird with the filename.

Is there a way to rename images in restAPI local server using python?

i created a localhost api to analyis images and compare them (computer vision) project!
my plan is to upload images from my data folder to the server, each image file in folder is named (fake_name.jpg/jpeg) i am trying to add the file name as a person name in parameters but can only do it manually and for each file.
i am also trying to figure out how to upload multiple files.
def image_to_base64(self, img):
# convert image to base64
prependInfo = 'data:image/jpeg;base64,'
encodedString = base64.b64encode(img).decode("utf-8")
fullString = str(prependInfo) + encodedString
return str(fullString)
# the following part is to create entry in database:
def create_person_entry(self,img):
base_url = "localhost:8080/service/api/person/create?"
parameters = {
"person-name": 'homer simson' #manual change name from here before each upload
}
data = {
"image-data": self.image_to_base64(img)
}
r = requests.post(base_url+urllib.parse.urlencode(parameters),headers{'Authorization':self.auth_tok}, data=data).json()
return r
#to import 1 image i used:
#with open("///data/homer simson.jpg", "rb") as img:
person_name = cvis.create_person(img.read())
print (person_name)
it uploads successfuly but i have to manualy name the person entry from parameters "person-name" for each person i upload! researched everywhere to automate solution!
edit1:
i managed to get this code working and it worked
# to upload multiple images
#folder with JPEG/JPG files to upload
folder = "/home///data/"
#dict for files
upload_list = []
for files in os.listdir(folder): with open("{folder}{name}".format(folder=folder, name=files), "rb") as data:
upload_list.append(files)
person_name = cvis.create_person(data.read())
print (person_name)
i managed to upload all images from directory to server it worked but now all my files are named homer simpson :)
i finally managed to get this right at the suggestion made by AKX his solution is below plz upvote, thanks
Now i need to figure out how to delete the previous no name entries.. will check API documentation.

Am I missing something – why not just add another argument to your create_person_entry() function?
def create_person_entry(self, name, img):
parameters = {
"person-name": name,
}
# ...
return r
# ...
cvis.create_person_entry("homer_simpson", img.read())
And if you have a folderful of images,
import os
import glob
for filename in glob.glob("people/*.jpg"):
file_basename = os.path.splitext(os.path.basename(filename))[0]
with open(filename, "rb") as img:
cvis.create_person_entry(file_basename, img.read())
will use the file's name sans extension, e.g. people/homer_simpson.jpg is homer_simpson.

Converting base64 to .jpg file, then saving in Django database

def upload_image(request):
if request.is_ajax and request.POST:
image = request.POST.get('image')
image_name = request.POST.get('image_name')
imgdata = base64.b64decode(image + '==')
extension = image_name.split('.')[1].lower()
image_name = '{}_{}_profile_image.{}'.format(request.user.first_name, request.user.last_name, extension)
with open(image_name, "wb") as image_file:
image_file.write(imgdata)
upload = ProfileImage(
file=image_file,
user = request.user.username
)
upload.save()
data = {
}
return JsonResponse(data)
I am trying to crop images in Django using Croppie.js. The images are then uploaded to an S3 bucket.
I have the cropping working and it is returning the image cropped as a base64 string. I decoded it and write it to a new image file so that it could be then saved in the database.
When it it gets to upload.save() I am getting the error.
AttributeError: '_io.BufferedWriter' object has no attribute '_committed'
I'm not sure what the problem is. This is my first time working with base64 images and im not sure if im missing something when i'm converting back to a file or what is going on.

I was able to find a solution by using ContentFile
from django.core.files.base import ContentFile
def upload_image(request):
if request.is_ajax and request.POST:
image = request.POST.get('image')
image_name = request.POST.get('image_name')
extension = image_name.split('.')[1].lower()
image_name = '{}_{}_profile_image.{}'.format(request.user.first_name, request.user.last_name, extension)
imgStr = image.split(';base64')
data = ContentFile(base64.b64decode(imgStr[1]), name=image_name)
upload = Upload(
file=data,
user = request.user.username
)
# Saves upload to S3 bucket
upload.save()
data = {
}
return JsonResponse(data)
It converts the base64 string to a file that is readable by django.

PIL - saving image as .jpg not working

I'm trying to overwrite save() method of the model to resize images. Every format works, except when saving a .jpg image. It's not saving images with .jpg extension.
I read the Pillow documentation and there's no JPG format.
class Business(models.Model):
photo = models.ImageField(_('photo'), storage=OverwriteStorage(),
upload_to=image_upload_to, blank=True, null=True)
def save(self, **kwargs):
"""
Changing dimensions of images if they are to big.
Set max height or width to 800px depending on the image is portrait or landscape.
"""
# Opening the uploaded image
im = Image.open(self.photo)
print(im)
output = BytesIO()
# set the max width or height
im.thumbnail((800, 800))
# find the ext of the file
ext = self.photo.name.split('.')[1].upper()
if ext in {'JPEG', 'PNG', 'GIF', 'TIFF'}:
# after modifications, save it to the output
im.save(output, format=ext, quality=100)
output.seek(0)
# change the imagefield value to be the newley modifed image value
self.photo = InMemoryUploadedFile(output, 'ImageField', "%s.jpg" % self.photo.name.split('.')[0],
'image/jpeg', sys.getsizeof(output), None)
super(User, self).save()
I don't know what I am missing here.
And what's the best way to do this on a custom User model. Using a signal, overwriting ImageField or ...
Any help is appreciated :)

You handled extension JPEG, but not JPG.
You may handle it simply with something like that before your if:
if ext == 'JPG':
ext = 'JPEG'

Important: You can not save the files as a .jpg
You must use another extension such as .jpeg
Example:
You have to use the Image object you import
from PIL import Image
def image_ex():
imagefile = 'images/original-image.jpg'
new_name = os.path.splitext('{}'.format(filename))[0]+'.jpeg'
Image.open(imagefile).rotate(270).filter(ImageFilter.DETAIL).save(new_name)
image_ex()
That works. Just remember dont save as a .jpg file extensions.