We are importing an screen capture from a web page direct into a variable in Python; and then producing a Numpy array using the following code :
To capture is a PNG image (note - the device url has an embedded cgi to do the capture work) :
response = requests.get(url.format(ip, device), auth=credentials)
Once screen is captured, covert to a Numpy array called image :
image = imread(BytesIO(response.content))
After analysis of image, we would like to FTP the captured PNG to a server for reference at a later date. The best solution we can find right now involves using imsave to create a file locally and then FTP with storbinary to take the local image and put it on the server.
Is it possible to FTP response.content; or a conversion of the numpy array back into a PNG (using imsave?) direct to the server and skip the local storage step?
Update
As per MattDMo comment, we tried:
def ftp_artifact (ftp_ip, ftp_dir, tid, artifact_name, artifact_path, imgdata) :
ftp = FTP(ftp_ip)
ftp.login("autoftp","autoftp")
ftp.mkd ("FTP/" + ftp_dir)
ftp.cwd("FTP/" + ftp_dir)
filepath = artifact_path
filename = artifact_name
f = BytesIO(imgdata)
ftp.storbinary ('STOR ' + filename, f)
ftp.quit()
Where imgdata is the result of io.imread. The result file is 5x bigger and not an image. The BytesIO object is the numpy array I presume?
In the ftplib module, the FTP.storbinary() method takes an open file object as its second argument. Since your BytesIO object can act as a file object, all you'd need to do is pass that - no need for a temporary file on the server.
EDIT
Without seeing your full code, what I suspect is happening is that you are passing the NumPy array to storbinary(), not the BytesIO object. You also need to make sure the object's read pointer is at the beginning by calling bytesio_object.seek(0) before uploading. The following code demonstrates how to do everything:
from ftplib import FTP
from io import BytesIO
import requests
r = requests.get("http://example.com/foo.png")
png = BytesIO(r.content)
# do image analysis
png.seek(0)
ftp = FTP("ftp.server.com")
ftp.login(user="username", passwd="password")
# change to desired upload directory
ftp.storbinary("STOR " + file_name, png)
try:
ftp.quit()
except:
ftp.close()
Took a bit of research but our student figured it out :
def ftp_image_to(ftp_ip, ftp_dir, filename, data):
ftp = FTP(ftp_ip)
print("logging in")
ftp.login('autoftp', 'autoftp')
print("making dir")
ftp.mkd('FTP/' + ftp_dir)
ftp.cwd('FTP/' + ftp_dir)
print("formatting image")
bytes = BytesIO()
plt.imsave(bytes, data, format='png')
bytes.seek(0)
print("storing binary")
ftp.storbinary('STOR ' + filename, bytes)
ftp.quit()
Thanks IH!
Related
I need to download a png file from a website and save the same in local directory .
The code is as below :
import pytesseract
from PIL import Image
from pathlib import Path
k = requests.get('https://somewebsite.com/somefile.png',stream =True)
Img=Image.open(k) # <----
Img.save("/new.png")
while executing it in JupyterNotebook
If I execute, i always get an error "response object has no attribute seek"
On the other hand , if I change the code to
Img= Image.open(k.raw), it works fine
I need to understand why it is so
You can save image data from a link using open() and write() functions:
import requests
URL = "https://images.unsplash.com/photo-1574169207511-e21a21c8075a?ixlib=rb-1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=880&q=80"
name = "IMG.jpg" #The name of the image once saved
Picture_request = requests.get(URL)
if Picture_request.status_code == 200:
with open(name, 'wb') as f:
f.write(Picture_request.content)
Per pillow the docs:
:param fp: A filename (string), pathlib.Path object or a file object.
The file object must implement file.read,
file.seek, and file.tell methods,
and be opened in binary mode.
response itself is just the response object. Using response.raw implements read, seek, and tell.
However, you should use response.content to get the raw bytes of the image. If you want to open it, then use io.BytesIO (quick explanation here).
import requests
from PIL import Image
from io import BytesIO
URL = "whatever"
name = "image.jpg"
response = requests.get(URL)
mybytes = BytesIO()
mybytes.write(response.content) # write the bytes into `mybytes`
mybytes.seek(0) # set pointer back to the beginning
img = Image.open(mybytes) # now pillow reads from this io and gets all the bytes we want
# do things to img
This question already has answers here:
Why can't I call read() twice on an open file?
(7 answers)
Closed 4 years ago.
I'm trying to store images on S3 from a Flask webserver. The server receives the image and processes it to create two copies (compressed + thumbnail), then uploads all three.
The two processes imaged are received fine, but the original gets corrupted. The code doesnt throw any errors.
This is all using Python 3.6, Flask 1.0.2, Boto3 1.9.88
Below is an excerpt from the code for the upload page:
form = UploadForm()
if form.validate_on_submit():
photo = form.photo.data
name, ext = os.path.splitext(form.photo.data.filename)
photo_comp, photo_thum = get_compressions(photo, 'JPEG')
pic_set = {}
pic_set['original'] = photo
pic_set['compressed'] = photo_comp
pic_set['thumbs'] = photo_thum
for pic in pic_set:
output = upload_to_s3(file=pic_set[pic], username=current_user.username, \
filetype = pic, \
bucket_name = current_app.config['S3_BUCKET'])
The function 'get_compressions()' produces a reduced size .jpeg of the file plus a thumbnail (apologies that the indenting formatting has come out wrong):
def get_compressions(file, filetype):
#Creates new compressed and thumbnail copies. Checks for alpha
#channel, removes if present, resaves as compressed .jpeg, then
#wraps into a Werkzeug FileStorage type.
name, ext = os.path.splitext(file.filename)
temp_compress = BytesIO()
temp_thumb = BytesIO()
image = Image.open(file)
if image.mode in ['RGBA', 'LA', 'RGBa']:
image2 = Image.new('RGB', image.size, '#ffffff')
image2.paste(image, None, image)
image = image2.copy()
image.save(temp_compress, format=filetype, quality=85, optimize=True)
image.thumbnail((400,400), Image.ANTIALIAS)
image.save(temp_thumb, format=filetype, optimize=True)
temp_thumb.seek(0)
temp_compress.seek(0)
file_comp = FileStorage(stream=temp_compress,
filename=name + '.' + filetype,
content_type='image/jpg',
name=file.name,
)
file_thum = FileStorage(stream=temp_thumb,
filename=name + '.' + filetype,
content_type='image/jpg',
name=file.name,
)
return file_comp, file_thum
Finally, the 'upload_to_s3()' function is a fairly straightforward save on an AWS S3:
def upload_to_s3(file, username, filetype, bucket_name, acl= os.environ.get('AWS_DEFAULT_ACL')):
s3.upload_fileobj(
Fileobj=file
, Bucket=bucket_name
, Key = "{x}/{y}/{z}".format(x=username,y=filetype,z=file.filename)
, ExtraArgs = {'ContentType': file.content_type}
)
print('Upload successful: ', file.filename)
return file.filename
My belief is that the compression is affecting the upload of the original file object - while the PIL image.save() returns a new object, the act of compressing appears to be affecting the original object somehow.
When trying to research this I noted that the Flask is multithreaded as standard, and that the Python GIL doesnt apply to I/O operations or image processing - not sure if this could be relevant.
Two options I'd tried to fix this was either:
Changing the code order execution so it goes original upload - compression - compressed upload, but this resulted in an error 'ValueError: I/O operation on a closed file'
Using copy.deepcopy() to make a new object prior to using get_compressions(), but this resulted in an 'TypeError: cannot serialize '_io.BufferedRandom' object'.
I'm not really sure how to proceed! Potentially could upload the original, then have the server process compression in the background (based on the uploaded file), but this this presents an issue for the client who wants to immediately retrieve the compressed version to load the page.
In your get_compressions function, you're reading the original file which is a FileStorage object, so your file pointer ends up at the end of the file and you end up writing a zero byte file to S3. So, you need to seek back to the start of the file, just like you've done for the compressed versions:
file.seek(0)
temp_thumb.seek(0)
temp_compress.seek(0)
I am trying to download multiple image files from the server. I am using Django for my backend.
Question related to single image has already been answered and I tried the code and it works on single image. In my application, I want to download multiple images in a single HTTP connection.
from PIL import Image
img = Image.open('test.jpg')
img2 = Image.open('test2.png')
response = HttpResponse(content_type = 'image/jpeg')
response2 = HttpResponse(content_type = 'image/png')
img.save(response, 'JPEG')
img2.save(response2, 'PNG')
return response #SINGLE
How can I fetch both img and img2 at once. One way I was thinking is to zip both images and unzip it on client size but I dont think that is good solution. Is there a way to handle this?
I looked around and find an older solution using a temporary Zip file on disk: https://djangosnippets.org/snippets/365/
It needed some updating, and this should work (tested on django 2.0)
import tempfile, zipfile
from django.http import HttpResponse
from wsgiref.util import FileWrapper
def send_zipfile(request):
"""
Create a ZIP file on disk and transmit it in chunks of 8KB,
without loading the whole file into memory. A similar approach can
be used for large dynamic PDF files.
"""
temp = tempfile.TemporaryFile()
archive = zipfile.ZipFile(temp, 'w', zipfile.ZIP_DEFLATED)
for index in range(10):
filename = 'C:/Users/alex1/Desktop/temp.png' # Replace by your files here.
archive.write(filename, 'file%d.png' % index) # 'file%d.png' will be the
# name of the file in the
# zip
archive.close()
temp.seek(0)
wrapper = FileWrapper(temp)
response = HttpResponse(wrapper, content_type='application/zip')
response['Content-Disposition'] = 'attachment; filename=test.zip'
return response
Right now, this takes my .png and writes it 10 times in my .zip, then sends it.
You could add your files/images to a ZIP file and return that one in the response. I think that is the best approach.
Here is some example code of how you could achieve that (from this post):
def zipFiles(files):
outfile = StringIO() # io.BytesIO() for python 3
with zipfile.ZipFile(outfile, 'w') as zf:
for n, f in enumarate(files):
zf.writestr("{}.csv".format(n), f.getvalue())
return outfile.getvalue()
zipped_file = zip_files(myfiles)
response = HttpResponse(zipped_file, content_type='application/octet-stream')
response['Content-Disposition'] = 'attachment; filename=my_file.zip'
Otherwise (if you don't like ZIP files) you could make individual requests from the client.
I'm fairly new to Python. Currently I'm making a prototype that takes an image, creates a thumbnail out of it and and uploads it to the ftp server.
So far I got the get image, convert and resize part ready.
The problem I run into is that using the PIL (pillow) Image library converts the image is a different type than that can be used when uploading using storebinary()
I already tried some approaches like using StringIO or BufferIO to save the image in-memory. But I'm getting errors all the time. Sometimes the image does get uploaded but the file appears to be empty (0 bytes).
Here is the code I'm working with:
import os
import io
import StringIO
import rawpy
import imageio
import Image
import ftplib
# connection part is working
ftp = ftplib.FTP('bananas.com')
ftp.login(user="banana", passwd="bananas")
ftp.cwd("/public_html/upload")
def convert_raw():
files = os.listdir("/home/pi/Desktop/photos")
for file in files:
if file.endswith(".NEF") or file.endswith(".CR2"):
raw = rawpy.imread(file)
rgb = raw.postprocess()
im = Image.fromarray(rgb)
size = 1000, 1000
im.thumbnail(size)
ftp.storbinary('STOR Obama.jpg', img)
temp.close()
ftp.quit()
convert_raw()
What I tried:
temp = StringIO.StringIO
im.save(temp, format="png")
img = im.tostring()
temp.seek(0)
imgObj = temp.getvalue()
The error I'm getting lies on the line ftp.storbinary('STOR Obama.jpg', img).
Message:
buf = fp.read(blocksize)
attributeError: 'str' object has no attribute read
For Python 3.x use BytesIO instead of StringIO:
temp = BytesIO()
im.save(temp, format="png")
ftp.storbinary('STOR Obama.jpg', temp.getvalue())
Do not pass a string to storbinary. You should pass a file or file object (memory-mapped file) to it instead. Also, this line should be temp = StringIO.StringIO(). So:
temp = StringIO.StringIO() # this is a file object
im.save(temp, format="png") # save the content to temp
ftp.storbinary('STOR Obama.jpg', temp) # upload temp
Right now, I have the following code:
pilimg = PILImage.open(img_file_tmp) # img_file_tmp just contains the image to read
pilimg.thumbnail((200,200), PILImage.ANTIALIAS)
pilimg.save(fn, 'PNG') # fn is a filename
This works just fine for saving to a local file pointed to by fn. However, what I would want this to do instead is to save the file on a remote FTP server.
What is the easiest way to achieve this?
Python's ftplib library can initiate an FTP transfer, but PIL cannot write directly to an FTP server.
What you can do is write the result to a file and then upload it to the FTP server using the FTP library. There are complete examples of how to connect in the ftplib manual so I'll focus just on the sending part:
# (assumes you already created an instance of FTP
# as "ftp", and already logged in)
f = open(fn, 'r')
ftp.storbinary("STOR remote_filename.png", f)
If you have enough memory for the compressed image data, you can avoid the intermediate file by having PIL write to a StringIO, and then passing that object into the FTP library:
import StringIO
f = StringIO()
image.save(f, 'PNG')
f.seek(0) # return the StringIO's file pointer to the beginning of the file
# again this assumes you already connected and logged in
ftp.storbinary("STOR remote_filename.png", f)