I'm getting uploaded files in my Flask framework app. This file is image which I want to process by OpenCV (in python). Saving this uploaded file to HDD first, will slow down whole operation (additional time of saving and loading image with OpenCV).
Is it possible to load image directly from Werkzeug FileStorage object (memory)?
EDIT: I think you might be able to use FileStorage.stream as input to your OpenCV logic—it's a file like object; if that doesn't work, see below. (See werkzeug.datastructures.FileStorage.stream)
Since FileStorage itself doesn't seem to be a file-like object, what you can do is save() it into one:
from cStringIO import StringIO
inmem_file = StringIO()
file_storage.save(inmem_file) # save to memory
inmem_file.reset() # seek back to byte 0, otherwise .read() will return ''
use_with_open_cv(inmem_file)
This is assuming OpenCV can work with arbitrary file-like objects, not just objects representing to real files.
Related
I have few icon pics who's binary form is stored in database . now I have few approaches:
To take the binary data from database and covert it into a png or jpeg file and store it on the user's device
or
To directly use the binary data without storing it on user's device
I prefer the second one . but the problem is the kivy widgets such as MDIcon or FloatButtton needs a string of address and not binary data so is their a way to resolve this .
And please let me know if their is a better way of solving this issue.
You can convert an image without saving it on the drive like this.
from io import BytesIO
def convertToJpeg(img):
with BytesIO() as f:
img.save(f, format='JPEG')
return f.getvalue()
I have a python function (using the Pythonista app) to show an image in the console. I have the image saved in a BytesIO object but the function requires a file path.
Is there any way to give it a path to the bytesIO or somehow give it the image without needing to save it as a file?
The specific function is console.show_image(image_path)
The general answer is that if the function you call expects a filesystem path and cannot handle a file-like object instead then your only solution is to write your data to a file (and ask the function's author to add support for file-like object, or if it's OSS implement it by yourself and send a merge request).
Question
What is a clean way to create a file object from raw binary information in Python?
More Info
The reason I need to do this is because I have the raw binary information comprising a jpeg image stored in ram. I need to put it inside some kind of file object so that I can resize the image using Python's Pillow library.
According to the pillow documentation, the file object needs to implement the read(), seek(), and tell() methods.
The file object must implement read(), seek(), and tell() methods, and be opened in binary mode.
I was able to find a mention of how to handle this situation under the documentation for PIL.Image.frombytes:
...If you have an entire image in a string, wrap it in a BytesIO object,
and use open() to load it.
This is what I ended up with that worked using BytesIO:
import io
import PIL
from PIL.Image import Image
file_body = <binary image data>
t_file = io.BytesIO(file_body)
img = PIL.Image.open(t_file)
Note: The comments mention tempfile.SpooledTemporaryFile. This seems like it should have worked, but it did not for some reason.
I understand that opening a file just creates a file handler that takes a fixed memory irrespective of the size of the file.
Django has a type called InMemoryUploadedFile that represents files uploaded via forms.
I get the handle to my file object inside the django view like this:
file_object = request.FILES["uploadedfile"]
This file_object has type InMemoryUploadedFile.
Now we can see for ourselves that, file_object has the method .read() which is used to read files into memory.
bytes = file_object.read()
Wasn't file_object of type InMemoryUploadedFile already "in memory"?
The read() method on a file object is way to access content from within a file object irrespective of whether that file is in memory or stored on the disk. It is similar to other utility file access methods like readlines or seek.
The behavior is similar to what is built into Python which in turn is built over the operating system's fread() method.
Read at most size bytes from the file (less if the read hits EOF
before obtaining size bytes). If the size argument is negative or
omitted, read all data until EOF is reached. The bytes are returned as
a string object. An empty string is returned when EOF is encountered
immediately. (For certain files, like ttys, it makes sense to continue
reading after an EOF is hit.) Note that this method may call the
underlying C function fread() more than once in an effort to acquire
as close to size bytes as possible. Also note that when in
non-blocking mode, less data than was requested may be returned, even
if no size parameter was given.
On the question of where exactly the InMemoryUploadedFile is stored, it is a bit more complicated.
Before you save uploaded files, the data needs to be stored somewhere.
By default, if an uploaded file is smaller than 2.5 megabytes, Django
will hold the entire contents of the upload in memory. This means that
saving the file involves only a read from memory and a write to disk
and thus is very fast.
However, if an uploaded file is too large, Django will write the
uploaded file to a temporary file stored in your system’s temporary
directory. On a Unix-like platform this means you can expect Django to
generate a file called something like /tmp/tmpzfp6I6.upload. If an
upload is large enough, you can watch this file grow in size as Django
streams the data onto disk.
These specifics – 2.5 megabytes; /tmp; etc. – are simply “reasonable
defaults”. Read on for details on how you can customize or completely
replace upload behavior.
One thing to consider is that in python file like objects have an API that is pretty strictly adhered to. This allows code to be very flexible, they are abstractions over I/O streams. These allow your code to not have to worry about where the data is coming from, ie. memory, filesystem, network, etc.
File like objects usually define a couple methods, one of which is read
I am not sure of the actually implementation of InMemoryUploadedFile, or how they are generated or where they are stored (I am assuming they are totally in memory though), but you can rest assured that they are file like objects and contain a read method, because they adhere to the file api.
For the implementation you could start checking out the source:
https://github.com/django/django/blob/master/django/core/files/uploadedfile.py#L90
https://github.com/django/django/blob/master/django/core/files/base.py
https://github.com/django/django/blob/master/django/core/files/uploadhandler.py
I have a file which contains a single image of a specific format at a
specific offset. I can already get a file-like for the embedded image
which supports read(), seek(), and tell(). I want to take advantage
of an existing PIL decoder to handle the embedded image, but be able to
treat the entire file as an "image file" in its own right.
I have not been able to figure out how to do this given the
documentation
available and was wondering if anyone had any insights as to how I could
do this.
The relevant chapter of the docs is this one and I think it's fairly clear: if for example you want to decode image files in the new .zap-format, you write a ZapImagePlugin.py module which must perform a couple things:
have a class ZapImageFile(ImageFile.ImageFile): with string attributes format and format_description, and a hook-method def _open(self) (of which more later);
at module level, Image.register_open('zap', ZapImageFile) and Image.register_extension('ZAP', '.zap')
The specs for the _open method are very clearly laid out in the chapter -- it must read image data and metadata from open binary file-like object self.fp, raise SyntaxError (or another exception) ASAP if it detects that the file's not actually in the right format, set at least self.size and self.mode attributes, and in order to allow reading the image, also self.tile, a list of tile descriptors again in the format specified in that chapter (including the file-offset, which you say you know, and a decoder -- if the raw or bit decoders, documented in the chapter, don't meet your needs, the chapter recommends studying the sources of some of the many supplied decoders, such as JPEG, PNG, etc).
What I did to solve this was to derive from the ImageFile.ImageFile child belonging to the embedded format instead of ImageFile.ImageFile directly. Then in _open() I replaced self.fp with the file-like to the embedded image, and called the parent's _open(). I can't say that I'm particularly happy doing it this way, but it seems to have worked.