Is it possible in Django to store image files in different buckets/containers in external object storage OpenStack Swift?
I have an issue related with creating proper way to upload image files through REST service request. The user is able to POST 'Task' instance via first endpoint, containing name, description, owner_id etc. He is also able to POST images via another endpoint, which has a relation many to one with Task.
Images should be stored in OpenStack Swift in my case (server already exists, was set up etc.) in unique containers/buckets as follows: owner_id_task_id. It is related, that users can upload files with the same names and extensions, but different content.
Another part will be sending also to the same containers in OpenStack Swift a files from Celery worker tasks (some processing based on uploaded files).
My goal is to achieve dynamically-created/overrided at runtime container structure, for storing raw images, and also post-processed images.
Any ideas how this problem can be solved?
Thanks for the help!
Yes. This is possible. You can use FileField.storage to configure which storage to use in individual model fields.
Related
I need to store large file (around 1Go) on my data warehouse. To explain the global contexte simply : the user selects some options in a webview and sends it to the middleware. The middleware uses these informations and sends new ones to a factory. The factory creates the 1Go file. Now I need to store it on the data warehouse in order to be download later by the user.
I use the framework django for the middleware and the factory is programmed in python.
So my question is : for this size of files, is it better to store it on a blob field on a database ? Or is it better to store it directly on the file systeme ?
Storing you blob in database using a BinaryField will harm the performance of your application as your querysets will have to handle a lot more data. Best practice is to store blobs in separate files and only store the file path - using models.FilePathField() field - in your database and serve the actual image as static file. See this post for more details.
I am building a web application based on Google App Engine using python; however I have seem to have hit a roadblock.
Currently, my site fetches remote image URLs from an external website. The URLs are placed in a list and sent back to my application. I want to be able to store the respective images (not the URLs) in my datastore; in order to avoid having to fetch the remote images each time as well as having to deal with broken links.
The solutions I have found online all deal with a user having to upload their own images. Which I tried implementing, but I am not sure what happens to an uploaded image (or how it gets converted into a blob) once the user hits the submit button.
To my understanding, a blob is a collection of binary data stored as a single entity (from Wikipedia). Therefore, I tried using the following:
class rentalPropertyDB(ndb.Model):
streetNAME = ndb.StringProperty(required=True)
image = ndb.BlobProperty(default=None)
class MainPage(BaseHandler):
def get(self):
self.render("index.html")
def post(self):
rental = rentalPropertyDB()
for image in img_urls:
rental.image = urlfetch.Fetch(image).content
rental.put()
The solution to this question: Image which is stored as a blob in Datastore in a html page
is identical to mine, however the solution suggests to upload the image to the blobstore and uses:
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
This confuses me because I am not sure what exactly 'file' refers to. Would I replace 'file' with the URL of each image? Or would I need to perform some operation to each image prior to replacing?
I have been stuck on this issue for at least two days now and would greatly appreciate any help that's provided. I think the main reason why this is confusing me so much is because of the variety of methods used in each solution. I.e. using Google Cloud Storage, URLFetch, Images API, and the various types of ndb.Blobstore's (BlobKeyProperty vs. BlobProperty) and etc.
Thank you.
Be careful with blob inside Models. A Model cannot be more than 1 Mo including the blobproperty.
If we listen Google, there is no good reason to use the blobstore. If you can, use Google Cloud Storage. It made to store files.
Sorry if the question seems stupid. I'm currently trying to create a website using Flask. This website will let user entry with an images. Let's say there is a price, a title, a description and an image. My problem here is with the images. I can store all the information I need in the databse but I really don't know what to do about the images. For example, when people enter the website, I'd want to display a couple of those images.
BUT, where do I store them, how do I serve them when they are called ? (For example, if I look at a user profile I'd like to see his post, images, etc...) I just don't know what to do about those images. If someone could explain like i'm an idiot or link me some information about that.
(For example, when you look at someone's profil on instagram you'll see every on his images, that's what i'd like too achieve)
Thanks !
You could store the images on the filesystem of your web server, then configure your webserver to serve them statically (if using Apache, for example, this would be done via the Alias Directive).
This would involve mapping URLs (e.g. mysite.com/img/) to a directory on the web server's filesystem (e.g. /home/images). Of course, your webserver will most likely be serving a directory by default, for example /var/www/, and you could simply store the images here.
Where you store the images (and how they are served) will depend on things specific to your application, such as security (note that if you serve a particular directory on your filesystem, any client could potentially download the contents).
EDIT
As per your comment - This would depend mostly on the size of the available storage on your web server. Also keep in mind that you should use images of a reasonable size, and a compressed format such as PNG or GIF.
Following the appengine Uploading a blob instructions, I have been available to upload/download images.
But I want to found a method for preventing duplicates, therefore I would like to know if it is possible to have a custom key for the blobstore objects, or use the MD5 as the Key, so that at least I could overwrite existing files.
Is there any hook or some extra parameter that I could use within the blobstore.create_upload_url that could help to specify a custom Key for the uploaded object?
Google is moving away from the blobstore. You can also use the Cloudstorage Client Library.
Some of the benefits:
since 1,9.0 free quota in the default GCS bucket
use folders and filenames and you can overwrite (replace) existing files.
create a serving_url for images and other files, which will be served by Google
and more ..
I have created this gist to show how to use GCS in Google App Engine.
Blob keys are guaranteed to be unique. You don't need to do anything for that.
EDIT:
If you want to rewrite the blob, you need to know the key of a blob that you want to update somewhere in your model. If you want, you can also store a hash or any other identifier (i.e. file name) in your model too. Then you can compare a hash of a new file, for example, with the hashes of previously stored files, and decide if you want to delete a duplicate record.
First than all, I don't even know if this is a session related question. But I could not think a better way to describe it in the title.
I'm developing a web application for registered users so they can create and manage trade unions.
A user can create several unions. Each union can store an image, a description and a name.
The index page shows the list of unions created by the currently registered user.
When the user clicks on a union from the list, all the pages of the application must show
in they headers the corresponding name and image stored for that union.
Also, all the options of the application must refer to the currently selected union.
That is the process for every selected union.
How could I do this on App Engine Python? What technique could I use? Is it something
related to sessions? I do the authentication process with the Gmail service.
I hope I explained myself clearly.
Thanks in advance!
You'd use the datastore to create a union as an entity class, with a description and a name. If your image is small you can store it in your entity, if it's large, you may store it in the blobstore and store a link to it inside your entity.
You can use the python User API for authentication. You don't really need any special session work if you're using the User API.