We Keep Coding

Create a Mongoengine FileField from a GridFs file_id or GridFSProxy

I have a string file_id of my file that is stored in mongodb in a fs collection (GridFs).
I need to store the file as a mongoengine FileField in a Document, and then return the file to an endpoint ... so access the file's content, content_type etc.
I am not sure how to create a FileField instance with the GridFs string id? And is it possible to get the content and content_type from the FileField?
The tutorials that I have seen all involve creating the FileField by writing the contents to mongodb, the difference is my contents are already in the GridFs and I have the string id.
class Test(Document):
file = FileField()
test = Test()
test.upload.put(image_file, content_type='image/png')
So far I have been able to create a GridFsProxy object using the id, and can read the file using this.
class Test(Document):
file = FileField()
file_id = StringField() # bb2832e0-2ca4-44bc-8b1b-e01a77003b92
file_proxy = GridFSProxy(Test.file_id)
file_proxy.read() # Gives me the file content
file_proxy.get(file_id).content_type #can return name, length etc.
test = Test()
test.file = file_proxy.read() # in mongodb I see it as an ObjectID
If i store the read() results of the GridFSProxy into the FileField(); it is stored in MongoDb as an ObjectID and then later when I retrieve the object I don't seem to be able to get the content_type of the file.
I need the content_type as it is important for how I return the file contents.
I'm not sure how to create the FileField using just the file_id and then use it when i retrieve the Document.
Any insight into using the FileField (and GridFSProxy) will be helpful.

The FileField is basically just a reference (ObjectId) pointing to the actual grid fs documents (stored in fs.chunks/fs.files). Accessing the content_type should be straightforward, you shouldn't have to use the GridFSProxy class at all, see below:
from mongoengine import *
class Test(Document):
file = FileField()
test = Test()
image_bytes = open("path/to/image.png", "rb")
test.file.put(image_bytes, content_type='image/png', filename='test123.png')
test.save()
Test.objects.as_pymongo() # [{u'_id': ObjectId('5cdac41d992db9bfcaa870df'), u'file': ObjectId('5cdac419992db9bfcaa870dd')}]
t = Test.objects.first()
t.file # <GridFSProxy: 5cdac419992db9bfcaa870dd>
t.file.content_type # 'image/png'
t.file.filename # 'test123.png'
content = t.file.read()

TypeError when open object of django filefield

I want to make a button named 'count' to count the number of page of the uploaded pdf file. However, there is TypeError invalid file:
How can I improve my code to tackle this error...?
PS. I'm newbie in coding using django 1.10, and just want to make some little tool to make my life easiler :)
Thanks in advance
My Models
from django.db import models
from PyPDF2 import PdfFileReader
class PdfFile(models.Model):
file = models.FileField(upload_to='document/')
num_of_pages = models.IntegerField(null=True)
def page_count(self):
pdf = PdfFileReader(open(self.file, 'rb'))
self.num_of_pages = pdf.getNumPages()
self.save()
My Views
def count(request, pk):
pdf = get_object_or_404(PdfFile, pk=pk)
pdf.page_count()
return redirect('img_preview', pk=pk)

You can't pass a filefield value directly to open. However, as the documentation shows, that object itself has an open method:
pdf = PdfFileReader(self.file.open())
Because of the way that class also defines read and __iter__, you might actually be able to pass the field directly:
pdf = PdfFileReader(self.file)

Convert blob to Django ImageField in python?

I read user image in BLOB file, but i want save it to image format in django model.
How can i convert this file to image file(.jpeg) and save it in django models.ImageField?
I use python 2.7 and django 1.9.
my model is:
class Staff(models.Model):
user = models.ForeignKey(User)
cn = models.CharField(max_length=100)
new_comer = models.NullBooleanField()
change_position = models.NullBooleanField()
change_manager = models.NullBooleanField()
acting = models.NullBooleanField()
expelled = models.NullBooleanField()
active = models.NullBooleanField()
avatar = models.ImageField(upload_to='/images/')
Please help me...

You need to try something like this.
import io
from django.core.files.base import File
# Set values of your model filed.
staff_instance = Staff()
staff_instance.user = user_instance
...
...
with io.BytesIO(blob_file) as stream:
django_file = File(stream)
staff_instance.avatar.save(some_file_name, django_file)
staff_instance.save()
I assume that blob is a byte array of the file.
To make it a file i need to convert it to a stream.
Thus I thought BytesIO would be a good choice.
You can directly save file to disk but if you want django to upload it to your upload_to directory, you need to use django.core.files.base.File class.
When you run django.core.files.base() method file will be saved to desired directory.
I guess you will use this for an data migration process, not in a view.
If this is the case than you could put this code at a django command.
Then you can use any django and project related resources.
Let me know if it helps.

FileField in Tastypie

I have a model for which I am making an api in tastypie. I have a field which stores the path to a file which I maintain manually (I am not using FileField since users are not uploading the files). Here is a gist of a model:
class FooModel(models.Model):
path = models.CharField(max_length=255, null=True)
...
def getAbsPath(self):
"""
returns the absolute path to a file stored at location self.path
"""
...
Here is my tastypie config:
class FooModelResource(ModelResource):
file = fields.FileField()
class Meta:
queryset = FooModel.objects.all()
def dehydrate_file(self, bundle):
from django.core.files import File
path = bundle.obj.getAbsPath()
return File(open(path, 'rb'))
In the api in the file field this returns full path to a file. I want tastypie to be able to serve the actual file or at least an url to a file. How do I do that? Any code snippets are appreciated.
Thank you

Decide on a URL scheme how your files will be exposed through the APIs first. You don't really need file or dehydrate_file (unless you want to change the representation of the file for the model itself in Tastypie). Instead just add an additional action on the ModelResource. Example:
class FooModelResource(ModelResource):
file = fields.FileField()
class Meta:
queryset = FooModel.objects.all()
def override_urls(self):
return [
url(r"^(?P<resource_name>%s)/(?P<pk>\w[\w/-]*)/download%s$" % (self._meta.resource_name, trailing_slash()), self.wrap_view('download_detail'), name="api_download_detail"),
]
def download_detail(self, request, **kwargs):
"""
Send a file through TastyPie without loading the whole file into
memory at once. The FileWrapper will turn the file object into an
iterator for chunks of 8KB.
No need to build a bundle here only to return a file, lets look into the DB directly
"""
filename = self._meta.queryset.get(pk=kwargs[pk]).file
wrapper = FileWrapper(file(filename))
response = HttpResponse(wrapper, content_type='text/plain') #or whatever type you want there
response['Content-Length'] = os.path.getsize(filename)
return response
GET .../api/foomodel/3/
Returns:
{
...
'file' : 'localpath/filename.ext',
...
}
GET .../api/foomodel/3/download/
Returns:
...contents of actual file...
Alternatively you could create a non-ORM Sub Resource file in FooModel. You would have to define resource_uri (how to uniquely identify each instance of the resource), and override dispatch_detail to do exactly what download_detail above does.

The only conversion tastypie does on a FileField is to look for an 'url' attribute on what you return, and return that if it exists, else it will return the string-ized object, which as you have noticed is just the filename.
If you want to return the file content as a field, you will need to handle the encoding of the file. You have a few options:
Simplest: Use CharField and use the base64 module to convert the bytes read from the file into a string
More general but functionally equivalent: write a custom tastypie Serializer that knows how to turn File objects into string representations of their contents
Override the get_detail function of your resource to serve just the file using whatever content-type is appropriate, to avoid the JSON/XML serialization overhead.

Django: manually create imagefield in model from existing file on server

This is killing me!
I'm using django-filebrowser, and I want to create a gallery app that leverages it's upload capabilities to manage images.
I have a Gallery model that allows the user to select or create a directory on the server, and upload files to that folder to appear in the gallery. I want to automatically trawl the directory that the user has uploaded images to and selected, and then automatically create Image instances for each image in the folder.
class Gallery(model.Models):
gallerydirectory = FileBrowserField(...)
title = ...
description ...
class Image(model.Models):
image_field = models.ImageField()
The problem is that FileBrowser represents images differently to Django, but I want to use DJango ImageFields as I can then use other apps (sorl thumbnails) on the template end.
I have all the data necessary for the file i.e. filename, path etc, I just can't get Django to create an instance of an ImageField, without actually uploading the image again. I simply want to populate it.
I have seen another thread here which suggests the following:
for image in filebrowser_image_objects_list:
f = File(open('path-to-file-on-server','r'))
i = Image()
i.image_field('filename.png',f.read())
but this is giving me a:
SuspiciousOperation error
Attempted access to '/filename.png' denied
which suggests that the path isn't being read correctly. I've printed the attributes of the File Object, and it is opening the correct image, it just isn't getting passed on to the ImageField
Help would be greately appreciated!
UPDATE
I've given up trying to get this work as it's just too messy. The problem I had above was that I had set the upload_field of the ImageField to '/' and it had gone unnoticed meaning that the file was being written to '/something.png'.
I've modified it so that the Image is now using a FileBrowserField as well instead of an ImageField.

I might be missing something, but this worked for me:
from a1.models import Model1
from django.core.files.images import ImageFile
m = Model1.objects.create()
m.f1 = ImageFile(open("1.png", "rb"))
m.save()
for the following model:
class Model1(models.Model):
f1 = models.ImageField()
This way it didn't work:
m.f1('1.png', File.read(open('1.png', 'r')))
It says:
TypeError: 'ImageFieldFile' object is not callable
Checked with Django 1.7, 1.11.

I'm marking this as answered, as this is the correct way to do this:
from django.core.files import File
image_model.image_field('path', File().read())
Programmatically saving image to Django ImageField

This works for me on Python3 with Django 2
from urllib.request import urlopen
from urllib.parse import urlparse
from io import BytesIO
from django.core.files.images import ImageFile
from .models import ModelWithImageField
# Solving for SSL Error
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
url = https://www.someurl.com/image.png?other_params_if_they_exist
image_file_name = urlparse(url).path.split('/')[-1]
image_file_content = BytesIO(urlopen(url).read())
ModelWithImageField().image_field.save(image_file_name, image_file_content)