this is my models.py file
class Post(models.Model):
"""docstring for Post"""
poster = models.ForeignKey(User, null= False,blank=True, default=User.objects.get(username="admin"))
post_image = models.ImageField(upload_to='posts', null=True, blank=True)
def save(self, url='', *args, **kwargs):
if self.post_image != '' and url != '': # Don't do anything if we don't get passed anything!
image = download_image(url) # See function definition below
try:
filename = urlparse.urlparse(url).path.split('/')[-1]
self.post_image = filename
tempfile = image
tempfile_io = io.StringIO() # Will make a file-like object in memory that you can then save
tempfile.save(tempfile_io, format=image.format)
self.post_image.save(filename, ContentFile(tempfile_io.getvalue()), save=False) # Set save=False otherwise you will have a looping save method
except Exception as e:
print ("Error trying to save model: saving image failed: " + str(e))
pass
super(Post, self).save(*args, **kwargs)
def download_image(url):
"""Downloads an image and makes sure it's verified.
Returns a PIL Image if the image is valid, otherwise raises an exception.
"""
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0'} # More likely to get a response if server thinks you're a browser
r = urllib.Request(url, headers=headers)
request = urllib.urlopen(r, timeout=10)
image_data = io.StringIO(request.read()) # StringIO imitates a file, needed for verification step
img = Image.open(image_data) # Creates an instance of PIL Image class - PIL does the verification of file
img_copy = copy.copy(img) # Verify the copied image, not original - verification requires you to open the image again after verification, but since we don't have the file saved yet we won't be able to. This is because once we read() urllib2.urlopen we can't access the response again without remaking the request (i.e. downloading the image again). Rather than do that, we duplicate the PIL Image in memory.
if valid_img(img_copy):
return img
else:
# Maybe this is not the best error handling...you might want to just provide a path to a generic image instead
raise Exception('An invalid image was detected when attempting to save a Product!')
def valid_img(img):
"""Verifies that an instance of a PIL Image Class is actually an image and returns either True or False."""
type = img.format
if type in ('GIF', 'JPEG', 'JPG', 'PNG'):
try:
img.verify()
return True
except:
return False
else: return False
def __unicode__(self):
return self.post_image.url
and my view.py is
def createpost(request):
# Handle file upload
new_img_id = 0
if request.method == 'POST':
external_url = request.POST['url']
p = Post(poster=request.user)
p.save(external_url)
new_img_id=p.id
post = Post.objects.filter(id=new_img_id)
return render_to_response('create.html',{'post': post},context_instance=RequestContext(request))
and this is where the url gets called
$.ajax({
type: "POST",
url: "/create/",
data: {'url': newURL, 'csrfmiddlewaretoken': csrftoken},
success: function(){}
});
in the console I am getting this
in save
NameError: name 'download_image' is not defined
and in the browser console I'm getting this
POST http://localhost:8000/create/ 500 (INTERNAL SERVER ERROR)
If anyone can understand where the origin or this problem may be please help :D
I did try changing the order of the defs but there was not difference
Since you functions are methods of Post, you need to call them as such. Methods are always referred to via the instance, so in this case self.download_image(url), and always need to take self as the first parameter, so def download_image(self, url). Both of these also apply to valid_img.
Note also that it is a very bad idea to override the signature of the save method. Lots of code both in Django and in third-party applications will not be expecting that parameter. Instead, get it from kwargs:
def save(self, *args, **kwargs):
url = kwargs.pop('url', '')
If you want to make an object method, first parameter should be self and you can call method via self.download_image(...)
Also you should write download_image method inside save method, if you want to use like this.
def save(self, ...):
def download_image():
...
download_image()
Related
I'm developing an Android application with backend developed using Tastypie and Django. I have a get request for which I want to optionally be able to retrieve the entire object (with complete related fields, rather than URIs). Below is part of the python code for the resource I'm talking about:
class RideResource(ModelResource):
user = fields.ForeignKey(UserResource, 'driver')
origin = fields.ForeignKey(NodeResource, 'origin', full=True)
destination = fields.ForeignKey(NodeResource, 'destination', full=True)
path = fields.ForeignKey(PathResource, 'path')
# if the request has full_path=1 then we perform a deep query, returning the entire path object, not just the URI
def dehydrate(self, bundle):
if bundle.request.GET.get('full_path') == "1":
self.path.full = True
else:
ride_path = bundle.obj.path
try:
bundle.data['path'] = _Helpers.serialise_path(ride_path)
except ObjectDoesNotExist:
bundle.data['path'] = []
return bundle
As you can see the RideResource has a foreign key pointing to PathResource. I'm using the dehydrate function to be able to inspect if the GET request has a parameter "full_path" set to 1. In that case I set programmatically the path variable to full=True. Otherwise I simply return the path URI.
The thing is that the code seems to work only the second time the GET is performed. I've tested it hundreds of times and, when I perform my GET with full_path=1, even tho it enters the if and sets self.path.full = True, the first time it only returns the URI of the PathResource object. While, if I relaunch the exact same request a second time it works perfectly...
Any idea what's the problem?
EDIT AFTER SOLUTION FOUND THANKS TO #Tomasz Jakub Rup
I finally managed to get it working using the following code:
def full_dehydrate(self, bundle, for_list=False):
self.path.full = bundle.request.GET.get('full_path') == "1"
return super(RideResource, self).full_dehydrate(bundle, for_list)
def dehydrate(self, bundle):
if not bundle.request.GET.get('full_path') == "1":
try:
bundle.data['path'] = _Helpers.serialise_path(bundle.obj.path)
except ObjectDoesNotExist:
bundle.data['path'] = []
return bundle
dehydrate is called after full_dehydrate. Overwrite full_dehydrate function.
def full_dehydrate(self, bundle, for_list=False):
self.path.full = bundle.request.GET.get('full_path') == "1"
return super(RideResource, self).full_dehydrate(bundle, for_list)
I am newer to Python, I am making Flask application. so, I want to write Test Cases for my application using unittest, and I am doing like this:
def test_bucket_name(self):
self.test_app = app.test_client()
response = self.test_app.post('/add_item', data={'name':'test_item','user_id':'1','username':'admin'})
self.assertEquals(response.status, "200 OK")
It is all work well. But I am posting some data and image with POST in one URL. So, My Question is that : "How do i send image with that data?"
Read the image into a StringIO buffer. Pass the image as another item in the form data, where the value is a tuple of (image, filename).
def test_bucket_name(self):
self.test_app = app.test_client()
with open('/home/linux/Pictures/natural-scenery.jpg', 'rb') as img1:
imgStringIO1 = StringIO(img1.read())
response = self.test_app.post('/add_item',content_type='multipart/form-data',
data={'name':'test_item',
'user_id':'1',
'username':'admin',
'image': (imgStringIO1, 'img1.jpg')})
self.assertEquals(response.status, "200 OK")
The above answer is correct, except in my case I had to use BytesIO like the following:
def create_business(self, name):
with open('C:/Users/.../cart.jpg', 'rb') as img1:
imgStringIO1 = BytesIO(img1.read())
return self.app.post(
'/central-dashboard',
content_type='multipart/form-data',
data=dict(name=name, logo=(imgStringIO1, 'cart.jpg')),
follow_redirects=True
)
A screenshot (portrait view) of my IDE and Traceback showing all the code pasted here, may be easier to read if you have a vertical monitor.
Context: Trying to save image from a URL to a Django ImageField hosted on EC2 with files on S3 using S3BotoStorage. I'm confused because all of this suggests that Django is still treating it like local storage, while it should S3.
The lines in question that seem to be causing the error:
def get_filename(self, filename):
return os.path.normpath(self.storage.get_valid_name(os.path.basename(filename)))
def get_valid_name(self, name):
"""
Returns a filename, based on the provided filename, that's suitable for
use in the target storage system.
"""
return get_valid_filename(name)
TypeError Exception: get_valid_name() missing 1 required positional argument: 'name'
Last Local vars Tracback before error at get_valid_name:
filename 'testimagefilename'
self <django.db.models.fields.files.ImageField: image>
(Only the stuff inside these two horizontal dividers is from me, the rest is from Django 1.9)
image.image.save('testimagefilename', File(temp), save=True)
Local vars from Traceback at that point (not sure about the ValueError on image, I think it's because it hasn't been created yet):
File <class 'django.core.files.base.File'>
image Error in formatting: ValueError: The 'image' attribute has no file associated with it.
requests <module 'requests' from '/usr/local/lib/python3.4/site-packages/requests/__init__.py'>
Image <class 'mcmaster.models.Image'>
NamedTemporaryFile <function NamedTemporaryFile at 0x7fb0e1bb0510>
temp <tempfile._TemporaryFileWrapper object at 0x7fb0dd241ef0>
Relevant snippets of Django source code:
files.py
def save(self, name, content, save=True):
name = self.field.generate_filename(self.instance, name)
if func_supports_parameter(self.storage.save, 'max_length'):
self.name = self.storage.save(name, content, max_length=self.field.max_length)
else:
warnings.warn(
'Backwards compatibility for storage backends without '
'support for the `max_length` argument in '
'Storage.save() will be removed in Django 1.10.',
RemovedInDjango110Warning, stacklevel=2
)
self.name = self.storage.save(name, content)
setattr(self.instance, self.field.name, self.name)
# Update the filesize cache
self._size = content.size
self._committed = True
# Save the object because it has changed, unless save is False
if save:
self.instance.save()
save.alters_data = True
def get_directory_name(self):
return os.path.normpath(force_text(datetime.datetime.now().strftime(force_str(self.upload_to))))
def get_filename(self, filename):
return os.path.normpath(self.storage.get_valid_name(os.path.basename(filename)))
def generate_filename(self, instance, filename):
# If upload_to is a callable, make sure that the path it returns is
# passed through get_valid_name() of the underlying storage.
if callable(self.upload_to):
directory_name, filename = os.path.split(self.upload_to(instance, filename))
filename = self.storage.get_valid_name(filename)
return os.path.normpath(os.path.join(directory_name, filename))
return os.path.join(self.get_directory_name(), self.get_filename(filename))
storage.py
def get_valid_name(self, name):
"""
Returns a filename, based on the provided filename, that's suitable for
use in the target storage system.
"""
return get_valid_filename(name)
text.py
def get_valid_filename(s):
"""
Returns the given string converted to a string that can be used for a clean
filename. Specifically, leading and trailing spaces are removed; other
spaces are converted to underscores; and anything that is not a unicode
alphanumeric, dash, underscore, or dot, is removed.
>>> get_valid_filename("john's portrait in 2004.jpg")
'johns_portrait_in_2004.jpg'
"""
s = force_text(s).strip().replace(' ', '_')
return re.sub(r'(?u)[^-\w.]', '', s)
get_valid_filename = allow_lazy(get_valid_filename, six.text_type)
I'd make a guess you didn't instantiate the Storage class. How are you setting Django to use the custom storage? If you do this in models.py
image = models.ImageField(storage=MyStorage)
It will fail exactly as you describe. It should be
image = models.ImageField(storage=MyStorage())
I've written a custom Django file upload handler for my current project. It's a proof-of-concept which allows you to compute a hash of an uploaded file without storing that file on-disk. It's a proof of concept, to be sure, but if I can get it to work, I can get onto the real purpose of my work.
Essentially, here's what I have so far, which is working fine with one major exception:
from django.core.files.uploadhandler import *
from hashlib import sha256
from myproject.upload.files import MyProjectUploadedFile
class MyProjectUploadHandler(FileUploadHandler):
def __init__(self, *args, **kwargs):
super(MyProjectUploadHandler, self).__init__(*args, **kwargs)
def handle_raw_input(self, input_data, META, content_length, boundary,
encoding = None):
self.activated = True
def new_file(self, *args, **kwargs):
super(MyProjectUploadHandler, self).new_file(*args, **kwargs)
self.digester = sha256()
raise StopFutureHandlers()
def receive_data_chunk(self, raw_data, start):
self.digester.update(raw_data)
def file_complete(self, file_size):
return MyProjectUploadedFile(self.digester.hexdigest())
The custom upload handler works great. The hash is accurate and works without storing any of the uploaded file to disk and only uses 64kb of memory at any one time.
The only problem I'm having is that I need to access another field from the POST request before processing the file, a text salt input by the user. My form looks like this:
<form id="myForm" method="POST" enctype="multipart/form-data" action="/upload/">
<fieldset>
<input name="salt" type="text" placeholder="Salt">
<input name="uploadfile" type="file">
<input type="submit">
</fieldset>
</form>
The "salt" POST variable is only made available to me after the request has been processed and the file has been uploaded, which doesn't work for my use case. I can't seem to find a way to access this variable in any way, shape, or form in my upload handler.
Is there a way for me to access each multipart variable as it comes across instead of just accessing the filess which are uploaded?
My solution didn't come easy, but here it is:
class IntelligentUploadHandler(FileUploadHandler):
"""
An upload handler which overrides the default multipart parser to allow
simultaneous parsing of fields and files... intelligently. Subclass this
for real and true awesomeness.
"""
def __init__(self, *args, **kwargs):
super(IntelligentUploadHandler, self).__init__(*args, **kwargs)
def field_parsed(self, field_name, field_value):
"""
A callback method triggered when a non-file field has been parsed
successfully by the parser. Use this to listen for new fields being
parsed.
"""
pass
def handle_raw_input(self, input_data, META, content_length, boundary,
encoding = None):
"""
Parse the raw input from the HTTP request and split items into fields
and files, executing callback methods as necessary.
Shamelessly adapted and borrowed from django.http.multiparser.MultiPartParser.
"""
# following suit from the source class, this is imported here to avoid
# a potential circular import
from django.http import QueryDict
# create return values
self.POST = QueryDict('', mutable=True)
self.FILES = MultiValueDict()
# initialize the parser and stream
stream = LazyStream(ChunkIter(input_data, self.chunk_size))
# whether or not to signal a file-completion at the beginning of the loop.
old_field_name = None
counter = 0
try:
for item_type, meta_data, field_stream in Parser(stream, boundary):
if old_field_name:
# we run this test at the beginning of the next loop since
# we cannot be sure a file is complete until we hit the next
# boundary/part of the multipart content.
file_obj = self.file_complete(counter)
if file_obj:
# if we return a file object, add it to the files dict
self.FILES.appendlist(force_text(old_field_name, encoding,
errors='replace'), file_obj)
# wipe it out to prevent havoc
old_field_name = None
try:
disposition = meta_data['content-disposition'][1]
field_name = disposition['name'].strip()
except (KeyError, IndexError, AttributeError):
continue
transfer_encoding = meta_data.get('content-transfer-encoding')
if transfer_encoding is not None:
transfer_encoding = transfer_encoding[0].strip()
field_name = force_text(field_name, encoding, errors='replace')
if item_type == FIELD:
# this is a POST field
if transfer_encoding == "base64":
raw_data = field_stream.read()
try:
data = str(raw_data).decode('base64')
except:
data = raw_data
else:
data = field_stream.read()
self.POST.appendlist(field_name, force_text(data, encoding,
errors='replace'))
# trigger listener
self.field_parsed(field_name, self.POST.get(field_name))
elif item_type == FILE:
# this is a file
file_name = disposition.get('filename')
if not file_name:
continue
# transform the file name
file_name = force_text(file_name, encoding, errors='replace')
file_name = self.IE_sanitize(unescape_entities(file_name))
content_type = meta_data.get('content-type', ('',))[0].strip()
try:
charset = meta_data.get('content-type', (0, {}))[1].get('charset', None)
except:
charset = None
try:
file_content_length = int(meta_data.get('content-length')[0])
except (IndexError, TypeError, ValueError):
file_content_length = None
counter = 0
# now, do the important file stuff
try:
# alert on the new file
self.new_file(field_name, file_name, content_type,
file_content_length, charset)
# chubber-chunk it
for chunk in field_stream:
if transfer_encoding == "base64":
# base 64 decode it if need be
over_bytes = len(chunk) % 4
if over_bytes:
over_chunk = field_stream.read(4 - over_bytes)
chunk += over_chunk
try:
chunk = base64.b64decode(chunk)
except Exception as e:
# since this is anly a chunk, any error is an unfixable error
raise MultiPartParserError("Could not decode base64 data: %r" % e)
chunk_length = len(chunk)
self.receive_data_chunk(chunk, counter)
counter += chunk_length
# ... and we're done
except SkipFile:
# just eat the rest
exhaust(field_stream)
else:
# handle file upload completions on next iteration
old_field_name = field_name
except StopUpload as e:
# if we get a request to stop the upload, exhaust it if no con reset
if not e.connection_reset:
exhaust(input_data)
else:
# make sure that the request data is all fed
exhaust(input_data)
# signal the upload has been completed
self.upload_complete()
return self.POST, self.FILES
def IE_sanitize(self, filename):
"""Cleanup filename from Internet Explorer full paths."""
return filename and filename[filename.rfind("\\")+1:].strip()
Essentially, by subclassing this class, you can have a more... intelligent upload handler. Fields will be announced with the field_parsed method to subclasses, as I needed for my purposes.
I've reported this as a feature request to the Django team, hopefully this functionality becomes a part of the regular toolbox in Django, rather than monkey-patching the source code as done above.
Based on the code for FileUploadHandler, found here at line 62:
https://github.com/django/django/blob/master/django/core/files/uploadhandler.py
It looks like the request object is passed into the handler and stored as self.request
In that case you should be able to access the salt at any point in your upload handler by doing
salt = self.request.POST.get('salt')
Unless I'm misunderstanding your question.
I'm trying to upload a picture with the python sdk:
Code:
graph = facebook.GraphAPI(self.current_user.access_token)
graph.put_object("me", "photos", name = "test", message = raw_picture_data)
But I get the error "GraphAPIError: (#324) Requires upload file". I don't think its a permissions issue as I've requested perms="user_photos,friends_photos,publish_stream". Does anyone know what this error means and how to resolve it?
I used this library to encode the image: http://atlee.ca/software/poster/
Add this to facebook.py:
from poster.encode import *
from poster.streaminghttp import register_openers
def put_photo(self, source, album_id=None, message=""):
object_id = album_id or "me"
register_openers()
content_type,body = multipart_encode( [ ('message',message),('access_token',self.access_token),('source',source) ] )
req = urllib2.Request("https://graph.facebook.com/%s/photos" % object_id, content_type,body )
try:
data = urllib2.urlopen(req).read()
except urllib2.HTTPError as e:
data = e.read()
try:
response = _parse_json(data)
if response.get("error"):
raise GraphAPIError(response["error"].get("code", 1),response["error"]["message"])
except ValueError:
response = data
return response
Call the function with the photo as a file like object:
graph = facebook.GraphAPI(access_token)
photo = open("myphoto.bmp","rb")
graph.put_photo(photo,"me","This is my brilliant photo")
The put_photo method has been submitted by someone (I forget who) as proposed a function to add to the API but it didn't work for me till I used poster to encode the image.
Hope this helps.
Just battled with a similar error a bit. I did not use the SDK but just a POST to the graphapi. For me, this error happened when i didnt supply a filename to the file upload field in the "form" sent to facebook.
This is my code (poster - http://pypi.python.org/pypi/poster/0.8.1)
from poster.encode import multipart_encode, MultipartParam
url = 'https://graph.facebook.com/me/photos?access_token=%s'%model.facebook_token
file_param = MultipartParam(name = 'source',
filename = 'photo.jpg', #this is crucial!!!
fileobj = blob_reader) #the blob reader is the fileobject for the file (with read() method)
message_param = MultipartParam(name = 'message',
value = 'test message')
datagen, headers = multipart_encode([file_param,
message_param])
from google.appengine.api import urlfetch
result = urlfetch.fetch(url,
payload = ''.join(datagen),
headers = headers,
method = 'POST')
return result.content