I am testing several cases of Django and matplotlib such as this question or in french.
Each time, it works on my mac, but does not on my server, where I receive the following error:
Internal Server Error: /mj/charts/mplimage.png
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py", line 128, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/base.py", line 126, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/root/src/jm/majority_judgment/views.py", line 39, in mplimage
canvas.print_png(response)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_agg.py", line 526, in print_png
with cbook.open_file_cm(filename_or_obj, "wb") as fh:
File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/cbook/__init__.py", line 624, in open_file_cm
fh, opened = to_filehandle(path_or_file, mode, True, encoding)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/cbook/__init__.py", line 615, in to_filehandle
raise ValueError('fname must be a PathLike or file handle')
ValueError: fname must be a PathLike or file handle
[28/Mar/2018 19:09:11] "GET /mj/charts/mplimage.png HTTP/1.1" 500 82804
Here is a minimal snippet:
def mplimage(request):
f = matplotlib.figure.Figure()
canvas = FigureCanvasAgg(f)
response = HttpResponse(content_type='image/png')
canvas.print_png(response)
plt.close(f)
return response
I have tried to update matplotlib, django and so on, but it did nothing...
At the moment, matplotlib's writing functions require the seek ducktype to use the response at a file. You can write to a buffer, like this:
import io
def mplimage(request):
f = matplotlib.figure.Figure()
# Code that sets up figure goes here; in the question, that's ...
FigureCanvasAgg(f)
buf = io.BytesIO()
plt.savefig(buf, format='png')
plt.close(f)
response = HttpResponse(buf.getvalue(), content_type='image/png')
return response
You can just replace the response with a buffer and then add the buffer to the response. This will give an appropriate object to canvas.print_png() and keep code changes to a minimum.
def mplimage(request):
f = matplotlib.figure.Figure()
buf = io.BytesIO()
canvas = FigureCanvasAgg(f)
canvas.print_png(buf)
response=HttpResponse(buf.getvalue(),content_type='image/png')
# if required clear the figure for reuse
f.clear()
# I recommend to add Content-Length for Django
response['Content-Length'] = str(len(response.content))
#
return response
Related
I am trying to save the base64 encoded image in the django rest framework. First of all, we make a code to insert the base64 encoded image into the imagefield and test it, and the following error appears.
binascii.Error: Incorrect padding
What I don't understand is that I've used the same code before and there was no such error. Can you help me? Here is my code.
serializers.py
from rest_framework import serializers
from .models import post, comment
class Base64ImageField (serializers.ImageField) :
def to_internal_value (self, data) :
from django.core.files.base import ContentFile
import base64
import six
import uuid
if isinstance(data, six.string_types):
if 'data:' in data and ';base64,' in data :
header, data = data.split(';base64,')
try :
decoded_file = base64.b64decode(data)
except TypeError :
self.fail('invalid_image')
file_name = str(uuid.uuid4())[:12]
file_extension = self.get_file_extension(file_name, decoded_file)
complete_file_name = "%s.%s" % (file_name, file_extension, )
data = ContentFile(decoded_file, name=complete_file_name)
return super(Base64ImageField, self).to_internal_value(data)
def get_file_extension (self, file_name, decoded_file) :
import imghdr
extension = imghdr.what(file_name, decoded_file)
extension = "jpg" if extension == "jpeg" else extension
return extension
class commentSerializer (serializers.ModelSerializer) :
class Meta :
model = comment
fields = '__all__'
class postSerializer (serializers.ModelSerializer) :
author = serializers.CharField(source='author.username', read_only=True)
image1 = Base64ImageField(use_url=True)
image2 = Base64ImageField(use_url=True)
image3 = Base64ImageField(use_url=True)
image4 = Base64ImageField(use_url=True)
image5 = Base64ImageField(use_url=True)
comment = commentSerializer(many=True, read_only=True)
class Meta:
model = post
fields = ['pk', 'author', 'title', 'text', 'image1', 'image2', 'image3', 'image4', 'image5', 'tag1', 'tag2', 'tag3', 'tag4', 'tag5', 'comment']
I'm not sure this applies to your situation, depending on where you're storing your encoded data.
I had the same error, but it related to some encoded session data. I cleared out the session data (cookies, cache etc) in the browser Devtools, and it fixed my issue.
Just posting this in case it applies or helps others who come along for the same reason.
Run following command in shell
from django.contrib.sessions.models import Session
Session.objects.all().delete()
More information at https://code.djangoproject.com/ticket/31592
I had the same error. I do all things clear cache but it doesn't work. Now change the browser to Mozilla. Now it's working.
I had the same issue. I guess it was caused by me using django 4.0.1 in the beginning and later switching back to django 2.2... (Maybe your issue was not caused by this but I just want to provide some idea on where the problem might be to all the reader who has this issue and visit this page.)
Django 4.0.1 should directly save the session data in string into the database but Django 2.2 saves and reads base64 encoded data into / from the same column session_data in table django_session in DB.
The string failed in base64.b64decode() in my case is .eJxVjEEOwiAQRe_C2pABCgWX7j0DmRlAqoYmpV0Z765NutDtf-_9l4i4rTVuPS9xSuIslDj9boT8yG0H6Y7tNkue27pMJHdFHrTL65zy83K4fwcVe_3WGtkEHfLg2IMroL1VZA0BFGMJPJhRkdEucypY2CfA7C2HgRHGor14fwDNWjfC:1nERxl:5jJRHXpQH7aZrf2-C99MnTIWARd_cUag76Xa2YjW1yw, which is obviously not valid base64 string since symbols : - . . do not exist in base64 character list at all.
My full traceback info is:
Internal Server Error: /admin
Traceback (most recent call last):
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\sessions\backends\base.py", line 189, in _get_session
return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Public\django2.2\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "C:\Users\Public\django2.2\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Users\Public\django2.2\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\admin\sites.py", line 241, in wrapper
return self.admin_view(view, cacheable)(*args, **kwargs)
File "C:\Users\Public\django2.2\lib\site-packages\django\utils\decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "C:\Users\Public\django2.2\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\admin\sites.py", line 212, in inner
if not self.has_permission(request):
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\admin\sites.py", line 186, in has_permission
return request.user.is_active and request.user.is_staff
File "C:\Users\Public\django2.2\lib\site-packages\django\utils\functional.py", line 256, in inner
self._setup()
File "C:\Users\Public\django2.2\lib\site-packages\django\utils\functional.py", line 392, in _setup
self._wrapped = self._setupfunc()
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\auth\middleware.py", line 24, in <lambda>
request.user = SimpleLazyObject(lambda: get_user(request))
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\auth\middleware.py", line 12, in get_user
request._cached_user = auth.get_user(request)
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\auth\__init__.py", line 182, in get_user
user_id = _get_user_session_key(request)
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\auth\__init__.py", line 59, in _get_user_session_key
return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\sessions\backends\base.py", line 54, in __getitem__
return self._session[key]
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\sessions\backends\base.py", line 194, in _get_session
self._session_cache = self.load()
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\sessions\backends\db.py", line 44, in load
return self.decode(s.session_data) if s else {}
File "C:\Users\Public\django2.2\lib\site-packages\django\contrib\sessions\backends\base.py", line 100, in decode
encoded_data = base64.b64decode(session_data.encode('ascii'))
It's possible you run the server in a different Environment, Activate that environment and try to run the server again.
incase the env folder is in your current Path here's what to do
kill the server and then run
source env/bin/activate
python manage.py runserver
Basically, I have this API end point that will be called if you make a POST request to it. The problem is for some reason, I can't convert the bytes to JSON so I can access the data.
My code:
#api_view(['POST'])
def create_user(request):
""" POST = Create user. """
# Check that a username with this email doesn't already exist
try:
data = {}
print("IS IT WORKING...?")
print(type(request.body))
print(request.body)
# Use double quotes to make it valid JSON
my_json = request.body.decode('utf8').replace("'", '"')
print("MY_JSON:")
print(my_json)
data = json.loads(my_json)
print("DATA:")
print(data)
s = json.dumps(data, indent=4, sort_keys=True)
print("s:")
print(s)
except User.DoesNotExist:
print("PLS WORK ON CONSOLE")
return Response(status=status.HTTP_409_CONFLICT)
I try to make a POST request to my path users/create/ using Postman, but when I print request.body to see the contents of my POST request, it is formatted incorrectly with a lot of random numbers and dashes. This is preventing me from converting it to JSON. It's a simple POST request with email and password fields.
This is what the weird formatting looks like: https://gyazo.com/fa1cc2f04637c02f79fe59790153ae40
This is what the "json" looks like after I have decoded it and converted with double quotes (Notice the weird dashes and numbers): https://gyazo.com/3ca41106117a4e9acdd96929469313a1
After that, it ERRORS because of the data = json.loads(my_json) because the input is incorrectly formatted.
Error message:
Internal Server Error: /users/create/
Traceback (most recent call last):
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/rest_framework/views.py", line 505, in dispatch
response = self.handle_exception(exc)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/rest_framework/views.py", line 465, in handle_exception
self.raise_uncaught_exception(exc)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
raise exc
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/rest_framework/views.py", line 502, in dispatch
response = handler(request, *args, **kwargs)
File "/Users/mightu/Desktop/jason_app/env/lib/python3.7/site-packages/rest_framework/decorators.py", line 50, in handler
return func(*args, **kwargs)
File "/Users/mightu/Desktop/jason_app/users/views.py", line 38, in create_user
data = json.loads(my_json)
File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I've used these SO posts (1, 2) to get me to where I am now (realizing my input is just wrong!).
EDIT: OK SO I FOUND THE SOLUTION. In Postman, you can send data as form-data and also as raw. I was using form-data and I guess that just ruined everything and created all the weird -----'s and integers. I switched to raw and directly typed the JSON formatting and my code NOW WORKS! All of it! But I totally don't understand why. I just had a hunch that the method I was sending data was simply incorrect. Can someone explain?
I'm working on a Python/Django/Wagtail project, and I have some api to return some paginated articles. It goes like this:
In URLs:
url(r'^morearticles/', views.get_live_articles),
In Views:
def get_live_articles(request):
context = {'articles': getLiveArticles(request) }
return render(request, 'app/components/articles-live.html', context, content_type="text/html; charset=utf-8")
The getLiveArticles function looks like this:
def getLiveArticles(request):
# Articles
a = get_articles() #this is getting the articles correctly
p = Paginator(a, 4)
page_n = request.GET.get('page')
try:
articles = p.page(page_n)
except Exception, e:
articles = []
return articles
However when hitting the api endpoint I get this:
Traceback (most recent call last):
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/john/app/app/views.py", line 90, in get_live_articles
context = {'articles': getLiveArticles(request) }
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/cache_utils/decorators.py", line 48, in wrapper
cache.set(key, value, timeout, **backend_kwargs)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/cache/backends/locmem.py", line 75, in set
pickled = pickle.dumps(value, pickle.HIGHEST_PROTOCOL)
PicklingError: Can't pickle <class 'wagtail.wagtailcore.blocks.base.RichTextBlockMeta'>: attribute lookup wagtail.wagtailcore.blocks.base.RichTextBlockMeta failed
I've came upon the Pickling error before, but never quite understood what it is about. Any idea of what could be causing this?
Let me know if I should provide more information. After debugging I think the issue has to be contained in these chunks of code, but I might be wrong.
EDIT: One of the fields of the objects has been turned into a StreamField object containing the main content lately. May that have something to do?
I think this is cache related (had same issue lately), where some objects can't be pickled for caching, and as you've confirmed you are using cache. So disabling the cache would solve the problem.
I suggest to use pickle compatible object types. On the other hand in Python documentation there is some guidelines how to implement / customize the pickling / unpickling logic.
Per one of the comments: I did change my code to:
providers = Provider.objects.all()
context = { 'providers':providers}
I know it didn't make a difference but figured I would try it anyway cause stranger things have happened. I am worried the error is within the module itself running on my version of django here.
I did see the other answers on this, and it confused me because I am just using what was documented here:
https://spapas.github.io/2015/11/27/pdf-in-django/#django-integration
for getting the django_xhtml2pdf stuff to work. My view is as such:
def providers_plain_old_view(request):
resp = HttpResponse(content_type='application/pdf')
context = {
'providers': Provider.objects.all()
}
result = generate_pdf('ipaswdb/provider/providers_plain_old_view.html', file_object=resp, context=context)
return result
Which I know now is bad in django 1.11.14 I am using, but no idea how to fix the error:
Traceback (most recent call last):
File "D:\Python27\lib\site-packages\django\core\handlers\exception.py", line 41, in inner
response = get_response(request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "D:\Programming\web\ipa_django\mysite\ipaswdb\views.py", line 312, in providers_plain_old_view
result = generate_pdf('ipaswdb/provider/providers_plain_old_view.html', file_object=resp, context=context)
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 62, in generate_pdf
generate_pdf_template_object(tmpl, file_object, context)
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 39, in generate_pdf_template_object
html = template_object.render(Context(context))
File "D:\Python27\lib\site-packages\django\template\backends\django.py", line 64, in render
context = make_context(context, request, autoescape=self.backend.engine.autoescape)
File "D:\Python27\lib\site-packages\django\template\context.py", line 287, in make_context
raise TypeError('context must be a dict rather than %s.' % context.__class__.__name__)
TypeError: context must be a dict rather than Context.
"GET /ipaswdb/provider_roster/ HTTP/1.1" 500 86485
I mean it wants me to call the generate_pdf function a different way in the latest django version?
The main issue lies in the line
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 39, in generate_pdf_template_object
html = template_object.render(Context(context))
in the error output. This is an issue the django-xhtml2pdf package not being up to date for 1.11. The call to render has changed from
html = template_object.render(Context(context))
to
html = template_object.render(context)
according to the upgrading to 1.11 notes https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/
django.template.loader section.
You can either wait for them to fix it, by submitting a bug report or implement the functionality the package provides in your views.py
I'm trying to implement a simple Django view that accept a file attachment and some other parameters and proxy the request on a remote API call.
Please note: the proxy is not the point of my question :)
This is how I implemented the view:
def image_upload(request):
token = request.POST['token']
image_file = request.FILES.values()[0]
files = {'file': ('myupload.txt', image_file.read())}
client_id = request.POST['client_id']
folder_id = request.POST['folder_id']
advert_id = request.POST['advert_id']
image_type = request.POST['image_type']
crop_image = request.POST['crop_image']
api_base_url = settings.API_BASE_URL
file_post_data = {'client_id': client_id, 'folder_id': folder_id, 'advert_id': advert_id,
'image_type': image_type, 'crop_image': crop_image}
auth_header = {'Authorization': 'Token ' + token}
r = requests.post(api_base_url + 'assets/image/upload/',
data = json.dumps(file_post_data),
headers = auth_header,
files = files)
return r.json()
The problem is that when I test this view (I use Django Test Client to do it) I get an error on the line "files = files)" that says "ValueError: cannot encode objects that are not 2-tuples".
The complete trace log is this one:
======================================================================
ERROR: test_image_upload (fbx.tests.FbxTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/tests.py", line 18, in test_image_upload
'image_type': 'A', 'crop_image': False, 'attachment': fp})
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/views.py", line 42, in image_upload
files = files)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 324, in request
prep = req.prepare()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 225, in prepare
p.prepare_body(self.data, self.files)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 385, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 99, in _encode_files
fields = to_key_val_list(data or {})
File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 136, in to_key_val_list
raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples
I've also tried a quick test using this to read a file: files = {'file': ('myupload.txt', open('/tmp/mytmp.txt', 'rb'))}
but it doesn't work. Do tou have any idea about how to fix this?
Thanks!
You cannot simultaneously post JSON data and multipart/form-data which is in essence what you're trying to do here. json.dumps returns a string so you're sending a string that looks like
'{"client_id": 1, "folder_id": 2, "advert_id": 3, "image_type": "jpeg", "crop_image": true}'
And then telling requests you want to use that in combination with a multipart/form-data request by sending something in via the files parameter. That is impossible and could possibly raise a better exception.
Either add the file to the JSON data after reading it into memory and send that with the appropriate Content-Type header or send the entire thing as a multipart/form-data request without using json.dumps and simply passing the dictionary you're creating to data. Use one or the other but not both.
That aside, you say that your exception comes from the line that only has files=files) on it, but it is not that line alone that causes the exception. The exception is raised by the function which happens to end on that line. The fact that your exception arises from that too is mere coincidence. This is an wart in Python that may possibly be fixed in Python 3.4. You should upgrade, because 3.4 will be awesome and newer versions of Django support Python 3.x.