I'm working on a Python/Django/Wagtail project, and I have some api to return some paginated articles. It goes like this:
In URLs:
url(r'^morearticles/', views.get_live_articles),
In Views:
def get_live_articles(request):
context = {'articles': getLiveArticles(request) }
return render(request, 'app/components/articles-live.html', context, content_type="text/html; charset=utf-8")
The getLiveArticles function looks like this:
def getLiveArticles(request):
# Articles
a = get_articles() #this is getting the articles correctly
p = Paginator(a, 4)
page_n = request.GET.get('page')
try:
articles = p.page(page_n)
except Exception, e:
articles = []
return articles
However when hitting the api endpoint I get this:
Traceback (most recent call last):
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/john/app/app/views.py", line 90, in get_live_articles
context = {'articles': getLiveArticles(request) }
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/cache_utils/decorators.py", line 48, in wrapper
cache.set(key, value, timeout, **backend_kwargs)
File "/Users/john/.virtualenvs/upgrade/lib/python2.7/site-packages/django/core/cache/backends/locmem.py", line 75, in set
pickled = pickle.dumps(value, pickle.HIGHEST_PROTOCOL)
PicklingError: Can't pickle <class 'wagtail.wagtailcore.blocks.base.RichTextBlockMeta'>: attribute lookup wagtail.wagtailcore.blocks.base.RichTextBlockMeta failed
I've came upon the Pickling error before, but never quite understood what it is about. Any idea of what could be causing this?
Let me know if I should provide more information. After debugging I think the issue has to be contained in these chunks of code, but I might be wrong.
EDIT: One of the fields of the objects has been turned into a StreamField object containing the main content lately. May that have something to do?
I think this is cache related (had same issue lately), where some objects can't be pickled for caching, and as you've confirmed you are using cache. So disabling the cache would solve the problem.
I suggest to use pickle compatible object types. On the other hand in Python documentation there is some guidelines how to implement / customize the pickling / unpickling logic.
Related
I am now writing tests for my web application in Django. I have an URL 127.0.0.1/home/device/(?P<item>[^/]+). I was testing an invalid URL path.
Here item is a device name in the database. Consider an invalid device and while testing I have given the following code:
response=self.client.get("/health/errorfiles/",{'item':'testdevice_R'})
This give me a 404 response. The same thing I have tried with:
response=self.client.get("/health/errorfiles/testdevice_R/")
But this time, the test runner executes my view function and gives a TypeError since it is not an invalid device.
In both of the methods, which one is correct usage for a get request?
views.py
def showfiles(request,item):
if request.user.is_anonymous(): ## check if the user is valid
return HttpResponseRedirect("/")
db = MySQLdb.connect(host='localhost',user=root,passwd='123mcvis',db='test_db')
s = db.cursor()
username=request.user
s.execute("Select tzone,type from device where user='%s' and device='%s'"%(username,item)
tz=s.fetchone()
timezone.activate(tz[0])
s.close()
return render(request,"Home/showdevices.html")
This is my view function and since the device is invalid, it shows typ error.
class showfile_test(TestCase):
def test_invalid(self):
response=self.client.get("/health/errorfiles/testdevice_R/")
self.assertEqual(response.status_code,404)
Traceback
ERROR: test_invalid (HealthApp.tests2.showfile_test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vishnumc/vishnu/project/django/official/version6.2.3/HealthApp/tests2.py", line 162, in test_showfiles_with_invalid_deviceid
response=self.client.get("/health/errorfiles/invaliddevice/")
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 529, in get
**extra)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 333, in get
return self.generic('GET', path, secure=secure, **r)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 409, in generic
return self.request(**r)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 494, in request
six.reraise(*exc_info)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/vishnumc/vishnu/project/django/official/version6.2.3/HealthApp/views.py", line 118, in showfiles
timezone.activate(tz[0]) ##setting current timezone to user's time zone for display timestamps
TypeError: 'NoneType' object has no attribute '__getitem__'
The first example is a GET request to /health/errorfiles/?item=testdevice_R. In the view, you would then find item in request.GET.
The second example is a GET request to /health/errorfiles/testdevice_R/. There is no data in request.GET and Django will pass item to your view since you have a named group (?P<item>[^/]+) in your URL pattern.
You should use the second version, because you want to test the URL pattern r'/home/device/(?P<item>[^/]+)'.
The second version of your test has uncovered problems in your view. You need to fix the view so that it doesn't raise TypeError.
Ideally, you shouldn't be writing raw SQL like that. Django allows you to do something like the following:
from .models import Device
from django.shortcuts import get_object_or_404, render
def showhome(request, item):
device = get_object_or_404(Device, user=request.user, item=item)
if device is not None:
timezone.activate(device.tzone)
return render(request,"Home/showdevices.html", {'device': device})
If you must use raw SQL then don't use string substitution %(username,item). Your current code exposes you to an SQL injection account. You should change it to:
s.execute("Select tzone,type from device where user=%s" and device=%s, (username, item))
Your code then has handle the case where tz is None, to avoid the TypeError.
tz = s.fetchone()
if tz is not None:
timezone.activate(tz[0])
else:
# Decide what to do if no items returned
Per one of the comments: I did change my code to:
providers = Provider.objects.all()
context = { 'providers':providers}
I know it didn't make a difference but figured I would try it anyway cause stranger things have happened. I am worried the error is within the module itself running on my version of django here.
I did see the other answers on this, and it confused me because I am just using what was documented here:
https://spapas.github.io/2015/11/27/pdf-in-django/#django-integration
for getting the django_xhtml2pdf stuff to work. My view is as such:
def providers_plain_old_view(request):
resp = HttpResponse(content_type='application/pdf')
context = {
'providers': Provider.objects.all()
}
result = generate_pdf('ipaswdb/provider/providers_plain_old_view.html', file_object=resp, context=context)
return result
Which I know now is bad in django 1.11.14 I am using, but no idea how to fix the error:
Traceback (most recent call last):
File "D:\Python27\lib\site-packages\django\core\handlers\exception.py", line 41, in inner
response = get_response(request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "D:\Python27\lib\site-packages\django\core\handlers\base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "D:\Programming\web\ipa_django\mysite\ipaswdb\views.py", line 312, in providers_plain_old_view
result = generate_pdf('ipaswdb/provider/providers_plain_old_view.html', file_object=resp, context=context)
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 62, in generate_pdf
generate_pdf_template_object(tmpl, file_object, context)
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 39, in generate_pdf_template_object
html = template_object.render(Context(context))
File "D:\Python27\lib\site-packages\django\template\backends\django.py", line 64, in render
context = make_context(context, request, autoescape=self.backend.engine.autoescape)
File "D:\Python27\lib\site-packages\django\template\context.py", line 287, in make_context
raise TypeError('context must be a dict rather than %s.' % context.__class__.__name__)
TypeError: context must be a dict rather than Context.
"GET /ipaswdb/provider_roster/ HTTP/1.1" 500 86485
I mean it wants me to call the generate_pdf function a different way in the latest django version?
The main issue lies in the line
File "D:\Python27\lib\site-packages\django_xhtml2pdf\utils.py", line 39, in generate_pdf_template_object
html = template_object.render(Context(context))
in the error output. This is an issue the django-xhtml2pdf package not being up to date for 1.11. The call to render has changed from
html = template_object.render(Context(context))
to
html = template_object.render(context)
according to the upgrading to 1.11 notes https://docs.djangoproject.com/en/1.11/ref/templates/upgrading/
django.template.loader section.
You can either wait for them to fix it, by submitting a bug report or implement the functionality the package provides in your views.py
I'm trying to run some tests for my Django-Rest-Framework API but am stuck on an error. When I run the following tests, I get the following errors.
Traceback (most recent call last):
File "C:\Users\Bill\SD\DjangoApps\vidapp\startapp\tests.py", line 21, in test_get_user
response = self.client.get('/user/1/')
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 473, in get
response = super(Client, self).get(path, data=data, **extra)
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 280, in get
return self.request(**r)
File "C:\Anaconda\lib\site-packages\rest_framework\test.py", line 143, in request
return super(APIClient, self).request(**kwargs)
File "C:\Anaconda\lib\site-packages\rest_framework\test.py", line 95, in request
request = super(APIRequestFactory, self).request(**kwargs)
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 444, in request
six.reraise(*exc_info)
File "C:\Anaconda\lib\site-packages\django\core\handlers\base.py", line 114, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
TypeError: __init__() takes exactly 1 argument (2 given)
Test Cases:
class UserTestCase(APITestCase):
def setUp(self):
helper.reset_test_db()
def test_get_user(self):
response = self.client.get('/user/1/')
print response.content
self.assertEqual(response.data, {'fname':'Generic','lname':'Name','token':'token1'})
URL Config:
url(r'^user/new/$', 'startapp.views.new_user'),
url(r'^user/1/$', 'startapp.views.get_1'),
Views:
class get_1(APIView):
def get(self, request):
user = db_models.User.objects.get(pk=1)
if(user is not None):
serial_user = serial.UserSerializer(user)
return Response(serial_user.data)
else:
return Response(status.HTTP_404_NOT_FOUND)
I know the view itself works because I tested that separately. The data is definitely present since helper.reset_test_db() puts it there (I know I should be using fixtures but this is for testing so I went with the simple route). The same error occurs for POST and other commands or when I use Django's TestCase instead of APITestCase. While this is my first time using Django's TestCase, I read both the Django and Django rest documents but can't seem to figure out this issue.
The view in your case is a class-based view.
So you have to add it to the urlconfig with as_view:
url(r'^user/1/$', startapp.views.get_1.as_view()),
I'm trying to implement a simple Django view that accept a file attachment and some other parameters and proxy the request on a remote API call.
Please note: the proxy is not the point of my question :)
This is how I implemented the view:
def image_upload(request):
token = request.POST['token']
image_file = request.FILES.values()[0]
files = {'file': ('myupload.txt', image_file.read())}
client_id = request.POST['client_id']
folder_id = request.POST['folder_id']
advert_id = request.POST['advert_id']
image_type = request.POST['image_type']
crop_image = request.POST['crop_image']
api_base_url = settings.API_BASE_URL
file_post_data = {'client_id': client_id, 'folder_id': folder_id, 'advert_id': advert_id,
'image_type': image_type, 'crop_image': crop_image}
auth_header = {'Authorization': 'Token ' + token}
r = requests.post(api_base_url + 'assets/image/upload/',
data = json.dumps(file_post_data),
headers = auth_header,
files = files)
return r.json()
The problem is that when I test this view (I use Django Test Client to do it) I get an error on the line "files = files)" that says "ValueError: cannot encode objects that are not 2-tuples".
The complete trace log is this one:
======================================================================
ERROR: test_image_upload (fbx.tests.FbxTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/tests.py", line 18, in test_image_upload
'image_type': 'A', 'crop_image': False, 'attachment': fp})
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/andrea/Documents/src/fbxapp/onboard/fbx/views.py", line 42, in image_upload
files = files)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 324, in request
prep = req.prepare()
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 225, in prepare
p.prepare_body(self.data, self.files)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 385, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 99, in _encode_files
fields = to_key_val_list(data or {})
File "/usr/local/lib/python2.7/dist-packages/requests/utils.py", line 136, in to_key_val_list
raise ValueError('cannot encode objects that are not 2-tuples')
ValueError: cannot encode objects that are not 2-tuples
I've also tried a quick test using this to read a file: files = {'file': ('myupload.txt', open('/tmp/mytmp.txt', 'rb'))}
but it doesn't work. Do tou have any idea about how to fix this?
Thanks!
You cannot simultaneously post JSON data and multipart/form-data which is in essence what you're trying to do here. json.dumps returns a string so you're sending a string that looks like
'{"client_id": 1, "folder_id": 2, "advert_id": 3, "image_type": "jpeg", "crop_image": true}'
And then telling requests you want to use that in combination with a multipart/form-data request by sending something in via the files parameter. That is impossible and could possibly raise a better exception.
Either add the file to the JSON data after reading it into memory and send that with the appropriate Content-Type header or send the entire thing as a multipart/form-data request without using json.dumps and simply passing the dictionary you're creating to data. Use one or the other but not both.
That aside, you say that your exception comes from the line that only has files=files) on it, but it is not that line alone that causes the exception. The exception is raised by the function which happens to end on that line. The fact that your exception arises from that too is mere coincidence. This is an wart in Python that may possibly be fixed in Python 3.4. You should upgrade, because 3.4 will be awesome and newer versions of Django support Python 3.x.
I'm working on a simple webservice in django. This is my first web app in django/python so I wouldn't be surprised if I'm missing something obvious here, but...
I'm currently trying to test my logic for filtering a url.
# Works as expected
response = self.client.post("/mysite/goodurl/")
self.assertEqual(response.status_code, 200)
# Has an exception rather than a 404
response = self.client.post("/mysite/badurl/")
self.assertEqual(response.status_code, 404)
So, the badurl case isn't simply being not found and throwing a 404, instead I'm getting this error:
Traceback (most recent call last):
File "/home/user/me/mysite/tests.py", line 55, in test_add_tracker
response = self.client.post("/mysite/badurl/")
File "/home/path/to/some/bin/dir/freeware/Python/lib/python2.7/site-packages/django/test/client.py", line 449, in post
response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
File "/home/path/to/some/bin/dir/freeware/Python/lib/python2.7/site-packages/django/test/client.py", line 262, in post
return self.request(**r)
File "/home/path/to/some/bin/dir/freeware/Python/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/home/path/to/some/bin/dir/freeware/Python/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 77, in wrapped_view
return view_func(*args, **kwargs)
TypeError: EtimeFetcher() got an unexpected keyword argument 'alias'
I've tried googling for the EtimeFetcher message, but no luck. Any ideas?
Uou may probably have some view to catch all Http404 errors. It seems that Django however had found a view to execute for the /badurl. Search in your code for statements containing the alias named parameter, like: "alias=yyyy". Or probably some url pattern, contaning 'alias' as extra parameter in urls.py?