I've tried this SO answer, this doc is inapplicable as I'm running nginx, I've added charset utf-8; to my nginx config and I'm still getting this error.
Summarised traceback is here:
UnicodeEncodeError at /
'ascii' codec can't encode character u'\xe1' in position 69: ordinal not in range(128)
Request Method: GET
Request URL: http://django/
Django Version: 1.4.20
Exception Type: UnicodeEncodeError
Exception Value:
'ascii' codec can't encode character u'\xe1' in position 69: ordinal not in range(128)
Exception Location: /opt/envs/venv/lib/python2.7/genericpath.py in getmtime, line 54
Unicode error hint
The string that could not be encoded/decoded was: choacán.jpg
I think this error is not about nginx. It's on the file creation step.
Python uses system locale when saving files.
Check your system locale:
$ python manage.py shell
> import os
> print os.popen("locale").read()
If it's incorrect you should set system locale.
But filenames like this can cause any kind of troubles for users. Please think about defining custom file storage for models.FileField and generating random file name for every file - it's good practice.
Related
I use Robot Framework with ride to do python-auto-test, but I got this problem in Robot Framework console:
"FAILED D:\eclipse\oppoAutoTest\AutoTest\../../PyAutoTest/src/testTcardCopy.py
FAILED D:\eclipse\oppoAutoTest\AutoTest\commonUserKeyWords\../../../PyAutoTest/src/testMkDir.py
FAILED ../../PyAutoTest/basic/testSwipeScreen.py 'ascii' codec can't decode byte 0xd7 in position 1: ordinal not in range(128)"
Can you help me ?
This is my Robot Framework code:
*** Settings ***
Library
../../eclipse_workplace_java/PyAutoTest/src/testTcardCopy.py
Resource commonUserKeyWords/BASIC.tsv
Resource commonUserKeyWords/CAMERA.tsv
*** Test Cases ***
bspTcardTest
${test_v_file_dir_path} set Variable /storage/sdcard0/tmp
${test_v_file_dir_ifexitd_result} ${test_v_file_dir_if_success}
${test_v_file_dir_stdout} ${test_v_file_dir_stdout_mount} 判断文件或文件
夹是否存在 ${test_v_file_dir_path}
log ${test_v_file_dir_ifexitd_result}
log ${test_v_file_dir_if_success}
log ${test_v_file_dir_stdout}
log ${test_v_file_dir_stdout_mount}
Should_Be_True '${test_v_file_dir_if_success}'=='True'
My python code also has utf-8:
# coding=utf-8
Can't solve typical issue with encodings. Cyrrlic text is received via post and error is raised
'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)
The text key itself has a look and it must cyrillic text in russian: u'\u043f\u0440\u043e'
After that error tried this way and some others:
key = key.decode('ascii').encode('utf8')
or :
key = key.decode('ascii')
Localy it works, error is raised in production only. Python system encoding in production is utf8
EDIT: in order to clear things up. Error is raised on form handler function(again, works localy, doesn't in production)
def search(request):
if request.method == 'POST':
key = request.POST.get("key")
if key is not None:
..
So it's str received from input form, and error is first raised at this point, so I supposed it must be decoded but it didn't help.
more traceback:
UnicodeEncodeError at /search/
'ascii' codec can't encode characters in position 0-2: ordinal not in range(128)
The error happened when I add record in admin. My model is like
class THMusic(models.Model):
originmusic = models.ForeignKey(Originalmusic)
name = models.CharField(max_length=100)
audiofile = models.FilePathField(path=os.path.dirname(os.path.abspath(__file__))+'/resource', recursive=True)
team = models.CharField(max_length=50)
album = models.CharField(max_length=50)
class THMusicAdmin(admin.ModelAdmin):
list_display = ('name', 'originmusic', )
But I get UnicodeDecodeError:
UnicodeDecodeError at /admin/thmusic/thmusic/add/
'ascii' codec can't decode byte 0xe3 in position 54: ordinal not in range(128)
Request Method: GET
Request URL: http://127.0.0.1:8006/admin/thmusic/thmusic/add/
Django Version: 1.6.5
Exception Type: UnicodeDecodeError
Exception Value:
'ascii' codec can't decode byte 0xe3 in position 54: ordinal not in range(128)
Exception Location: /usr/local/lib/python2.7/dist-packages/django/forms/fields.py in __init__, line 1057
Python Executable: /usr/bin/python2.7
Python Version: 2.7.6
Python Path:
['/home/hyzhappy/djangoprojects/myblog',
'/usr/local/lib/python2.7/dist-packages/simplejson-3.6.3-py2.7-linux-i686.egg',
'/home/hyzhappy/djangoprojects/myblog',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-i386-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/local/lib/python2.7/dist-packages/PIL',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PILcompat',
'/usr/lib/python2.7/dist-packages/gst-0.10',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/lib/python2.7/dist-packages/ubuntu-sso-client',
'/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']
If I delete "recursice=True" in FilePathField, it will go well. But it's not my expectation
More detail
Unicode error hint
The string that could not be encoded/decoded was: urce/������
UnicodeDecodeError('ascii', '/home/hyzhappy/djangoprojects/myblog/thmusic/resource/\xe3\x83\x97\xe3\x83\xac\xe3\x82\xa4\xe3\x83\xa4\xe3\x83\xbc\xe3\x82\xba\xe3\x82\xb9\xe3\x82\xb3\xe3\x82\xa2.mp3', 54, 55, 'ordinal not in range(128)')
Probably not the right answer, but you can check this as a possible cause.
Sometimes in Django (I had the same case with tastypie), the UnicodeDecodeError occurs because of an initial unrelated error which tries to display but can not because of weird character.
In my case, it was because I put French accents in my code comments (not ascii). This usually does not create errors, but when there is actually an error somewhere and Django tries to display this error in the debug mode, it can not because of the non ascii chars and raises a UnicodeDecodeError instead of raising the initial error.
Just try to remove any suspicious character in your code (if there is any), and check again to see if the error is still the same.
Under the hood, FilePathField uses os.walk/os.listdir to find the files. These functions have two "modes" of operation in python: byte string and unicode. The mode is selected by the type of path given, i.e. if you feed it an unicode string, the resulting file list will be unicode, otherwise it will be a byte string.
So, to fix this problem, just feed a unicode path to the FilePathField, like this (note the u at the beginning of u'/resource'):
audiofile = models.FilePathField(path=os.path.dirname(os.path.abspath(__file__))+u'/resource', recursive=True)
Django 1.6.1
Python 3.2.3
Apache 2.2.2
Ubuntu 12.04
Postgres 9.1.11
I'm getting this error when trying to import a serialized XML file. I exported it from one database that I was using for a development server, and moments later I import it to another database on a different development server (on the same machine). I'm very baffled as to why it's not handling the UTF-8 characters on import. I didn't do anything special. Plus, the same code worked when I last tested it. Here's the code I'm using to export & import, in an app's views.py...
from django.shortcuts import render, render_to_response, get_object_or_404
from django.http import HttpResponseRedirect, HttpResponse
from django.contrib.auth.decorators import user_passes_test
from django.core import serializers
#user_passes_test(lambda u: u.is_superuser)
def export_serial(request):
'''
This exports data using Django serializing.
See: https://docs.djangoproject.com/en/dev/topics/serialization/
'''
XMLSerializer = serializers.get_serializer("xml")
xml_serializer = XMLSerializer()
xml_serializer.serialize(list(ThisModel.objects.all())
+ list(ThatModel.objects.all())
+ list(ThirdModel.objects.all()))
data = xml_serializer.getvalue()
response = HttpResponse(mimetype='text/xml')
response['Content-Disposition'] = 'attachment; filename=my-backup.xml'
response.write(data)
return response
#user_passes_test(lambda u: u.is_superuser)
def import_serial(request):
'''
This is for importing data that was saved/backed up, as serialized.
'''
xmlfile = '/home/zamphatta/path/to/my-backup.xml'
serialfile = open(xmlfile,'r')
data = serialfile.read()
serialfile.close()
for deserialized_object in serializers.deserialize("xml", data):
deserialized_object.save()
template = loader.get_template('z-style/index.htm')
context = RequestContext(request, {
'title': data,
})
return HttpResponse(template.render(context))
I Googled & I checked out all StackOverflow questions that were related (but the answers don't clue me in to an answer for this), and still don't have a clue why this is throwing me this error. All I know is that non-ascii characters are making it puke, which doesn't make sense 'cause Python has no problem with UTF-8 characters and I'm not doing any weird stuff that would cause the code to suddenly treat it differently.
UPDATE: Here's a fuller version of the error message...
UnicodeDecodeError at /sdb/import/
'ascii' codec can't decode byte 0xe2 in position 5367: ordinal not in range(128)
Request Method: GET
Request URL: http://coral/sdb/import/
Django Version: 1.6.1
Exception Type: UnicodeDecodeError
Exception Value:
'ascii' codec can't decode byte 0xe2 in position 5367: ordinal not in range(128)
Exception Location: /usr/lib/python3.2/encodings/ascii.py in decode, line 26
Python Executable: /usr/bin/python
Python Version: 3.2.3
Python Path:
['/usr/local/lib/python3.2/dist-packages/distribute-0.6.49-py3.2.egg',
'/usr/local/lib/python3.2/dist-packages/CherryPy-3.2.4-py3.2.egg',
'/usr/lib/python3.2',
'/usr/lib/python3.2/plat-linux2',
'/usr/lib/python3.2/lib-dynload',
'/usr/local/lib/python3.2/dist-packages',
'/usr/local/lib/python3.2/dist-packages/setuptools-0.6c11-py3.2.egg-info',
'/usr/lib/python3/dist-packages',
'/home/dfy/code/zamphatta/',
'/home/dfy/code/zamphatta/zamphatta/']
Seems when the server was started up python3 manage.py runserver, the import would work just fine. It wasn't working when I was importing it through Django when Django was running on Apache via wsgi. So I'm guessing Apache was somehow interfering with it. It works now, as long as I import straight through Django.
I am getting this error in Django:
UnicodeDecodeError at /category/list/
'utf8' codec can't decode byte 0xf5 in position 7: invalid start byte
Request Method: GET
Request URL: ...
Django Version: 1.3.1
Exception Type: UnicodeDecodeError
Exception Value:
'utf8' codec can't decode byte 0xf5 in position 7: invalid start byte
Exception Location: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py in iterencode, line 264
...
I should save Turkish characters in the database. How can I fix this error?
A start-byte of 0xf5 would indicate the start of a 4-character UTF-8 encoding. One strong possibility is that the input isn't UTF-8 at all but ISO-8859-9, the Turkish ISO encoding. On that codepage 0xf5 is a lowercase o with tilde or õ.
Below code solved my problem. Thank you.
if isinstance(encObject, unicode):
myStr = encObject.encode('utf-8')
http://www.fileformat.info/info/unicode/char/f5/index.htm
it is an o with a tilde
try
some_string.decode('latin1','replace')