UTF-8 errors in PowerShell and CMD - python

I'm trying to execute a script on a schedule it's coded in utf-8 at the beginning of the file:
#-*- coding: utf-8 -*- .
I've set Task Scheduler in windows to run a batch file that calls the py file for execution.
python C:/Users/admin_4190248/Desktop/Howard/redshift_howard.py
However, when the script runs in either CMD or PowerShell, it throws a "can't encode error" like this:
UnicodeDecodeError: 'ascii' codec can't encode character u'\u2013'...
Which I don't understand, since the file executes fine in an ipython console.
My question is therefore, is there another (more ipython like) tool that I can use to execute my script on a schedule? Or, can you help to solve this UTF-8 issue within CMD?

Related

Output ascii characters to stdout in Python 3

I have a file named 'xxx.py' like this:
print("a simple string")
and when I run that like this (Python 3):
python xxx.py >atextfile.txt
I get a unicode file.
I would like an ascii file.
I don't mind if an exception is thrown if a non-ascii character is attempted to be printed.
What is a simple change I can make to my code that will output ascii characters?
My searches turn up solutions that all seem too verbose for such a simple problem.
[Edit] to report what I learned from setting LC_CTYPE:
I am running on windows 7.
When running on the powershell commandline I get a unicode file (two bytes/character)
When running in a .bat file without LC_CTYPE set I get an ascii file (could be utf-8 as #jwodder pointed out).
When running in a .bat file with LC_CTYPE=ascii set I get presumable an ascii file (1 byte/character).
The stdout encoding is defined by the environment that is executing the python script, e.g.:
$ python -c "import sys; print(sys.stdout.encoding)"
UTF-8
$ LC_CTYPE=ascii python -c "import sys; print(sys.stdout.encoding)"
US-ASCII
Try adjusting your environment before running the script. You can force the encoding value for Python by setting the PYTHONIOENCODING environment variable.

Encoding trouble when run python application using autorun

I'm trying to run my kivy app using autorun on my Raspberry Pi.
After restarting my OS it will run but during execution I faced up with encoding problem on the next lines of code:
CommonData.deviceSettings.Measurement.Alchogol = {}
for s in alchogolSettings:
key = s["Name"].encode('utf-8').strip()
value = s["Value"].encode('utf-8').strip()
CommonData.deviceSettings.Measurement.Alchogol.update({key: value})
The next error occured during execution
'ascii' codec can't encode characters in position 0-14: ordinal not in
range(128)
In the top of my .py file I setup next instructions:
#!/usr/bin/python
# -*- coding: utf8 -*-
The most interesting in this situation that if I will run this program from usual terminal it launch successfully, but when I'm trying to launch it using autorun this problem occurs
Does anybody know why this problem occurs and how to avoid it?
I found the reason of this problem. In my case I run a python script which is launch terminal and start another python script. The last script print some cyrrilic text to the terminal. This is where the problem lies. After deleting this print instruction I avoided this error. And this print instruction was located one line after the code I showed in this post

SyntaxError: Non-UTF-8 code starting with '\xc4'

Problem while importing module
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import win32com.client
when I run it in eclipse, syntaxError occurs.
but it runs perfectly on Windows console.
how to type the right coding of pywin32?
For eclipse unicode console support:
Add -Dfile.encoding=UTF-8 to eclipse.ini which is in the eclipse install directory.
In eclipse – Run\Run Configurations\Python Run\configuration\Common\make sure UTF-8 is selected
In eclipse – Window\Preferences\General\Workspace\Text file encoding\making sure UTF-8 is selected
In [python install path]\Lib\site.py – chane from encoding = “ascii” to encoding = “utf-8”
Make sure you’re using unicode supporting fonts in eclipse – Window\Preferences\Appearance\Colors and Fonts\Debug\Console font\Edit

Windows CMD line errors in python for foreign language

I am downloading data from a MySQL database. Some of the data is in Korean. When I try to print the string before putting it in a table (Qt), the windows command prompt returns:
File "C:\Python27\lib\encodings\cp437.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2: character maps to (undefined)
However, when I use IDLE to run the code, it prints the Korean language fine. This caused me alot of headache when trying to debug why my program was not working as I just click the python file from the folder to run it. Finally when using idle it turned out everything works.
Is there something wrong with my python installation, windows installation, or python code trying to just print the characters? I assumed it wouldnt be the python code as it works in IDLE. Also, using a special function to print in windows seems bad as it limits the codes portability to another OS (or will every OS have this problem?)
IDLE is based on tkinter, which is based on tcl/tk, which supports the entire Basic Multilingual Plane (BMP). (But tcl/tk does not support supplementary planes with other characters). On Windows, the Python interactive interpreter runs in the same console window used by Command Prompt. This only supports Code Page subsets of the BMP, sometimes only 256 of 2^^16 characters.
The codepage that supports ASCII and Korean is 949. (Easy Google search.) In Command Prompt, chcp 949 should change to that codepage. If you then start Python, you should be able to display Korean characters.

Encoding difference in Eclipse and Windows console

I have a Python script which works perfectly in Eclipse Console (Run configuration).
When I try to launch this script on a Windows 7 console, I have the encoding error:
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc9' in position 0: ordinal not in range(128)
I changde the code page of my Windows console to use the same one as in Eclipse (Window->Perference->General->Worspace->Text file encoding):
chcp 1252
At the beginning of the script, I add:
# -*- coding: cp1252 -*-
But it changes nothing.
It works on Eclipse console, so I do not want to decode/encode all my strings for Windows console.
Have you any idea or advice to fix that behaviour?
You could try setting both eclipse's and the windows cmd line's encodings to Utf-8 and see if that works, unless you absolutely need the cp1252 encoding.
The issue is that Python will expect your 8 bit strings to contain ASCII only, not Unicode. u'\xc9 is a Unicode character. Perhaps Eclipse is more friendly than Windows 7 console. You should use the unicode command to convert characters to Unicode as you get them:
value = unicode(value, "utf-8")
See this article for more.

Categories