We Keep Coding

How to solve UnicodeDecodeError in Python 3.6?

I am switched from Python 2.7 to Python 3.6.
I have scripts that deal with some non-English content.
I usually run scripts via Cron and also in Terminal.
I had UnicodeDecodeError in my Python 2.7 scripts and I solved by this.
# encoding=utf8
import sys
reload(sys)
sys.setdefaultencoding('utf8')
Now in Python 3.6, it doesnt work. I have print statements like print("Here %s" % (myvar)) and it throws error. I can solve this issue by replacing it to myvar.encode("utf-8") but I don't want to write with each print statement.
I did PYTHONIOENCODING=utf-8 in my terminal and I have still that issue.
Is there a cleaner way to solve UnicodeDecodeError issue in Python 3.6?
is there any way to tell Python3 to print everything in utf-8? just like I did in Python2?

It sounds like your locale is broken and have another bytes->Unicode issue. The thing you did for Python 2.7 is a hack that only masked the real problem (there's a reason why you have to reload sys to make it work).
To fix your locale, try typing locale from the command line. It should look something like:
LANG=en_GB.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_ALL=
locale depends on LANG being set properly. Python effectively uses locale to work out what encoding to use when writing to stdout in. If it can't work it out, it defaults to ASCII.
You should first attempt to fix your locale. If locale errors, make sure you've installed the correct language pack for your region.
If all else fails, you can always fix Python by setting PYTHONIOENCODING=UTF-8. This should be used as a last resort as you'll be masking problems once again.
If Python is still throwing an error after setting PYTHONIOENCODING then please update your question with the stacktrace. Chances are you've got an implied conversion going on.

I had this issue when using Python inside a Docker container based on Ubuntu 18.04.
It appeared to be a locale issue, which was solved by adding the following to the Dockerfile:
ENV LANG C.UTF-8

To everyone using pickle to load a file previously saved in python 2 and getting an UnicodeDecodeError, try setting pickle encoding parameter:
with open("./data.pkl", "rb") as data_file:
samples = pickle.load(data_file, encoding='latin1')

For a Python-only solution you will have to recreate your sys.stdout object:
import sys, codecs
sys.stdout = codecs.getwriter('utf-8')(sys.stdout.detach())
After this, a normal print("hello world") should be encoded to UTF-8 automatically.
But you should try to find out why your terminal is set to such a strange encoding (which Python just tries to adopt to). Maybe your operating system is configured wrong somehow.
EDIT: In my tests unsetting the env variable LANG produced this strange setting for the stdout encoding for me:
LANG= python3
import sys
sys.stdout.encoding
printed 'ANSI_X3.4-1968'.
So I guess you might want to set your LANG to something like
en_US.UTF-8. Your terminal program doesn't seem to do this.

Python 3 (including 3.6) is already Unicode supported. Here is the doc - https://docs.python.org/3/howto/unicode.html
So you don't need to force Unicode support like Python 2.7. Try to run your code normally. If you get any error reading a Unicode text file you need to use the encoding='utf-8' parameter while reading the file.

for docker with python3.6, use LANG=C.UTF-8 python or jupyter xxx works for me, thanks to #Daniel and #zhy

I mean you could write an custom function like this:
(Not optimal i know)
import sys
def printUTF8(input):
print(input.encode("utf-8"))

Python Import Script Naming Errors

So I have a python script calling some other pythons scripts in the working directory. I usually use naming conventions like v1.0.3_ModuleName.py to keep track of newer versus older versions of my script. When I tried to import my module:
import v1.0.3_ModuleName
I recieved the good ole: SyntaxError: invalid syntax error. Now I realized my error quickly and took out the periods.
This make me wonder, what other file names with result in errors when you try to import them into python?

If you're using linux you could make a symbolic link to your module that doesn't include dots and numbers :) But this isn't a portable solution.
More about module naming.

Python 2.7: "unresolved import: ConfigParser"

I recently wrote a Python 2.7 script (using PyDev on Eclipse) that took advantage of the built-in ConfigParser module, and the script works perfectly. But when I exported it and sent it to a colleague, he could not get it to work. He keeps getting an "unresolved import: ConfigParser" error even though we are using the exact same settings. This isn't supposed to happen as ConfigParser is built-in.
I've Googled everywhere but could not seem to find any working solution. Any help would be appreciated.

ConfigParser was renamed to configparser in python 3. Chances are he's using 3 and cannot find the old py2 name.
You can use:
try:
import configparser as ConfigParser
except ImportError:
import ConfigParser

To see what's happening it may be nice comparing on both computers which sys.path is being used (i.e.: put at the start of the module being run the code below and compare the output in each case):
import sys
print '\n'.join(sorted(sys.path))
Now, if the error is not when running the code (i.e.: it runs fine and you get no exceptions), and he gets the error only in PyDev, probably the interpreter configuration in his side is not correct and one of the paths printed through the command above is not being added to the PYTHONPATH (it could be that he's on a virtual env and didn't add the paths to the original /Lib or has added some path that shouldn't be there -- or even has some ConfigParser module somewhere else which is conflicting with the one from the Python standard library).

Python japanese module is not found

I run following python script.
pygame2exe.py
ImportError: No module named japanese
What's wrong?
Do not you know solutions?

The script makes use of japanese encoding
# -*- coding: sjis -*-
[...]
args.append('japanese,encodings');
It's a shame cause it could use UTF-8 that works out of the box.
You can't run this script unless you install the japanese module. I can't find any reference of it on the web, and I can read in the code :
# make standalone, needs at least pygame-1.5.3 and py2exe-0.3.1
# fixed for py2exe-0.6.x by RyoN3 at 03/15/2006
If you haven't installed the last version of pygame and py2exe, I would start by that since they may embed the module you need.

To add to e-satis' explanation, the "japanese" module is provided by the Japan PUG, but I don't think you've actually needed it since around Python 2.2. I believe that all the Japanese codecs are included in a standard Python install these days. I certainly don't use this module, and I handle SJIS in my programs just fine.
So I think you could just get rid if the forced import, and do fine. That is, delete these lines:
args.append('-p')
args.append('japanese,encodings') # JapaneseCodecを強制的に含める
Since you don't have the "japanese" module on your system, if the program runs OK on your system, then the frozen version should be OK without this module.
However, I would recommend using Unicode throughout instead of byte strings, and if you insist on byte strings, I'd at least put them in UTF-8.

Is there any way to get python omnicomplete to work with non-system modules in vim?

The only thing I can get python omnicomplete to work with are system modules. I get nothing for help with modules in my site-packages or modules that I'm currently working on.

Once I generated ctags for one of my site-packages, it started working for that package -- so I'm guessing that the omnicomplete function depends on ctags for non-sys modules.
EDIT: Not true at all.
Here's the problem -- poor testing on my part -- omnicomplete WAS working for parts of my project, just not most of it.
The issue was that I'm working on a django project, and in order to import django.db, you need to have an environment variable set. Since I couldn't import django.db, any class that inherited from django.db, or any module that imported a class that inherited from django.db wouldn't complete.

I get completion for my own modules in my PYTHONPATH or site-packages. I'm not sure what version of the pythoncomplete.vim script you're using, but you may want to make sure it's the latest.
EDIT: Here's some examples of what I'm seeing on my system...
This file (mymodule.py), I puth in a directory in PYTHONPATH, and then in site-packages. Both times I was able to get the screenshot below.
myvar = 'test'
def myfunction(foo='test'):
pass
class MyClass(object):
pass

Just ran across this on Python reddit tonight: PySmell. Looks like what you're looking for.
PySmell is a python IDE completion helper.
It tries to statically analyze Python source code, without executing it, and generates information about a project’s structure that IDE tools can use.

While it's important to note that you must properly set your PYTHONPATH environmental variable, per the the previous answer, there is a notable bug in Vim which prevents omnicompletion from working when an import fails. As of Vim 7.2.79, this bug hasn't been fixed.

Trouble-shooting tip: verify that the module you are trying to omni-complete can be imported by VIM. I had some syntactically correct Python that VIM didn't like:
:python import {module-name}
Traceback (most recent call last):
File "<string>", line 1, in ?
File "modulename/__init__.py", line 9
class empty_paranthesis():
^
SyntaxError: invalid syntax
Case-in-point, removing the parenthesis from my class definition allowed VIM to import the module, and subsequently OmniComplete on that module started to work.

I think your after the pydiction script. It lets you add your own stuff and site-packages to omni complete.
While your at it, add the following to your python.vim file...
set iskeyword+=.
This will let you auto-complete package functions e.g. if you enter...
os.path.
and then [CTRL][N], you'll get a list of the functions for os.path.