We Keep Coding

What character set is "é" from? (Python: Filename with "é", how to use os.path.exists , filecmp.cmp, shutil.move?)

What Character set is é from? In Windows notepad having this character in an ANSI text file will save fine. Insert something like 😍 and you'll get an error. é seems to work fine in ASCII terminal in Putty (Are CP437 and IBM437 the same?) where as 😍 does not.
I can see that 😍 is Unicode, not ASCII. But what is é? It doesn't give errors I get with Unicode in Notepad, but Python was throwing SyntaxError: Non-ASCII character '\xc3' in file on line , but no encoding declared; before I added a "magic comment" as suggested by Python NLTK: SyntaxError: Non-ASCII character '\xc3' in file (Sentiment Analysis -NLP).
I added the "magic comment" and don't get that error, but os.path.isfile() is saying a filename with é doesn't exist. Ironic that the character é is in Marc-André Lemburg, the author of the PEP the error links to.
EDIT: If I print the path of the file, the accented e shows up as ├⌐ but I can copy and paste é into the command prompt.
EDIT2: See below
Private > cat scratch.py ### LOL cat scratch :3
# coding=utf-8
file_name = r"Filéname"
file_name = unicode(file_name)
Private > python scratch.py
Traceback (most recent call last):
File "scratch.py", line 3, in <module>
file_name = unicode(file_name)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: ordinal not in range(128)
Private >
EDIT3:
Private > PS1="Private > " ; echo code below ; cat scratch.py ; echo ======= ; echo output below ; python scratch.py
code below
# -*- coding: utf-8 -*-
file_name = r"Filéname"
file_name = unicode(file_name, encoding="utf-8")
# I have code here to determine a path depending on the hostname of the
# machine, the folder paths contain no Unicode characters, for my debug
# version of the script, I will hardcode the redacted hostname.
hostname = "One"
if hostname == "One":
folder = "C:/path/folder_one"
elif hostname == "Two":
folder = "C:/path/folder_two"
else:
folder = "C:/path/folder_three"
path = "%s/%s" % (folder, file_name)
path = unicode(path, encoding="utf-8")
print path
=======
output below
Traceback (most recent call last):
File "scratch.py", line 18, in <module>
path = unicode(path, encoding="utf-8")
TypeError: decoding Unicode is not supported
Private >

You need to tell unicode what encoding the string is in, in this case it's utf-8 not ascii, and the file header should be # -*- coding: utf-8 -*-, Encoding Declarations
# -*- coding: utf-8 -*-
file_name = r"Filéname"
file_name = unicode(file_name, encoding="utf-8")
1 Help on class unicode in module __builtin__:
2
3 class unicode(basestring)
4 | unicode(object='') -> unicode object
5 | unicode(string[, encoding[, errors]]) -> unicode object
6 |
7 | Create a new Unicode object from the given encoded string.
8 | encoding defaults to the current default string encoding.
9 | errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.
And as I mentioned in my previous comment you will save yourself a lot of headaches by switching to Python 3. Python 2 on a Windows filesystem with unicode characters can be a nightmare.

Why not monkey patch sys.getfilesystemencoding()?

In Python can read the filesystem encoding with sys.getfilesystemencoding().
But there seems to be no official way to set the filesystem encoding.
See: How to change file system encoding via python?
I found this dirty hack:
import sys
sys.getfilesystemencoding = lambda: 'UTF-8'
Is there a better solution, if changing environment variable LANG before starting the interpreter is not an option?
Background, why I want this:
This works:
user#host:~$ python src/setfilesystemencoding.py
LANG: de_DE.UTF-8
sys.getdefaultencoding(): ascii
sys.getfilesystemencoding(): UTF-8
This does not work:
user#host:~$ LANG=C python src/setfilesystemencoding.py
LANG: C
sys.getdefaultencoding(): ascii
sys.getfilesystemencoding(): ANSI_X3.4-1968
Traceback (most recent call last):
File "src/setfilesystemencoding.py", line 10, in <module>
with open('/tmp/german-umlauts-üöä', 'wb') as fd:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-22: ordinal not in range(128)
Here is the simple script:
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals, print_function
import os, sys
print('LANG: {}'.format(os.environ['LANG']))
print('sys.getdefaultencoding(): {}'.format(sys.getdefaultencoding()))
print('sys.getfilesystemencoding(): {}'.format(sys.getfilesystemencoding()))
with open('/tmp/german-umlauts-üöä', 'wb') as fd:
fd.write('foo')
I hopped that above monkey patching would solve this ... but it doesn't. Sorry, this question does not make sense any more. I close it.
My solution: use LANG=C.UTF-8

ConfigParser and global variables

I have a python file that calls a function in another directory.
I would like to use the config variable DATA_DIR in the function directly without importing configuration at each time.
The main file looks like this :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import ConfigParser
config = ConfigParser.ConfigParser()
config.read('static.cfg')
global __DATA_DIR__
__DATA_DIR__ = config.get('Directories', '__DATA_DIR__')
from src.directory import file
file.function()
The function looks like this :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
def function():
global __DATA_DIR__
print (__DATA_DIR__)
The configuration file looks like this :
[Directories]
__DATA_DIR__=/directorie/to/config.cfg
When executing the first main program, I had this error :
NameError: global name 'DATA_DIR' is not defined

Why not pass the config argument to the function. This would need confirmation but I would imagine that only the read method actually reads and parses the actual file, and the config.get method only gives you data from an internal datastructure, so passing the config object and doing a config.get inside the function would be pretty efficient.
so :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import ConfigParser
from src.directory import file
config = ConfigParser.ConfigParser()
config.read('static.cfg')
file.function( config )
and in your file :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
def function( cfg ):
print ( cfg.get("__DATA_DIR__") )

Since you are importing your function, you should pass in your __DATA_DIR__ variable to it:
from src.directory import file
file.function(__DATA_DIR__)
#your other file
def function(data):
print (data)

Python forgetting variable name?

I'm learning Python and I'm have some issues.
I got this error message:
Traceback (most recent call last):
File "main_console.py", line 8, in <module>
from util import Util
File "../utils/util.py", line 13, in <module>
class Util:
File "../utils/util.py", line 73, in Util
config.write(configfile)
NameError: name 'config' is not defined
Following is my code (this is inside a function):
config = ConfigParser.ConfigParser()
for index, list in enumerate(my_list):
config.add_section(str(index))
config.set(str(index), 'id', list.name)
config.set(str(index), 'host', list.host)
# Creating the folder
myFolder = "/etc/element/"
if not os.path.exists(myFolder):
os.makedirs(myFolder)
# Creating the file
filePath = "/etc/element/settings.cfg"
with open(filePath, 'wb') as configfile:
config.write(configfile)
Note: I'm using Sublime Text 3 as IDE. "myFolder" have the same problem a little time ago - if I type filePath = myFolder + '/settings.cfg' I got the same error of "not defined". Finally, I imported ConfigParser as following: import ConfigParser.
Is Python forgetting my variable name or I'm doing something wrong?

It is possible ConfigParser or some other import also defines a config variable that is being used by your config.write().
Sometimes it is safer to just import the functions you need, that way you know exactly what is defined in your file. It means you have to explicitly import everything that you use in other files, but it prevents any unknown duplication in your namespace.
You would do this like:
from ConfigParser import ConfigParser # instead of import ConfigParser
# Then
config = ConfigParser()
Secondly, config is a common variable - try renaming it to something like myConfig and see if it still happens.

configfile is the object that you want to use for write(). You are using a parser to write.

Thanks for replies, but I found the solution.
I go to Nano and edit the file with this error, so I see the indentation is absolutaly wrong. It was just a Sublime text 3 issue, now's solved.

Create and use settings module in Python

I am new to Python so bear with me. I use the pyDev plugin fore eclipse. There are three files:
tool.py:
from gui import Tool_Window
import wx
import settings
if __name__ == '__main__':
window = wx.App()
Tool_Window(None, settings.WindowHeader)
window.MainLoop()
Tool_Window.py:
from Tool import settings
import wx
class Tool_Window(wx.Frame):
def __init__(self, parent, title):
super(Tool_Window,self).__init__(parent, title = title)
self.SetDimensions(settings.WindowOpenX,
settings.WindowOpenY,
settings.WindowWidth,
settings.WindowHeight)
settings.py:
WindowHeader = 'The SuperAwesome Tool'
WindowOpenX = 500
WindowOpenY = 100
WindowWidth = 200
WindowHeight = 400
The "tool.py" file is in a package called "Tool", as is the "settings.py" file, and "Tool_Window" is in the package "gui".
I am getting error messages from a previous file I had in the project, now renamed to the "settings.py". I have tried cleaning the project in Eclipse, but nothing happens. The error message looks like:
Traceback (most recent call last):
File "/home/oystein/workspaces/python/awesome.tool/src/Tool/tool.py", line 8, in <module>
Tool_Window(None, settings.WindowHeader)
File "/home/oystein/workspaces/python/awesome.tool/src/gui/__init__.py", line 12, in __init__
# ;-)
AttributeError: class GeneralParameters has no attribute 'WindowParameters'
Previously I had a class named GeneralParameters with a sub-class WindowParameters, as I wanted to access static variables for settings. I relaized Python couldn't do it like that and changed it to the "settings.py" file.
I run the program from "tool.py"
Can anyone see what's wrong here?

You are running stale byte-code, remove the .pyc files and rerun your code.
The traceback reads the source from the .py file but is built from the bytecode, and the fact that is shows that the error is on a line that only consists of a comment is a hint that things are no longer in sync.
Normally, Python will clean up the .pyc file when stale, but only if the .py modification time is newer.