Running Python in a standard GNU terminal emulator on Ubuntu 14.04, I get the expected behavior when typing interactively:
>>> len('tiθ')
4
>>> len(u'tiθ')
3
The same thing happens when running an explicitly utf8-encoded script in Spyder:
# -*- coding: utf-8 -*-
print(len('tiθ'))
print(len(u'tiθ'))
...gives the following output, regardless of whether I run it in a new dedicated interpreter, or run in a Spyder-default interpreter (shown here):
>>> runfile('/home/dan/Desktop/mwe.py', wdir=r'/home/dan/Desktop')
4
3
But when typing interactively in a Python console within Spyder:
>>> len('tiθ')
4
>>> len(u'tiθ')
4
This issue has been brought up elsewhere, but that question regards differences between Windows and Linux. Here, I'm getting different results in different consoles on the same system, and the Python startup message in the terminal emulator and in the console within Spyder are identical:
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
What is going on here, and how can I get Python-within-Spyder to behave like Python-in-the-shell with regard to unicode strings? #martijn-pieters makes the comment on this question that
Spyder does all sorts of things to break normal a Python environment.
But I'm hoping there's a way to un-break this particular feature, since it makes it really hard to debug scripts in the IDE when I can't rely on my interactive typed commands to yield the same results as scripts run as a whole with their coding: utf-8 declaration.
UPDATES
In the GNU terminal:
>>> repr(u'tiθ')
"u'ti\\u03b8'"
>>> import sys
>>> sys.stdin.encoding
'UTF-8'
>>> sys.getdefaultencoding()
'ascii'
In Spyder console:
>>> repr(u'tiθ')
"u'ti\\xce\\xb8'"
>>> import sys
>>> sys.stdin.encoding # returns None
>>> sys.getdefaultencoding()
'UTF-8'
So knowing that, can I convince Spyder to behave like the GNU terminal?
After a bit of research, it seems that the strange behavior is at least in part built into Python (see this discussion thread; briefly, Python sets sys.stdin.encoding to None as default, and only changes it if it detects that the host is tty and it can detect the tty's encoding).
That said, a hackish workaround was just to tell Spyder to use /usr/bin/python3 as its executable instead of the default (which was Python 2.7.6). When running a Python 3 console within Spyder or within a GNU terminal emulator, I get results different (better!) than before, but importantly the results are consistent, regardless of whether running a script or typing interactively, and regardless of using GNU terminal or Spyder console:
>>> len('tiθ')
3
>>> len(u'tiθ')
3
>>> import sys
>>> sys.stdin.encoding
'UTF-8'
>>> sys.getdefaultencoding()
'utf-8'
This leads to other problems within Spyder, however: its sitecustomize.py script is not Python 3 friendly, for example, so every new interpreter starts with
Error in sitecustomize; set PYTHONVERBOSE for traceback:
SyntaxError: invalid syntax (sitecustomize.py, line 432)
The interpreter seems to work okay anyway, but it makes me nervous enough that I'm not considering this an "acceptable" answer, so if anyone else has a better idea...
Related
I'm going through free Google Python training course. Real basic stuff. My OS is MacOS. Using MacOS Terminal. I can run python modules like hello.py.
I can run this to see my version:
% python --version
Python 3.9.12
However, this fails:
% python help(sys)
zsh: unknown file attribute: y
What do I need to change in my setup to resolve and be able to call simple help in Python?
Parentheses have a special meaning in zsh, so they must be escaped here to pass the argument as-is to Python. You can use \ in front of the parentheses (python help\(sys\)), but it's conventional to just quote the entire argument:
python 'help(sys)'
However, this still won't work (because the python command interprets it as a filename), you need to pass code via the -c flag to Python:
python -c 'help(sys)'
And that still won't work since you haven't loaded the sys package in that code. Do that, to get working code:
python -c 'import sys; help(sys)'
Konrad is right, but you are likely looking for a way to start Python interactive shell and execute statements there.
% python
Python 3.10.7 (main, Sep 15 2022, 01:52:03) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> help(sys)
When I'm trying to write anything on the python27 interpreter, I'm getting syntax error, no matter what I'm typing (Not just print, literally anything). Couldn't find anyone else experiencing the same issue, so no clue why this is happening.
I've recently installed Python3 and PyCharm, but I still want to use python27 for some legacy code I have. The problem started to happen after I installed Python3 and PyCharm (Not sure if PyCharm may be related or not).
my OS is Windows10 x64.
Any ideas anyone?
Python27:
Python3:
I can reproduce this perfectly by using a non-break space unicode character as the last character:
Python 2.7.16 (default, Apr 11 2019, 01:11:37)
[GCC 4.2.1 Compatible FreeBSD Clang 6.0.0 (tags/RELEASE_600/final 326565)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Test"
File "<stdin>", line 1
print "Test"
^
SyntaxError: invalid syntax
>>> import os
File "<stdin>", line 1
import os
^
SyntaxError: invalid syntax
>>> 111
File "<stdin>", line 1
111
^
SyntaxError: invalid syntax
>>>
When I copy/paste that into IPython based on python 3, it shows up:
In [1]: test = 'print "Test" '
Out[1]: 'print "Test"\xa0'
In [2]: import unicodedata
In [3]: unicodedata.name(test[-1])
Out[3]: 'NO-BREAK SPACE'
I expect the same to happen with any space-like unicode character.
When typing code into Python, use End to go to the end of the line, and then one or more Backspace to remove any extra whitespace, before you press Enter. If your cursor moves when you press End, that would confirm the issue.
If not, then maybe it is whatever the terminal program sends to Python when you press Enter.
I would advise to just remove python 2.7 and port any remaining 2.7 specific code to python 3.
Edit Since you still need Python 2, I would suggest to try running a python 2 script. If that works, the problem is probably with the terminal setup (like text encoding, line feed etc.) being different than what Python 2 expects. If scripts work, try one of the many different terminal programs available for ms-windows.
I don't know about the meaning of calculator mode in Python, and I've put the portion of documentation below.
If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefore, if you want to write a somewhat longer program, you are better off using a text editor to prepare the input for the interpreter and running it with that file as input instead. This is known as creating a script. As your program gets longer, you may want to split it into several files for easier maintenance. You may also want to use a handy function that you’ve written in several programs without copying its definition into each program.
To support this, Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter. Such a file is called a module; definitions from a module can be imported into other modules or into the main module (the collection of variables that you have access to in a script executed at the top level and in calculator mode).
(Emphasis mine)
Here's the original document.
Interactive mode and Calculator mode are the same thing. This is a mode that comes with Python. If you have installed Python then you have also installed something called the Python Shell.
There are two ways you can access the Python Shell:
Typing python or python[version-number] in your command
prompt/terminal window:
Assuming that you have python in your PATH variable, you can access
the Python Shell by simply typing python or python[version-number]
in your command prompt/terminal window.
Running the Python Shell in the Python IDLE(Integrated Development Environment) GUI:
To run the Python Shell in the Python IDLE GUI, you can type(again i'm assuming that the path to your python installation folder, is in your PATH variable), just type idle into your command prompt\terminal window and this should start the Python Shell in the Python IDLE GUI.
Of course the exact text for the Python Shell heading will vary between OS's, but all of them look very similar. Here is an example of what the heading appears like on my Mac:
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
As you can tell from the text above, a newline in the Python Shell is denoted by three caret symbols: >>>. For each newline three new carets are printed. Using Python Shell is different from typing a script because the script is predefined and the shell is written line-by-line.
Here is an example to illustrate my point further:
>>> xyz = 100
>>> for i in range(1, 10):
... xyz += i
... print(xyz)
...
101
103
106
110
115
121
128
136
145
As you can tell from the above program, indention is noted by three dots: ..., and the only time the Python Shell shows only one line at a time unless it is 'echoing' back what you typed in.
Why is it called interactive?
One of the main reason it's called interactive is that to display variable values or run the module in general you don't have to explicitly invoke the Python interpreter. Take the example below:
>>> name = "some name"
>>> print(name)
some name
>>> name
'some name'
As displayed above, you can access the values of a variable without needing to call print on the variable. This can be very useful when debugging or trying to understand your code.
The Python Shell is not really a practical way to write long/complex programs. A better choice would be to use the Python IDLE built-in script editor or another text-editor or IDE.
I'm trying to understand how PYTHONIOENCODING environment variable fits with Python2.7, so I tried the following things with the interactive prompt:
antox#antox-pc ~/Scrivania $ export PYTHONIOENCODING='latin1'
antox#antox-pc ~/Scrivania $ /usr/bin/python2.7
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdin.encoding
'latin1'
>>> sys.stdout.encoding
'latin1'
>>> b = 'ÿ'
>>> b
'\xc3\xbf' #Shouldn't I get something like '\xff' because I set PYTHONIOENCODING to latin1? It looks as if utf-8 is been used instead
>>> print '\xff'
� # Why this odd character? Shouldn't I get 'ÿ' always for the reason above?
My questions/doubts are indicated as comments.
By setting PYTHONIOENCODING in the environment, you're telling Python to not trust your terminal/OS's information regarding the encoding -- you're saying that you know better, and the terminal device actually accepts that encoding, not whatever the OS &c will tell Python.
So in this case you're saying that (whatever it claims otherwise) your terminal actually accepts and properly formats bytes in latin-1.
That is probably not the case (if you don't set that environment variable what does sys.stdout.encoding say? utf-8, I guess?) so it's not surprising that you don't get the display you want:-).
On your specific question,
sys.getdefaultencoding()
tells you what encoding Python will use to translate between actual text (that is, Unicode) and byte strings, in situations where it has no other indication (I/O to stdin/stdout is not one of those situations, as it uses the encoding attribute of those files).
>>> b = 'ÿ'
This has nothing to do with sys.stdin/stdout -- rather, your terminal is sending, after the open quote, some "escape sequence" that boils down to proper utf-8 (my Mac's Terminal app does, for example). If this was in a .py file without a proper source-encoding preamble, it would be a syntax error -- the interactive interpreter has become a softy in 2.7.9:-)
>>> print '\xff'
� # Why this odd character? Shouldn't I get 'ÿ' always for the reason above?
You've told Python that your terminal accepts and properly displays latin-1 byte sequences (even though the terminal probably wants utf-8 ones and tells Python that, you've told Python to ignore what the terminal says about its encoding, or rather, what the OS says the terminal says:-).
So the byte of value 255 is sent as-is, and the terminal doesn't like it one bit (since the terminal doesn't actually accept latin-1!) and displays an error-marker.
Here's a typical example on my Mac (where the Terminal does actually accept 'utf-8'):
ozone:~ alex$ PYTHONIOENCODING=latin-1 python -c "print u'\xff'"
?
ozone:~ alex$ PYTHONIOENCODING=utf-8 python -c "print u'\xff'"
ÿ
ozone:~ alex$ python -c "print u'\xff'"
ÿ
Letting Python properly detect the terminal encoding on its own, or forcing it to what happens to be the right one, displays correctly.
Forcing the encoding to one the terminal does not in fact accept, unsurprisingly, does not display correctly.
Should you ever attach to your machine's serial port an ancient teletype which does in fact accept latin-1 (but the OS doesn't detect that fact properly), PYTHONIOENCODING will help you properly do Python I/O on that ancient teletype. Otherwise, it's unlikely that said environment setting will be of much use to you:-).
I'm a newbie programmer so I'll do my best to clearly ask my question. I'm running Python scripts in Mac 10.6.5 and now trying to write and save to a text file (following instructions in HeadsUp Python book). Whenever I hit function+F5 (as instructed) I get the same "invalid syntax" error and Idle highlights the "1" in "Python 3.1.3" of the header. Here's the header to which I'm referring:
Python 3.1.3 (r313:86882M, Nov 30 2010, 09:55:56) [GCC 4.0.1 (Apple Inc. build 5494)] on darwin Type "copyright", "credits" or "license()" for more information.
Extremely frustrating. I've checked and rechecked the code but this doesn't seem to be code related because the "syntax error" is in regards to the header text that posts in every Idle/Python session. Help anyone?
... and Idle highlights the "1" in "Python 3.1.3" of the header ...
Standalone Python scripts used to contain a "header", but that would be just
#!/usr/bin/env python
or, depending on the name of the interpreter maybe
#!/usr/bin/env python3.1
Not sure I understand your question, though.
you are writing your script in the wrong IDLE window ! when starting IDLE, it opens 2 windows: one for writing a script and another one with an interactive python shell. executing the content of the interactive python shell makes no sense.
#squashua: I have the same issue when I try to run the code either in IDLE or Ubuntu terminal.
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25)
it highlights "5" as syntax error.