We Keep Coding

Unexpected line given when Python interpreter reports exception [duplicate]

This question already has an answer here:
Why does Python read from the current directory when printing a traceback?
(1 answer)
Closed 3 years ago.
When the Python interpreter reports an error/exception (I'm just going to say "error" to refer to both of these from now on), it prints the line number and contents of the line that caused the error.
Interestingly, if you have a long-running Python script which causes an error and change the .py file while the script is running, then the interpreter can report an incorrect line as raising the error, based on the changed contents of the .py file.
MWE:
sample.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
raise Exception("foo", "bar")
This script runs for 10 seconds, then raises an exception.
sample2.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
"""
This
is
just
some
filler
to
demonstrate
the
behavior
"""
raise Exception("foo", "bar")
This file is identical to sample.py except that it has some junk between the end of the loop and the line raises the following exception:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
What I Did
python3 sample.py
In a second terminal window, mv sample.py sample.py.bak && cp sample2.py sample.py before sample.py finishes execution
Expected Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
Here, the interpreter reports that there was an exception on line 7 of sample.py and prints the Exception.
Actual Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
"""
Exception: ('foo', 'bar')
Here, the interpreter also reports """ when it reports the exception.
It seems to be looking in the file on disk to find this information, rather than the file loaded into memory to run the program.
Source of my Confusion
The following is my mental model for what happens when I run python3 sample.py:
The interpreter loads the contents of sample.py into memory
The interpreter performs lexical analysis, semantic analysis, code generation, etc. to produce machine code
The generated code is sent to the CPU and executed
If an error is raised, the interpreter consults the in-memory representation of the source code to produce an error message
Clearly, there is a flaw in my mental model.
What I want to know:
Why does the Python interpreter consult the file on disk to generate error message, rather than looking in memory?
Is there some other flaw in my understanding of what the interpreter is doing?

As per the answer linked by #b_c,
Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback.
[...]
When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on
[...]
The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it.

Handling usernames in file paths using Python

I'm writing a program that I would like to make a stand alone for my company. It works perfectly when I run it from the sublime text shell and I have everything set up to go except one issue that I can't seem to solve; file paths that involve usernames. Does anyone have any suggestion on how to handle this?
An example is
wb.save(r'C:\Users******\Desktop\Excel.xlsx')
I want to make the ****** part either be automatic or an input box.

Use os.path.expanduser() with '~' where you want the home directory:
import os
print(os.path.expanduser('~/Desktop/Excel.xlsx'))
Alternatively use pathlib.Path:
from pathlib import Path
print(Path.home() / 'Desktop' / 'Excel.xlsx')

os.getlogin() will do
import os
path = os.path.join(r'C:\Users',os.getlogin(),'Desktop','Excel.xlsx')
print(path)

Awesome! Looks like that worked but it presented another error now when I create it as a stand alone.
Wait originally works when I run it from the shell using this code, where EC is expected conditions:
wait.until(EC.frame_to_be_available_and_switch_to_it(driver.find_element_by_name('AppBody')))
Whenever I run it as a stand alone though I get the following error:
Traceback (most recent call last):
File "Stand_Alone_CAS_Automation", line 57, in <module>
NameError: name 'wait' is not defined
[17344] Failed to execute script Stand_Alone_CAS_Automation

Maya Python: IOError: [Errno 9] Bad file descriptor

I created a maya python toolchain for my team. All works well, just on one machine i seem to have problems. I narrowed it down to the print command. Like this test library called "temp.py":
import os
# from pymel.core import *
print "Hello"
after importing it with
import temp
it produces this output (only on that one computer!):
// Error: 9
# Traceback (most recent call last):
# File "<maya console>", line 1, in <module>
# File "C:\maya_scripts\temp.py", line 4, in <module>
# print "Hello"
# IOError: [Errno 9] Bad file descriptor //
I've tried Maya Version 2016, 2016.5 and 2017. All the same result. Python 2.5 as standalone hasn't got that problem. To me that sounds like some kind of configuration problem, but then again it behaves the same over 3 different maya installations, so deleting the prefs didn't help either.

It's hard to know what's really happening here. But try this
import sys
sys.__stdout__.write("hello")
sys.__stdout__.write(str(sys.stdout))
Then check your output window (not the listener). In a vanilla maya you'd expect to see something like
<maya.Output object at 0x00000217E827FC10>
after "hello". If you see something else, some script has tried to hijack sys.stdout in this installation. You can probably work around it by creating an environment variable called MAYA_SKIP_USERSETUP_PY, setting it to 1, and restarting -- that should stop whatever script is being naughty from autoloading.
This ought to reset it to what you're looking for:
import maya.utils as utils
sys.stdout = utils.Output()
However you need to comb through the scripts on that machine and find out who is messing with sys.stdout behind your back

The error is from your module, you may overwrite the print function
maya 2016 is python 2.7.6 and maya 2017 is in python 3.x
on 2017 you must use print("")

PylintSensor explodes on multi-module report

When running an analysis I got the following error:
ERROR: Error during SonarQube Scanner execution
java.lang.IllegalStateException: Cannot analyse the
file'/home/user/project/package/module.py', details:
'java.lang.IllegalArgumentException: 302 is not a valid line for pointer.
File [moduleKey=com.project, relative=package/module.py,
basedir=/home/user/project] has 95 line(s)'
And that's correct, the file does have 95 lines. I checked the report generated by Pylint for that file (the last one in .sonar/pylint) and there are messages for two modules:
************* Module project.package.module
project/package/module.py:19: [C0301(line-too-long), ] Line too long (112/80)
project/package/module.py:1: [C0111(missing-docstring), ] Missing module docstring
************* Module django.contrib.admin.views.main
/home/user/.virtualenvs/project-env/local/lib/python2.7/site-
packages/django/contrib/admin/views/main.py:302: [W0201(attribute-defined-
outside-init), ChangeList.get_query_set] Attribute 'has_filters' defined outside __init__
So that's whence the 302nd line comes: the file in which the super class is defined. PyLint analyses the class in its entirely, for which it jumps to the virtualenv, then reports messages for both files. When the sensor processes that, it assumes there's only one file.
I think the sensor should be ignoring anything outside of the current file, since anything inside the project will be analysed eventually, and anything outside of the project will not be reported.
Is there a way to disable the analysis of dependencies in PyLint? (If so that's probably what Sonar should be doing, right?)

Ok, so the fix was simple enough: update Pylint.

Really weird issue with shelve (python)

I create a file called foo_module.py containing the following code:
import shelve, whichdb, os
from foo_package.g import g
g.shelf = shelve.open("foo_path")
g.shelf.close()
print whichdb.whichdb("foo_path") # => dbhash
os.remove("foo_path")
Next to that file I create a directory called foo_package than contains an empty __init__.py file and a file called g.py that just contains:
class g:
pass
Now when I run foo_module.py I get a weird error message:
Exception TypeError: "'NoneType' object is not callable" in ignored
But then, if I rename the directory from foo_package to foo, and change the import line in foo_module.py, I don't get any error. Wtf is going on here?
Running Python 2.6.4 on WinXP.

I think you've hit a minor bug in 2.6.4's code related to the cleanup at end of program. If you run python -v you can see exactly at what point of the cleanup the error comes:
# cleanup[1] foo_package.g
Exception TypeError: "'NoneType' object is not callable" in ignored
Python sets references to None during the cleanup at the end of program, and it looks like it's getting confused about the status of g.shelf. As a workaround you could set g.shelf = None after the close. I would also recommend opening a bug in Python's bug tracker!

After days of hair loss, I finally had success using an atexit function:
import atexit
...
cache = shelve.open(path)
atexit.register(cache.close)
It's most appropriate to register right after opening. This works with multiple concurrent shelves.
(python 2.6.5 on lucid)

This is indeed a Python bug, and I've posted a patch to the tracker issue you opened (thanks for doing that).
The problem is that shelve's del method calls its close method, but if the shelve module has already been through cleanup, the close method fails with the message you see.
You can avoid the message in your code by adding 'del g.shelf' after g.shelf.close. As long as g.shelf is the only reference to the shelf, this will result in CPython calling the shelve's del method right away, before the interpreter cleanup phase, and thus avoid the error message.

It seems to be an exception in a shutdown function registered by the shelve module. The "ignored" part is from the shutdown system, and might get its wording improved sometime, per Issue 6294. I'm still hoping for an answer on how to eliminate the exception itself, though...

for me a simple shelve.close() on an unclosed one did the job.
shelve.open('somefile') returns a "persistent dictionary for reading and writing" object which i used throughout the app's runtime.
when I terminated the app I received the "TypeError" Exception as mentioned.
I plased a 'close()' call in my termination sequence and that seemed to fix the problem.
e.g.
shelveObj = shelve.open('fileName')
...
shelveObj.close()

OverByThere commented on Jul 17, 2018
This seems to be fixable.
In short open /usr/lib/python3.5/weakref.py and change line 109 to:
def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):
And line 117 to:
_atomic_removal(d, wr.key)
Note you need to do this with spaces, not tabs as this will cause other errors.