After a bit over 8 years of using Python I've run today into issue with Python 3.8: it executed code that I commented out.
I was able to interrupt it as it was going through code path that should have been blocked by the comment to get this screenshot:
As the function names indicate, the operation in question is somewhat time-consuming to rollback and I would love to know what happened to avoid dealing with that in the future.
My current best explanation is that since the code is run on a remote machine for whatever reason the commenting out did not go through when the code started, but did for the stack trace.
Does anyone had a similar experience or have an idea of what might have happened?
I confirmed my hypothesis from the comments, with a file like:
import time
def dont_run():
raise Exception("oh no i ran it")
time.sleep(10)
dont_run()
I saved that file, and ran it. While it was running I commented out the last line and re-saved the file, I then got this error:
$ py main.py
Traceback (most recent call last):
File "main.py", line 10, in <module>
# dont_run()
File "main.py", line 6, in dont_run
raise Exception("oh no i ran it")
Exception: oh no i ran it
So I think what must have happened here is that you ran the file before the file was saved to disk (perhaps a race between two network requests and you got unlucky).
Related
The problem that I'm having is frankly baffling. I was doing some python coding, come back the next day, and without changing anything, the code stopped working, giving a FileNotFoundError. I've whittled it down to a laughably simple minimum code to produce the error.
f = open('positions.tsv','w')
Which produces the error
Traceback (most recent call last):
File "c:/Users/mrhel/Documents/GitHub/gas-collisions/test.py", line 1, in <module>
f = open('positions.tsv','w')
FileNotFoundError: [Errno 2] No such file or directory: 'positions.tsv'
I have been scouring the internet for hours for a solution but everything I come across is someone trying to create a file in a folder that doesn't exist or forgetting to set the mode to 'w'. I copied the exact code that this is happening with onto a Linux machine and it runs just fine, it is specifically not working on my Windows 10 machine.
I have tried rebooting the computer, which doesn't work either. Why would this simple line of code not work specifically on Windows?
Edit:
How I run the code is either python test.py or C:/Users/mrhel/AppData/Local/Microsoft/WindowsApps/python3.8.exe c:/Users/mrhel/Documents/GitHub/gas-collisions/test.py
In any case, the expected behavior according to the open() documentation
is that positions.tsv should be created upon calling this function with the 'w' mode.
When I do create the file in advance, I get an error when I try to write to it.
f = open('positions.tsv','w')
f.write('testing')
f.close()
Error:
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:/Users/mrhel/Documents/GitHub/gas-collisions/test.py", line 3, in <module>
f.close()
OSError: [Errno 9] Bad file descriptor
Edit 2:
I tried creating a file and reading it, and that works as expected. It's specifically writing that isn't working
Edit 3:
So since it runs in another folder, I decided to check for write permissions, and this is what I got. When I check properties of the folder with windows, all users have allow ticked.
>>> os.access('my_folder', os.W_OK)
False
This is clearly a permissions issue, but I'm not sure how to rectify it.
This turned out to be a result of "Controlled Folder Access" in Windows Defender Ransomware Protection. Turning off this feature allows it to function as expected. It's not clear how Ransomware Protection decided that my code is not to be trusted, but for posterity's sake, that is how this was fixed.
[autoreload of R17_Plot_profiles failed: Traceback (most recent call last):
File "/home/jdrevon/anaconda3/lib/python3.7/site-packages/IPython/extensions/autoreload.py", line 244, in check
superreload(m, reload, self.old_objects)
File "/home/jdrevon/anaconda3/lib/python3.7/site-packages/IPython/extensions/autoreload.py", line 378, in superreload
module = reload(module)
File "/home/jdrevon/anaconda3/lib/python3.7/imp.py", line 314, in reload
return importlib.reload(module)
File "/home/jdrevon/anaconda3/lib/python3.7/importlib/__init__.py", line 168, in reload
raise ModuleNotFoundError(f"spec not found for the module {name!r}", name=name)
ModuleNotFoundError: spec not found for the module 'R17_Plot_profiles'
]
This problem occurs in Python 3.7 on Spyder when I modify part of my code and I press F9 to execute some parts of it. Even if I execute my main python file which load every sub-python .py codes I have this error message. However, It does not stop the code from running but the file didn't save the modifications made. This is the first times it happens since I imported the 'R17_Plot_profiles.py' python file.
However in the case where I execute the main profile, I have the previous message but I have also the following one :
Reloaded modules: reconstruct, R2_Spice_Kernels, R4_Offset_Correction, R5_State_Vectors_twoway_v2, R6_Foot_Print, R7_Framework_Conversion,
R8_BendAng_ImpParam_up, R8_BendAng_ImpParam_dn, R9_Avg_BendAng_ImpParam, R10_Altitude_above_Ellipsoid, R11_Refractivity_and_Bending_Radius_v2,
R12a_Electron_Density_up, R12b_Electron_Density_dn, R12_Electron_Density_avg, R13_Neutral_Number_Density,
R14_Pressure_and_Temperature, R16_Errors, R17_Plot_profiles,
R3_Read_L2_Data_v2
So I don't know how to solve the autoreload problem. I've tried to downgrade my IPython version to 7.5 like some people tried and succeed but for me It doesn't change anything.
Edit : It's important to notice that if I restart the kernel, the first execution do not show the error message until I modify directly the python code.
I did few modifications yesterday and it seems to be solved. Since the error occurence started to be random at some points, i'm not sure for 100% but it can maybe solve the issue.
The problem could be came from the use of os.chdir function, I replaced it by specifying the entire path to save my figures until to go to the path using os.chdir and then save my figures in it.
It can maybe produce some conflicts of path and repository to find the python script which is no longer in the path where the script is currently working and therefore can't find it in order to save the modifications.
I keep you informed if at some point the error comes back but for now after some trials the error didn't show up anymore.
This question already has an answer here:
Why does Python read from the current directory when printing a traceback?
(1 answer)
Closed 3 years ago.
When the Python interpreter reports an error/exception (I'm just going to say "error" to refer to both of these from now on), it prints the line number and contents of the line that caused the error.
Interestingly, if you have a long-running Python script which causes an error and change the .py file while the script is running, then the interpreter can report an incorrect line as raising the error, based on the changed contents of the .py file.
MWE:
sample.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
raise Exception("foo", "bar")
This script runs for 10 seconds, then raises an exception.
sample2.py
from time import sleep
for i in range(10):
print(i)
sleep(1)
"""
This
is
just
some
filler
to
demonstrate
the
behavior
"""
raise Exception("foo", "bar")
This file is identical to sample.py except that it has some junk between the end of the loop and the line raises the following exception:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
What I Did
python3 sample.py
In a second terminal window, mv sample.py sample.py.bak && cp sample2.py sample.py before sample.py finishes execution
Expected Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
Exception: ('foo', 'bar')
Here, the interpreter reports that there was an exception on line 7 of sample.py and prints the Exception.
Actual Behavior
The interpreter reports the following:
Traceback (most recent call last):
File "sample.py", line 7, in <module>
"""
Exception: ('foo', 'bar')
Here, the interpreter also reports """ when it reports the exception.
It seems to be looking in the file on disk to find this information, rather than the file loaded into memory to run the program.
Source of my Confusion
The following is my mental model for what happens when I run python3 sample.py:
The interpreter loads the contents of sample.py into memory
The interpreter performs lexical analysis, semantic analysis, code generation, etc. to produce machine code
The generated code is sent to the CPU and executed
If an error is raised, the interpreter consults the in-memory representation of the source code to produce an error message
Clearly, there is a flaw in my mental model.
What I want to know:
Why does the Python interpreter consult the file on disk to generate error message, rather than looking in memory?
Is there some other flaw in my understanding of what the interpreter is doing?
As per the answer linked by #b_c,
Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback.
[...]
When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on
[...]
The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it.
I wanted to use this library to scrape data from otodom. I've read the docs however I'm stuck on a very basic level, trying to do basic import. The code I use is taken directly from the docs. On top of that I've tried to launch example.py from Github but I'm getting the same error.
Here's the excerpt from the code:
import otodom, os, logging
from otodom.category import get_category
from otodom.offer import get_offer_information
Error message I get:
Traceback (most recent call last):
File ".\otodom_import.py", line 2, in
from otodom.category import get_category
File "C:\Users\Dom\Anaconda3\lib\site-packages\otodom\category.py", line 9, in
from otodom.utils import get_response_for_url, get_url
File "C:\Users\Dom\Anaconda3\lib\site-packages\otodom\utils.py", line 14, in
from scrapper_helpers.utils import caching, normalize_text, key_sha1,
get_random_user_agent
File "C:\Users\Dom\Anaconda3\lib\site-packages\scrapper_helpers\utils.py",
line 22, in
MAX_FILENAME_LENGTH = subprocess.check_output("getconf NAME_MAX /",
shell=True).strip()
File "C:\Users\Dom\Anaconda3\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Users\Dom\Anaconda3\lib\subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'getconf NAME_MAX /' returned non-zero
exit status 1.
Thanks in advance!
EDIT: Don't understand downvotes, I realize that this is a niche module but I did my homework. I've read the docs and tried to find help in other places. If there is a chance that as a noobie I'll find at least one person that can help me, why wouldn't I use this opportunity? Even info that module is buggy can help.
getconf, which the dependency scrapper_helpers module is trying to call, is a POSIX command not available on Windows.
You can track it down in C:\Users\Dom\Anaconda3\lib\site-packages\scrapper_helpers\utils.py and replace the line 22 with:
MAX_FILENAME_LENGTH = 255
But then the question is what other error might pop up given that the module obviously expects to run on a POSIX-compatible system - for example, it will attempt to use /var/tmp/scrapper-helpers/ as its temporary cache path which is also not available on Windows (but Python will interpret it as <DRIVE_LETTER>:\var\tmp\scrapper-helpers and attempt to create it).
Alternatively, looking at its code, you might get away with just declaring the following environment variables:
set MAX_FILENAME_LENGTH=255
set CACHE_DIR=%TEMP%\scapper-helpers\
I am having trouble porting a working, Windows Python application to Linux. I am having some problems, because I did not write the code and am just learning Python. I am having trouble fixing the issues that it keeps throwing up. So here is a kind of error that right now I am stuck with
Traceback (most recent call last):
File "alpha_beta", line 237, in <module>
main()
File "alpha_beta", line 185, in main
ABCCmd()
File "alpha_beta.py", line 74, in ABCCmd
File "C:\softs\Python\Lib\shutil.py", line 80, in copy
File "C:\softs\Python\Lib\shutil.py", line 47, in copyfile
IOError: [Errno 13] Permission denied: '/myPath/XFiles.bin.addr_patched
Any pointers on how to fix it will be much appreciated
Edit:
1) What I mean by I am stuck is, the traceback of the error goes to C:\softs\Python\Lib but actually I am currently executing this code in Ubuntu. Why would the traceback reference to windows library
2) Another thing that bothers me is it says there is an IOError.But when I try to add permission for the denied one it gives me a chmod: changing permissions of /myPath/xFiles.bin.addr_patched': Operation not permitted
Edit 2:
I had commented out a module because I thought it wasn't very useful. Since Now I am anyway discussing the porting issues, I thought I can bring up this additional problem as well since I think the issue is the same and the fix should be similar. On including #pdb module in the python code, I get the following error
traceback (most recent call last):
File "alpha_beta", line 6, in <module>
import pdb
File "C:\softs\Python\Lib\pdb.py", line 14, in <module>
File "C:\softs\Python\Lib\pprint.py", line 39, in <module>
ImportError: No module named cStringIO
I looked at the importer_Cache and looks like this
'': None, '/usr/lib/python2.6/plat-linux2': None, '/usr/lib/python2.6/': None
'/usr/lib/pymodules/python2.6/gtk-2.0': None, '/usr/lib/python2.6/lib-tk': None,
'/usr/lib/python2.6/lib-old': <imp.NullImporter object at 0x7f1269048070>, '/usr/
/python2.6/dist-packages/gtk-2.0': None, '/usr/lib/python2.6/dist-packages/PIL': None,
'/usr/local/lib/python2.6/dist-packages': None, '/usr/lib/python2.6/dist-packages':
None
*SOLVED : There was a weird wrapper in the script which was causing the python script to execute from a different location. Still don't understand how though. After modifying my script to just directly execute as python myScript.py it runs fine. Thank you all anyway for the help
The mixture of Windows and Unix style paths in your error messages makes me think that you may have some filenames that are hard coded or manually created using os specific path separators ('\' or '/'). If you can figure out where the gOptions.inputTf and gWorkingTfFile values are assigned you should look into using os.path.join to help you use the correct paths.
from os.path import join
file = join('A', 'B')
# 'A/B' on unix systems
# 'A\B' on windows systems